* Apple Metal Performance Shader (MPS) device support
Following https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/
Should work with Apple M1 devices with PyTorch nightly installed with command `--device mps`. Usage examples:
```bash
python train.py --device mps
python detect.py --device mps
python val.py --device mps
```
* Update device strategy to fix MPS issue
* Code refactor for general.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update restapi.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add PyTorch AMP check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* Cleanup
* Cleanup
* Robust for DDP
* Fixes
* Add amp enabled boolean to check_train_batch_size
* Simplify
* space to prefix
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Improve mAP0.5-0.95
Two changes provided
1. Added limit on the maximum number of detections for each image likewise pycocotools
2. Rework process_batch function
Changes #2 solved issue #4251
I also independently encountered the problem described in issue #4251 that the values for the same thresholds do not match when changing the limits in the torch.linspace function.
These changes solve this problem.
Currently during validation yolov5x.pt model the following results were obtained:
from yolov5 validation
Class Images Labels P R mAP@.5 mAP@.5:.95: 100%|██████████| 157/157 [01:07<00:00, 2.33it/s]
all 5000 36335 0.743 0.626 0.682 0.506
from pycocotools
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.505
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.685
These results are very close, although not completely pass the competition issue #2258.
I think it's problem with false positive bboxes matched ignored criteria, but this is not actual for custom datasets and does not require an additional solution.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove line to retain pycocotools results
* Update val.py
* Update val.py
* Remove to device op
* Higher precision int conversion
* Update val.py
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Create docker.yml
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update docker.yml
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update docker.yml
* Update docker.yml
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* Cleanup2
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add `@threaded` decorator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Pin downloads to release version
Fixes a release version to avoid forward-compatibility issues in future releases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Ability to dowlnoad older assets
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* Cleanup2
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* FROM nvcr.io/nvidia/pytorch:22.04-py3
* Update Docker
* Update Docker
* Update Docker
* Update Docker
* Update TRT auto-install
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* Cleanup
* Cleanup cpu
* Cleanup cpu
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* PyTorch Hub `_verbose=False` fix2
* Update downloads.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update hubconf.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update
* Update
* Update
* Update
* Update
* Update
* Update
* Update
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Update restapi.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update restapi.py
* Update restapi.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update restapi.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add redundant weights download mirrors
This PR seeks to improve access worldwide to YOLOv5 weights. Universal access to AI for all is our core value, and we are against any censorship or restriction efforts.
I've uploaded the official YOLOv5 v6.1 weights to a primary backup bucket on GCP and to secondary backup on Google Drive at https://drive.google.com/drive/folders/1EFQTEUeXWSFww0luse2jB9M1QNZQGwNl. Autodownload with try the first two locations (GitHuB release assets and GCP bucket), and point users to the Google Drive folder if the first two fail.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* support nomedia
* support nomedia for validation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update train.py
* Revert no plot evolve
evolve plots do not contain any images
* Revert plot_results
contains no media
* Update wandb_utils.py
* sync-bn cleanup
* Cleanup
* Rename nomedia -> noplots
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update .pre-commit-config.yaml
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update .pre-commit-config.yaml
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update CONTRIBUTING.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update README.md
* Update README.md
* Update README.md
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Added a way to skip dependency auto-installation.
Setting the environment variable `YOLOv5_AUTOINSTALL=False` will
skip installing any missing dependencies as if the user had passed
`install=False` to `check_requirements`.
* Cleanup
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update `git_describe()`
Add .git path check to avoid `fatal: not a git repository (or any of the parent directories): .git` printout
* Update general.py
* add version warning
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update __init__.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Print dataset scan only `if RANK in (-1, 0)`
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Simplify callbacks.py return
* Indent args (pytorch convention)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Copy wandb param dict before training to avoid overwrites.
Copy the hyperparameter dict retrieved from wandb configuration before passing it to `train()`. Training overwrites parameters in the dictionary (eg scaling obj/box/cls gains), which causes the values reported in wandb to not match the input values. This is confusing as it makes it hard to reproduce a run, and also throws off wandb's Bayesian sweep algorithm.
* Cleanup
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* fix: disable usage of root logger
`logging.basicConfig` configures Python's root logger. This prohibits
fine control of logging, overwrites logging configuration done outside
the package, and is not best practice. Instead, the used logger is now
configured directly, and the root logger is untouched.
Example:
If yolov5 is used as part of another project with some sophisticated
logging, the internal `logging.basicConfig` call overwrites all the
external configuration.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update general.py
* Update general.py
* Comment kaggle
* Uncomment kaggle
Co-authored-by: Maximilian Strobel <Maximilian.Strobel@infineon.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit suggestions
updates:
- [github.com/asottile/pyupgrade: v2.31.0 → v2.31.1](https://github.com/asottile/pyupgrade/compare/v2.31.0...v2.31.1)
- [github.com/pre-commit/mirrors-yapf: v0.31.0 → v0.32.0](https://github.com/pre-commit/mirrors-yapf/compare/v0.31.0...v0.32.0)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update yolo.py
* Update activations.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update activations.py
* Update tf.py
* Update tf.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update restapi.py
* Update restapi.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* precommit: yapf
* align isort
* fix
# Conflicts:
# utils/plots.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update setup.cfg
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update setup.cfg
* Update setup.cfg
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update wandb_utils.py
* Update augmentations.py
* Update setup.cfg
* Update yolo.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update val.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* simplify colorstr
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* val run fix
* export.py last comma
* Update export.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update hubconf.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* PyTorch Hub tuple fix
* PyTorch Hub tuple fix2
* PyTorch Hub tuple fix3
* Update setup
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* 🎉🆕 now can read Chinese image path.
use "cv2.imdecode(np.fromfile(f, np.uint8), cv2.IMREAD_COLOR)" instead of "cv2.imread(f)" for Chinese image path.
* Update datasets.py
* Update __init__.py
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Stems not working correctly for YOLOv5l with current .rstrip() implementation. After fix:
```
YOLOv5l summary: 468 layers, 46563709 parameters, 46563709 gradients, 109.3 GFLOPs
```
* Update loss.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update loss.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: add default font as fallback
Add default font as fallback if the downloading of the Arial.ttf font
fails for some reason, e.g. no access to public internet.
* Update plots.py
Co-authored-by: Maximilian Strobel <Maximilian.Strobel@infineon.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update AutoAnchor
* Update AutoAnchor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Update plots for Chinese fonts
* make is_chinese() non-str safe
* Add global FONT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update general.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* New flag 'stop_training' in util.callbacks.Callbacks class to prematurely stop training from callback handler
* Removed most of the new checks, leaving only the one after calling 'on_train_batch_end'
* Cleanup
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* log best result in summary
* comment added
* add space for `flake8`
* log `best/epoch`
* fix `dimension` for epoch
ValueError: all the input arrays must have same number of dimensions
* log `best/` in `utils.logger.__init__`
* fix pre-commit
1. missing whitespace around operator
2. over-indented
Consider that the default value is CIOU,adjust the order of judgment could reduce the number of judgments.
And “elif CIoU:” didn't need 'if'.
Co-authored-by: 李杰 <360751194@qq.comqq.com>
* Check TensorRT>=8.0.0 version
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Making `ar_thr` available as a hyperparameter
* Disabling ar_thr as hyperparameter and computing from the dataset instead
* Fixing bug in ar_thr computation
* Fix `ar_thr` to 100
* Find out a bug. When set batch_size = -1 to use the autobatch.
reproduce:
* Fix type conflict
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
When OpenCV retrieving image fail, original code would modify source images **inplace**, which may result in plotting bounding boxes on a black image. That is, before inference, source image `im0s[i]` is OK, but after inference before `Process predictions`, `im0s[i]` may have been changed.
* Remove NCOLS from tqdm
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* reformat
* Single-line argparser argument
* Update README.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update README.md
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Add feature map to save npy files
Add feature map to save npy files,export npy files with 32 feature maps per layer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update plots.py
* Update plots.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update plots.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* prune unused imports
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* handle exceptions| attempt CI
* update
* Pre-commit manual run
* yaml one-liner
* Update ci-testing.yml
* Comment W&B CI
Leave as example for future separate CI
* Update ci-testing.yml
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* AutoBatch, AutoAnchor `LOGGER`
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update autoanchor.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Improve plots.py robustness
Addresses issues #5374, #5395, #5611
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix to resource warning allocation; utilize file.open within a context manager
* rename fh to f
in keeping with naming convention
Co-authored-by: Ayman Saleh <aymansaleh@Aymans-MacBook-Pro-2.local>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update __init__.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* notebook_init
* notebook_init
* notebook_init
* notebook_init
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* notebook_init
* Created using Colaboratory
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Logger consolidation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix `save_one_box()`
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Resolves https://github.com/ultralytics/yolov5/pull/5341#issuecomment-961774729
Tests:
```python
import glob
import re
from pathlib import Path
def increment_path(path, exist_ok=False, sep='', mkdir=False):
# Increment file or directory path, i.e. runs/exp --> runs/exp{sep}2, runs/exp{sep}3, ... etc.
path = Path(path) # os-agnostic
if path.exists() and not exist_ok:
path, suffix = (path.with_suffix(''), path.suffix) if path.is_file() else (path, '')
dirs = glob.glob(f"{path}{sep}*") # similar paths
matches = [re.search(rf"%s{sep}(\d+)" % path.stem, d) for d in dirs]
i = [int(m.groups()[0]) for m in matches if m] # indices
n = max(i) + 1 if i else 2 # increment number
path = Path(f"{path}{sep}{n}{suffix}") # increment path
if mkdir:
path.mkdir(parents=True, exist_ok=True) # make directory
return path
print(increment_path('runs'))
print(increment_path('export.py'))
print(increment_path('abc.def.dir'))
print(increment_path('abc.def.file'))
```
* precommit: isort
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update isort config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update name
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Meshgrid `indexing='ij'` for PyTorch 1.10
Will not merge currently as breaks backwards compatibility.
* Meshgrid `indexing='ij'` for PyTorch 1.10
Will not merge currently as breaks backwards compatibility.
* Add check_version hard argument
* Update comment
* take EXIF orientation tags into account when fixing corrupt images
* fit 120 char
* sort imports
* Update local exif_transpose comment
We have a local inplace version that is faster than the official as the image is not copied. AutoShape() uses this for Hub models, but here it is not important as the datasets.py usage is infrequent (AutoShape() it is applied every image).
* Update datasets.py
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* take image files with uppercase extensions into account in autosplit
* case fix
* Refactor implementation
Removes additional variable (capital variable names are also only for global variables), and uses the same methodology as implemented earlier in datasets.py L409.
* Remove redundant rglob characters
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Autofix duplicate labels
PR changes duplicate label handling from report error and ignore image-label pair to report warning and autofix image-label pair.
This should fix this common issue for users and allow everyone to get started and get a model trained faster and easier than before.
* sign fix
* Cleanup
* Increment cache version
* all to any fix
* Add train class filter feature to datasets.py
Allows for training on a subset of total classes if `include_class` list is defined on datasets.py L448:
```python
include_class = [] # filter labels to include only these classes (optional)
```
* segments fix
* added callbacks
* added back callback to main
* added save_dir to callback output
* merged in upstream
* removed ghost code
* fixed parsing error for google temp links
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
`check_file()` is now limited to searching opt-in directories: /data, /models, /utils. This prevents large non-project directories like /.git and /venv from being searched, which may cause `check_file()` to slow significantly.