664 Commits

Author SHA1 Message Date
Jihoon Kim
fcb225c1c8
Shuffle all 4(or 9) images in mosaic augmentation (#4787)
Thank you for sharing nice open-source codes 👍

I applied to shuffle the order of all 4(or 9) images in mosaic augmentation

Currently, the order of images in mosaic augmentation is not completely random.
The remaining images except the first are randomly arranged. Apply shuffle all to increase the diversity of data composition.
2021-09-15 00:57:06 +02:00
Glenn Jocher
c3a93d783d
Add TensorFlow formats to export.py (#4479)
* Initial commit

* Remove unused export_torchscript return

* ROOT variable

* Add prefix to fcn arg

* fix ROOT

* check_yaml into run()

* interim fixes

* imgsz=(320, 320)

* Hardcode tf_raw_resize False

* Finish opt elimination

* Update representative_dataset_gen()

* Update export.py with TF methods

* SiLU and GraphDef fixes

* file_size() directory handling feature

* export fixes

* add lambda: to representative_dataset

* Detect training False default

* Fuse false for TF models

* Embed agnostic NMS arguments

* Remove lambda

* TensorFlow.js export success

* Add pb to Usage

* Add *_tfjs_model/ to ignore files

* prepend YOLOv5 to function headers

* Remove end --- comments

* parameterize tfjs export pb file

* update run() data default /ROOT

* update --include help

* update imports

* return ct_model

* Consolidate TFLite export

* pb prerequisite to tfjs

* TF modules CamelCase

* Remove exports from tf.py and cleanup

* pass agnostic NMS arguments

* CI

* CI

* ignore *_web_model/

* Add tensorflow to CI dependencies

* CI tensorflow-cpu

* Update requirements.txt

* Remove tensorflow check_requirement

* CI coreml tfjs

* export only onnx torchscript

* reorder exports torchscript first
2021-09-12 15:52:24 +02:00
Glenn Jocher
c47be26f34
Replace path.absolute() with path.resolve() (#4763) 2021-09-11 22:46:33 +02:00
Glenn Jocher
cd810c8286
Centralize user_config_dir() decision making (#4755) 2021-09-11 16:32:08 +02:00
Glenn Jocher
22ee6fb7c1
Update is_writeable() for 2 methods (#4744)
* Writeable test

* Fix

* Cleanup
2021-09-10 17:52:33 +02:00
Glenn Jocher
a144536f88
Fix is_writeable() for 3 OS support (#4743)
* Fix `is_writeable()` for 3 OS support

* Update general.py
2021-09-10 17:06:22 +02:00
Josh Veitch-Michaelis
19e28e3bfe
Add callbacks to train function in W&B sweep (#4742)
* add callbacks to train function in wandb sweep

Fix following https://github.com/ultralytics/yolov5/pull/4688 which modified the function signature to `train`

* Cleanup

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-09-10 16:28:52 +02:00
Glenn Jocher
4a025ae97f
Fix user_config_dir() for GCP/AWS functions (#4726)
* Fix `user_config_dir()` for GCP/AWS functions

Compatability fix for GCP functions and AWS lambda for user config directory in https://github.com/ultralytics/yolov5/pull/4628

* Windows skip check
2021-09-09 17:57:46 +02:00
JD Costa
deb434aefa
Make CONFIG_DIR configurable per environment variable (#4727) 2021-09-09 15:29:18 +02:00
Glenn Jocher
8e94bf62d9
Add user_config_dir('Ultralytics') (#4715)
* Add `user_config_dir`

* Linux to .config
2021-09-08 18:13:59 +02:00
Glenn Jocher
25a7e1dae5
Update check_yaml() comment (#4713)
* Update `check_yaml()` comment

* Cleanup
2021-09-08 16:01:03 +02:00
Glenn Jocher
f984cce52a
Fix check_suffix() (#4712)
Fix a bug when `file=''`
2021-09-08 15:06:31 +02:00
Glenn Jocher
a2b3c71636
Add suffix checks (#4711)
* Add suffix checks

* Cleanup

* Cleanup2

* Cleanup3
2021-09-08 14:36:12 +02:00
Zhiqiang Wang
8e5f9ddbdb
Remove redundant ComputeLoss code (#4701) 2021-09-08 12:48:33 +02:00
Kalen Michael
2317f86ca4
Optimised Callback Class to Reduce Code and Fix Errors (#4688)
* added callbacks

* added back callback to main

* added save_dir to callback output

* reduced code count

* updated callbacks

* added default callback class to main, added missing parameters to on_model_save

* Glenn updates

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-09-07 18:32:15 +02:00
Glenn Jocher
548745181a
EarlyStopper updates (#4679) 2021-09-05 19:09:53 +02:00
Glenn Jocher
234e8ae6fa
Fix for plot_evolve() string argument (#4639) 2021-09-01 15:00:13 +02:00
Glenn Jocher
de534e9221
Fix 3 for Arial.ttf redownloads with hub inference (#4629)
Fix 3 for Arial.ttf redownloads with hub inference, follow-on to #4628.
2021-08-31 20:54:10 +02:00
Glenn Jocher
a4e8f78c5e
Fix 2 for Arial.ttf redownloads with hub inference (#4628) 2021-08-31 20:05:17 +02:00
Glenn Jocher
ba0f80874f
Fix for Arial.ttf redownloads with hub inference (#4627) 2021-08-31 15:01:41 +02:00
Glenn Jocher
50a9828679
DDP torch.jit.trace() --sync-bn fix (#4615)
* Remove assert

* debug0

* trace=not opt.sync

* sync to sync_bn fix

* Cleanup
2021-08-30 18:35:07 +02:00
Glenn Jocher
bb5ebc290e
Close matplotlib plots after opening (#4612)
* Close plots

* Replace fig.close() for plt.close()
2021-08-30 17:22:21 +02:00
Ayush Chaurasia
11e7c7b48d
W&B: Don't log models in evolve operation (#4611) 2021-08-30 17:07:20 +02:00
Glenn Jocher
3a72d4a7e3
Update check_git_status() warning (#4610) 2021-08-30 17:05:45 +02:00
Glenn Jocher
e7d1842a98
Auto-download Arial.ttf on init (#4606)
* Auto-download Arial.ttf on init

* Fix ROOT
2021-08-30 14:58:22 +02:00
Glenn Jocher
dc13820c9d
TensorBoard on_train_end() speed improvements (#4605) 2021-08-30 13:01:41 +02:00
Glenn Jocher
b894e69dfc
Update mosaic plots font size (#4596) 2021-08-29 18:05:49 +02:00
Glenn Jocher
dbbc6b5c48
Re-order plots.py to class-first (#4595) 2021-08-29 17:44:51 +02:00
Glenn Jocher
e5e5ebc799
Auto-UTF handling (#4594) 2021-08-29 17:15:18 +02:00
Glenn Jocher
de44376d1b
Create Annotator() class (#4591)
* Add Annotator() class

* Download Arial

* 2x for loop

* Cleanup

* tuple 2 list

* max_size=1920

* bold logging results to

* tolist()

* im = annotator.im

* PIL save in detect.py

* Smart asarray in detect.py

* revert to cv2.imwrite

* Cleanup

* Return result asarray

* Add `Profile()` profiler

* CamelCase Timeout

* Resize after mosaic

* pillow>=8.0.0

* daemon imwrite

* Add cv2 support

* Remove plot_wh_methods and plot_one_box

* pil=False for hubconf.py annotations

* im.shape bug fix

* colorstr common.py

* join daemons

* Update t.daemon

* Removed daemon saving
2021-08-29 16:46:13 +02:00
Takumi Karasawa
7b35971ba5
Fix bug in plot_one_box when label is None (#4588) 2021-08-29 15:08:27 +02:00
Glenn Jocher
bbfafeabdb
Add Profile() profiler (#4587)
* Add `Profile()` profiler

* CamelCase Timeout
2021-08-29 13:49:04 +02:00
Glenn Jocher
93cc015748
Add EarlyStopping feature (#4576)
* Add EarlyStopping feature

* Add comment

* Cleanup

* Cleanup2

* debug

* debug2

* debug3

* debug3

* debug4

* debug5

* debug6

* debug7

* debug8

* debug9

* debug10

* debug11

* debug12

* Cleanup

* Add TODO for known DDP issue
2021-08-28 19:03:52 +02:00
Glenn Jocher
8ac96b7975
Fix for AP calculation limits 0.0 - 1.0 (#4563)
This PR brings alignment in AP computation practices with Detectron2 and MMDetection. 

Problem first noted by @yusiyoh in https://github.com/ultralytics/yolov5/issues/4546
2021-08-27 13:23:29 +02:00
Glenn Jocher
e899d6e8fb
Fix for corrupt JPEGs auto-fix PR (#4560)
Auto-fix corrupt JPEGs PR introduced a bug whereby the f.seek() operation read all of the bytes in the image, resulting in the PIL image having nothing to read upon the .save() operation. 

Fix was to re-open the image using PIL before saving.
2021-08-27 13:01:21 +02:00
Glenn Jocher
11f85e7e71
Auto-fix corrupt JPEGs (#4548)
* Autofix corrupt JPEGs

This PR automatically re-saves corrupt JPEGs and trains with the resaved images. WARNING: this will overwrite the existing corrupt JPEGs in a dataset and replace them with correct JPEGs, though the filesize may increase and the image contents may not be exactly the same due to lossy JPEG compression schemes. Results may vary by JPEG decoder and hardware.

Current behavior is to exclude corrupt JPEGs from training with a warning to the user, but many users have been complaining about large parts of their dataset being excluded from training.

* Clarify re-save reason
2021-08-26 15:51:04 +02:00
Glenn Jocher
7b1643b5b5
Add install=True argument to check_requirements (#4512)
* Add `install=True` argument to `check_requirements`

* Update general.py
2021-08-23 14:38:30 +02:00
Ayush Chaurasia
7316b78e36
W&B: Refactor the wandb_utils.py file (#4496)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Fix

* fix

* refactor constructor

* refactor

* refactor

* refactor

* PyCharm reformat

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-23 13:10:07 +02:00
Huu Quan, CAP
1d65e8194d
Fix missing labels after albumentations (#4455)
* fix missing labels after augmentation

* Update datasets.py

Cleanup

Co-authored-by: Huu Quan <huuquan@HuuQuans-MacBook.local>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-18 12:07:09 +02:00
Jiacong Fang
808bcad3bb
Add TensorFlow and TFLite export (#1127)
* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* Put representative dataset in tfl_int8 block

* detect.py TF inference

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* Add models/tf.py for TensorFlow and TFLite export

* Set auto=False for int8 calibration

* Update requirements.txt for TensorFlow and TFLite export

* Read anchors directly from PyTorch weights

* Add --tf-nms to append NMS in TensorFlow SavedModel and GraphDef export

* Remove check_anchor_order, check_file, set_logging from import

* Reformat code and optimize imports

* Autodownload model and check cfg

* update --source path, img-size to 320, single output

* Adjust representative_dataset

* detect.py TF inference

* Put representative dataset in tfl_int8 block

* weights to string

* weights to string

* cleanup tf.py

* Add --dynamic-batch-size

* Add xywh normalization to reduce calibration error

* Update requirements.txt

TensorFlow 2.3.1 -> 2.4.0 to avoid int8 quantization error

* Fix imports

Move C3 from models.experimental to models.common

* implement C3() and SiLU()

* Fix reshape dim to support dynamic batching

* Add epsilon argument in tf_BN, which is different between TF and PT

* Set stride to None if not using PyTorch, and do not warmup without PyTorch

* Add list support in check_img_size()

* Add list input support in detect.py

* sys.path.append('./') to run from yolov5/

* Add int8 quantization support for TensorFlow 2.5

* Add get_coco128.sh

* Remove --no-tfl-detect in models/tf.py (Use tf-android-tfl-detect branch for EdgeTPU)

* Update requirements.txt

* Replace torch.load() with attempt_load()

* Update requirements.txt

* Add --tf-raw-resize to set half_pixel_centers=False

* Add --agnostic-nms for TF class-agnostic NMS

* Cleanup after merge

* Cleanup2 after merge

* Cleanup3 after merge

* Add tf.py docstring with credit and usage

* pb saved_model and tflite use only one model in detect.py

* Add use cases in docstring of tf.py

* Remove redundant `stride` definition

* Remove keras direct import

* Fix `check_requirements(('tensorflow>=2.4.1',))`

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-17 13:18:16 +02:00
Glenn Jocher
f3e3f7603f
TFLite prep (#4436) 2021-08-16 17:25:06 +02:00
Glenn Jocher
19d03a955c
Remove DDP process group timeout (#4422) 2021-08-15 18:32:41 +02:00
Glenn Jocher
24bea5e4b7
Standardize headers and docstrings (#4417)
* Implement new headers

* Reformat 1

* Reformat 2

* Reformat 3 - math

* Reformat 4 - yaml
2021-08-14 21:17:51 +02:00
Glenn Jocher
2da4e7acf7
Merge PIL and OpenCV in plot_one_box(use_pil=False) (#4416)
* Merge PIL and OpenCV box plotting functions

* Add ASCII check to plot_one_box

* Cleanup

* Cleanup2
2021-08-14 17:44:15 +02:00
Glenn Jocher
63e09fdc48
Remove encoding='ascii' (#4413)
* Remove `encoding='ascii'`

* Reinstate `encoding='ascii'` in emojis()
2021-08-14 13:47:20 +02:00
Glenn Jocher
e086347377
Update plots.py (#4407) 2021-08-14 12:50:27 +02:00
Glenn Jocher
3e7c59ad3b
Fix rename utils.google_utils to utils.downloads (#4393) 2021-08-11 23:40:47 +02:00
Ayush Chaurasia
a4e4553040
W&B: Fix for 4360 (#4388)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Fix

* fix

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-11 21:56:25 +02:00
Ayush Chaurasia
75d323dd8b
W&B: Add advanced features tutorial (#4384)
* Improve docstrings and run names

* default wandb login prompt with timeout

* return key

* Update api_key check logic

* Properly support zipped dataset feature

* update docstring

* Revert tuorial change

* extend changes to log_dataset

* add run name

* bug fix

* bug fix

* Update comment

* fix import check

* remove unused import

* Hardcore .yaml file extension

* reduce code

* Reformat using pycharm

* Remove redundant try catch

* More refactoring and bug fixes

* retry

* Reformat using pycharm

* respect LOGGERS include list

* Initial readme update

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-08-11 18:35:43 +02:00
Sudhanshu Singh
f409d8e54f
Update README.md (#4309)
remove unnecessary "`"
2021-08-04 22:11:48 +02:00