Commit Graph

357 Commits (c8c8da60792e37e2941fc27ee4d0594fcdcee34a)

Author SHA1 Message Date
Ayush Chaurasia 3067429307
Add support for list-of-directory data format for wandb (#2719) 2021-04-06 16:57:13 +02:00
Ayush Chaurasia 514ebcdf33
Fix: #2674 (#2683)
* Set resume flag to false

* Check existance of val dataset
2021-04-02 11:54:50 +02:00
Ayush Chaurasia 2a28ef374b
Set resume flag to false (#2657) 2021-03-31 13:47:54 +02:00
Phat Tran 9c803f2f7e
Add --label-smoothing eps argument to train.py (default 0.0) (#2344)
* Add label smoothing option

* Correct data type

* add_log

* Remove log

* Add log

* Update loss.py

remove comment (too versbose)

Co-authored-by: phattran <phat.tranhoang@cyberlogitec.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-03-29 18:45:46 +02:00
Glenn Jocher 866bc7d640
Speed profiling improvements (#2648)
* Speed profiling improvements

* Update torch_utils.py

deepcopy() required to avoid adding elements to model.

* Update torch_utils.py
2021-03-29 15:19:07 +02:00
zzttqu 1e8ab3f5f2
Add tqdm pbar.close() (#2644)
When using tqdm, sometimes it can't print in one line and roll to next line.
2021-03-29 12:21:25 +02:00
Glenn Jocher 2e95cf3d79
Improve git_describe() fix 1 (#2635)
Add stderr=subprocess.STDOUT to catch error messages.
2021-03-28 17:09:06 +02:00
Ayush Chaurasia 518c09578e
W&B resume ddp from run link fix (#2579)
* W&B resume ddp from run link fix

* Native DDP W&B support for training, resuming
2021-03-28 16:11:36 +02:00
Glenn Jocher 6e8c5b7678
Improve git_describe() (#2633)
Catch 'fatal: not a git repository' returns and return '' instead (observed in GCP Hub checks).
2021-03-28 15:39:31 +02:00
Glenn Jocher 0ff5aeca61
Create date_modified() (#2616)
Updated device selection string with fallback for non-git directories.
```python
def select_device(device='', batch_size=None):
    # device = 'cpu' or '0' or '0,1,2,3'
    s = f'YOLOv5 🚀 {git_describe() or date_modified()} torch {torch.__version__} '  # string
...
```
2021-03-26 12:52:47 +01:00
maxupp 196bf10603
Add '*.mpo' to supported image formats (#2615)
Co-authored-by: Max Uppenkamp <max.uppenkamp@inform-software.com>
2021-03-26 12:45:22 +01:00
Glenn Jocher 77415a42e5
Update git_describe() for remote dir usage (#2606) 2021-03-26 01:44:00 +01:00
Glenn Jocher 3bb414890a
resume.py typo (#2603) 2021-03-25 20:55:20 +01:00
Glenn Jocher d4456e43b2
Update segment2box() comment (#2600) 2021-03-25 15:12:49 +01:00
Glenn Jocher 333ccc5b0f
YOLOv5 PyTorch Hub models >> check_requirements() (#2592)
Improved user-feedback following requirements auto-update.
2021-03-24 17:51:39 +01:00
Glenn Jocher 75feeb797c
YOLOv5 PyTorch Hub models >> check_requirements() (#2591)
Prints 'Please restart runtime or rerun command for update to take effect.' following package auto-install to inform users to restart/rerun.
2021-03-24 16:42:54 +01:00
Glenn Jocher 2b329b0945
Enhanced check_requirements() with auto-install (#2575)
* Update check_requirements() with auto-install

This PR builds on an idea I had to automatically install missing dependencies rather than simply report an error message. 

YOLOv5 should now 1) display all dependency issues and not simply display the first missing dependency, and 2) attempt to install/update each missing/VersionConflict package.

* cleanup

* cleanup 2

* Check requirements.txt file exists

* cleanup 3
2021-03-24 01:05:59 +01:00
Ayush Chaurasia 1bf9365280
W&B DDP fix (#2574) 2021-03-23 16:54:34 +01:00
Ayush Chaurasia e8fc97aa38
Improved W&B integration (#2125)
* Init Commit

* new wandb integration

* Update

* Use data_dict in test

* Updates

* Update: scope of log_img

* Update: scope of log_img

* Update

* Update: Fix logging conditions

* Add tqdm bar, support for .txt dataset format

* Improve Result table Logger

* Init Commit

* new wandb integration

* Update

* Use data_dict in test

* Updates

* Update: scope of log_img

* Update: scope of log_img

* Update

* Update: Fix logging conditions

* Add tqdm bar, support for .txt dataset format

* Improve Result table Logger

* Add dataset creation in training script

* Change scope: self.wandb_run

* Add wandb-artifact:// natively

you can now use --resume with wandb run links

* Add suuport for logging dataset while training

* Cleanup

* Fix: Merge conflict

* Fix: CI tests

* Automatically use wandb config

* Fix: Resume

* Fix: CI

* Enhance: Using val_table

* More resume enhancement

* FIX : CI

* Add alias

* Get useful opt config data

* train.py cleanup

* Cleanup train.py

* more cleanup

* Cleanup| CI fix

* Reformat using PEP8

* FIX:CI

* rebase

* remove uneccesary changes

* remove uneccesary changes

* remove uneccesary changes

* remove unecessary chage from test.py

* FIX: resume from local checkpoint

* FIX:resume

* FIX:resume

* Reformat

* Performance improvement

* Fix local resume

* Fix local resume

* FIX:CI

* Fix: CI

* Imporve image logging

* (:(:Redo CI tests:):)

* Remember epochs when resuming

* Remember epochs when resuming

* Update DDP location

Potential fix for #2405

* PEP8 reformat

* 0.25 confidence threshold

* reset train.py plots syntax to previous

* reset epochs completed syntax to previous

* reset space to previous

* remove brackets

* reset comment to previous

* Update: is_coco check, remove unused code

* Remove redundant print statement

* Remove wandb imports

* remove dsviz logger from test.py

* Remove redundant change from test.py

* remove redundant changes from train.py

* reformat and improvements

* Fix typo

* Add tqdm tqdm progress when scanning files, naming improvements

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-03-23 00:44:50 +01:00
Glenn Jocher 9b11f0c58b
PyTorch Hub models default to CUDA:0 if available (#2472)
* PyTorch Hub models default to CUDA:0 if available

* device as string bug fix
2021-03-14 23:16:17 -07:00
Glenn Jocher 2d41e70e82
Scipy kmeans-robust autoanchor update (#2470)
Fix for https://github.com/ultralytics/yolov5/issues/2394
2021-03-14 21:58:12 -07:00
Yann Defretin 38ff499b26
Update autosplit() with annotated_only option (#2466)
* Be able to create dataset from annotated images only

Add the ability to create a dataset/splits only with images that have an annotation file, i.e a .txt file, associated to it. As we talked about this, the absence of a txt file could mean two things:

* either the image wasn't yet labelled by someone,
* either there is no object to detect.

When it's easy to create small datasets, when you have to create datasets with thousands of images (and more coming), it's hard to track where you at and you don't want to wait to have all of them annotated before starting to train. Which means some images would lack txt files and annotations, resulting in label inconsistency as you say in #2313. By adding the annotated_only argument to the function, people could create, if they want to, datasets/splits only with images that were labelled, for sure.

* Cleanup and update print()

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-03-14 17:11:27 -07:00
Glenn Jocher 08d4918d7f
labels.jpg class names (#2454)
* labels.png class names

* fontsize=10
2021-03-12 22:15:41 -08:00
Glenn Jocher f01f3223d5
Integer printout (#2450)
* Integer printout

* test.py 'Labels'

* Update train.py
2021-03-12 13:35:44 -08:00
Glenn Jocher e8a2b83268
GCP sudo docker userdata.sh (#2393)
* GCP sudo docker

* cleanup
2021-03-07 20:21:49 -08:00
Glenn Jocher 7c2c95732c
AWS wait && echo "All tasks done." (#2391) 2021-03-07 20:18:30 -08:00
Glenn Jocher ba18528b47
bbox_iou() stability and speed improvements (#2385) 2021-03-06 13:07:34 -08:00
Yonghye Kwon cd8ed3521d
image weights compatible faster random index generator v2 for mosaic augmentation (#2383)
image weights compatible faster random index generator v2 for mosaic augmentation
2021-03-05 22:58:26 -08:00
Ryan Avery 2c56ad5436
Confusion matrix background axis swap (#2114) 2021-03-02 16:09:52 -08:00
Glenn Jocher fab5085674
EMA bug fix 2 (#2330)
* EMA bug fix 2

* update
2021-03-01 17:13:34 -08:00
Iden Craven a82dce7faa
Fix labels being missed when image extension appears twice in filename (#2300) 2021-02-25 18:05:38 -08:00
Glenn Jocher ec1d8496ba
Improved model+EMA checkpointing (#2292)
* Enhanced model+EMA checkpointing

* update

* bug fix

* bug fix 2

* always save optimizer

* ema half

* remove model.float()

* model half

* carry ema/model in fp32

* rm model.float()

* both to float always

* cleanup

* cleanup
2021-02-24 18:26:46 -08:00
Glenn Jocher 0070995bd5
Amazon AWS EC2 startup and re-startup scripts (#2282) 2021-02-24 01:43:59 -08:00
Glenn Jocher d5d275b6e9
Amazon AWS EC2 startup and re-startup scripts (#2185)
* Amazon AWS EC2 startup and re-startup scripts

* Create resume.py

* cleanup
2021-02-23 23:10:14 -08:00
xiaowo1996 83dc1b4484
FLOPS min stride 32 (#2276)
Signed-off-by: xiaowo1996 <429740343@qq.com>
2021-02-23 09:38:56 -08:00
Glenn Jocher 6f5d6fcdaa
Robust objectness loss balancing (#2256) 2021-02-20 11:19:01 -08:00
Glenn Jocher c09964c27c
Update inference default to multi_label=False (#2252)
* Update inference default to multi_label=False

* bug fix

* Update plots.py

* Update plots.py
2021-02-19 15:39:09 -08:00
Glenn Jocher 47faf95079 reset head 2021-02-19 15:20:41 -08:00
Glenn Jocher d2e754b67b
Add isdocker() (#2232)
* Add isdocker()

* Update general.py

* Update general.py
2021-02-16 15:58:07 -08:00
Glenn Jocher 26c2e54c8f
Add check_imshow() (#2231)
* Add check_imshow()

* Update general.py

* Update general.py
2021-02-16 13:56:47 -08:00
Glenn Jocher 7b833e37bf
LoadStreams() frame loss bug fix (#2222) 2021-02-15 11:02:20 -08:00
Glenn Jocher bdd88e1ed7
YOLOv5 Segmentation Dataloader Updates (#2188)
* Update C3 module

* Update C3 module

* Update C3 module

* Update C3 module

* update

* update

* update

* update

* update

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update datasets

* update

* update

* update

* update attempt_downlaod()

* merge

* merge

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* parameterize eps

* comments

* gs-multiple

* update

* max_nms implemented

* Create one_cycle() function

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* GitHub API rate limit fix

* update

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* ComputeLoss

* astuple

* epochs

* update

* update

* ComputeLoss()

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* merge

* merge

* merge

* merge

* update

* update

* update

* update

* commit=tag == tags[-1]

* Update cudnn.benchmark

* update

* update

* update

* updates

* updates

* updates

* updates

* updates

* updates

* updates

* update

* update

* update

* update

* update

* mosaic9

* update

* update

* update

* update

* update

* update

* institute cache versioning

* only display on existing cache

* reverse cache exists booleans
2021-02-11 21:22:45 -08:00
NanoCode012 a5d5f9262d
Changed socket port and added timeout (#2176) 2021-02-10 13:01:48 -08:00
Transigent c9bda112ae
Update utils/datasets.py to support .webp files (#2174)
Simply added 'webp' as an image format to the img_formats array so that webp image files can be used as training data.
2021-02-10 00:16:38 -08:00
Glenn Jocher 86897e3663
Update train.py test batch_size (#2148)
* Update train.py

* Update loss.py
2021-02-06 10:29:32 -08:00
train255 e9b3de463a
Unique *.cache filenames fix (#2134)
* fix #2121

* Update test.py

* Update train.py

* Update autoanchor.py

* Update datasets.py

* Update log_dataset.py

* Update datasets.py

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-05 11:12:43 -08:00
Glenn Jocher 4bdc5a397e
LoadImages() pathlib update (#2140) 2021-02-05 09:06:23 -08:00
Ayush Chaurasia 73a0669930
Start setup for improved W&B integration (#1948)
* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Update util files

* Update wandb_utils.py

* Remove word size

* Change path of labels.zip

* remove unused imports

* remove --rect

* log_dataset.py cleanup

* log_dataset.py cleanup2

* wandb_utils.py cleanup

* remove redundant id_count

* wandb_utils.py cleanup2

* rename cls

* use pathlib for zip

* rename dataloader to dataset

* Change import order

* Remove redundant code

* remove unused import

* remove unused imports

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-01 21:38:41 -08:00
Glenn Jocher 9646ca438a
Update plot_study() (#2112) 2021-02-01 13:51:11 -08:00
Glenn Jocher be9edffded
Update test.py --task speed and study (#2099)
* Add --speed benchmark

* test range 256 - 1536

* update

* update

* update

* update
2021-01-31 13:58:49 -08:00