Commit Graph

21 Commits (4495e00016cb18b35011bf99da1beb4eb639186b)

Author SHA1 Message Date
imyhxy 4495e00016
Fix indentation in `log_training_progress()` (#4126) 2021-07-23 14:55:00 +02:00
Ayush Chaurasia 0cc7c58787
W&B: fix refactor bugs (#4069) 2021-07-19 13:57:13 +02:00
Glenn Jocher f7d8562060
`val.py` refactor (#4053)
* val.py refactor

* cleanup

* cleanup

* cleanup

* cleanup

* save after eval

* opt.imgsz bug fix

* wandb refactor

* dataloader to train_loader

* capitalize global variables

* runs/hub/exp to runs/detect/exp

* refactor wandb logging

* Refactor wandb operations (#4061)

Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
2021-07-19 10:43:01 +02:00
Ayush Chaurasia 62409eea08
W&B sweeps support (#3938)
* Add support for W&B Sweeps

* Update and reformat

* Update search space

* reformat

* reformat sweep.py

* Update sweep.py

* Move sweeps files to wandb dir

* Remove print

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-07-14 19:13:02 +02:00
Ayush Chaurasia ffb6e11050
W&B: Update Tables API and comply with new dataset_check (#3772)
* Update tables API and windows path fix

* update dataset check
2021-06-25 20:45:33 +02:00
Ayush Chaurasia 75c0ff43af
[x]W&B: Don't resume transfer learning runs (#3604)
* Allow config cahnge

* Allow val change in wandb config

* Don't resume transfer learning runs

* Add entity in log dataset
2021-06-21 14:00:25 +02:00
Glenn Jocher fad27c0046
Update DDP for `torch.distributed.run` with `gloo` backend (#3680)
* Update DDP for `torch.distributed.run`

* Add LOCAL_RANK

* remove opt.local_rank

* backend="gloo|nccl"

* print

* print

* debug

* debug

* os.getenv

* gloo

* gloo

* gloo

* cleanup

* fix getenv

* cleanup

* cleanup destroy

* try nccl

* return opt

* add --local_rank

* add timeout

* add init_method

* gloo

* move destroy

* move destroy

* move print(opt) under if RANK

* destroy only RANK 0

* move destroy inside train()

* restore destroy outside train()

* update print(opt)

* cleanup

* nccl

* gloo with 60 second timeout

* update namespace printing
2021-06-19 16:30:25 +02:00
Ayush Chaurasia daab682b06
Suppress wandb images size mismatch warning (#3611)
* supress wandb images size mismatch warning

* supress wandb images size mismatch warning

* PEP8 reformat and optimize imports

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-06-14 18:54:58 +02:00
Ayush Chaurasia 53d4fc2e26
W&B: Allow changed in config variable #3588 2021-06-11 23:18:57 +02:00
Charles Frye 19100ba007
Improves docs and handling of entities and resuming by WandbLogger (#3264)
* adds latest tag to match wandb defaults

* adds entity handling, 'last' tag

* fixes bug causing finished runs to resume

* removes redundant "last" tag for wandb artifact
2021-05-21 23:42:53 +02:00
Ayush Chaurasia d2a17289c9
Explicitly convert artifact path to posix_path (#3067)
* Explicitly convert artifact path to posix_path

* Remove redudant str() casting
2021-05-07 19:53:56 +02:00
Ayush Chaurasia b18ca31ff5
use check_file for --data (#3035) 2021-05-05 14:28:19 +02:00
Ayush Chaurasia 801b469878
Improve performance of dataset Logger (#2943)
* Improve performance of Dataset Logger

* Fix scaling bug
2021-04-30 21:54:26 +02:00
Glenn Jocher f7bc685c2c
Implement yaml.safe_load() (#2876)
* Implement yaml.safe_load()

* yaml.safe_dump()
2021-04-21 14:34:45 +02:00
Glenn Jocher 0cae7576a9
utils/wandb_logging PEP8 reformat (#2755)
* wandb_logging PEP8 reformat

* Update wandb_utils.py
2021-04-10 21:09:23 +02:00
Ayush Chaurasia 3067429307
Add support for list-of-directory data format for wandb (#2719) 2021-04-06 16:57:13 +02:00
Ayush Chaurasia 514ebcdf33
Fix: #2674 (#2683)
* Set resume flag to false

* Check existance of val dataset
2021-04-02 11:54:50 +02:00
Ayush Chaurasia 518c09578e
W&B resume ddp from run link fix (#2579)
* W&B resume ddp from run link fix

* Native DDP W&B support for training, resuming
2021-03-28 16:11:36 +02:00
Ayush Chaurasia 1bf9365280
W&B DDP fix (#2574) 2021-03-23 16:54:34 +01:00
Ayush Chaurasia e8fc97aa38
Improved W&B integration (#2125)
* Init Commit

* new wandb integration

* Update

* Use data_dict in test

* Updates

* Update: scope of log_img

* Update: scope of log_img

* Update

* Update: Fix logging conditions

* Add tqdm bar, support for .txt dataset format

* Improve Result table Logger

* Init Commit

* new wandb integration

* Update

* Use data_dict in test

* Updates

* Update: scope of log_img

* Update: scope of log_img

* Update

* Update: Fix logging conditions

* Add tqdm bar, support for .txt dataset format

* Improve Result table Logger

* Add dataset creation in training script

* Change scope: self.wandb_run

* Add wandb-artifact:// natively

you can now use --resume with wandb run links

* Add suuport for logging dataset while training

* Cleanup

* Fix: Merge conflict

* Fix: CI tests

* Automatically use wandb config

* Fix: Resume

* Fix: CI

* Enhance: Using val_table

* More resume enhancement

* FIX : CI

* Add alias

* Get useful opt config data

* train.py cleanup

* Cleanup train.py

* more cleanup

* Cleanup| CI fix

* Reformat using PEP8

* FIX:CI

* rebase

* remove uneccesary changes

* remove uneccesary changes

* remove uneccesary changes

* remove unecessary chage from test.py

* FIX: resume from local checkpoint

* FIX:resume

* FIX:resume

* Reformat

* Performance improvement

* Fix local resume

* Fix local resume

* FIX:CI

* Fix: CI

* Imporve image logging

* (:(:Redo CI tests:):)

* Remember epochs when resuming

* Remember epochs when resuming

* Update DDP location

Potential fix for #2405

* PEP8 reformat

* 0.25 confidence threshold

* reset train.py plots syntax to previous

* reset epochs completed syntax to previous

* reset space to previous

* remove brackets

* reset comment to previous

* Update: is_coco check, remove unused code

* Remove redundant print statement

* Remove wandb imports

* remove dsviz logger from test.py

* Remove redundant change from test.py

* remove redundant changes from train.py

* reformat and improvements

* Fix typo

* Add tqdm tqdm progress when scanning files, naming improvements

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-03-23 00:44:50 +01:00
Ayush Chaurasia 73a0669930
Start setup for improved W&B integration (#1948)
* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Add helper functions for wandb and artifacts

* cleanup

* Reorganize files

* Update wandb_utils.py

* Update log_dataset.py

We can remove this code, as the giou hyp has been deprecated for a while now.

* Reorganize and update dataloader call

* yaml.SafeLoader

* PEP8 reformat

* remove redundant checks

* Update util files

* Update wandb_utils.py

* Remove word size

* Change path of labels.zip

* remove unused imports

* remove --rect

* log_dataset.py cleanup

* log_dataset.py cleanup2

* wandb_utils.py cleanup

* remove redundant id_count

* wandb_utils.py cleanup2

* rename cls

* use pathlib for zip

* rename dataloader to dataset

* Change import order

* Remove redundant code

* remove unused import

* remove unused imports

Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2021-02-01 21:38:41 -08:00