0639d9a591
Fix updated validation_batch_size fallback
2021-09-02 14:44:53 -07:00
5db057dca0
Fix misnamed arg, tweak other train script args for better defaults.
2021-09-02 14:15:49 -07:00
fb94350896
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
2021-09-01 17:46:40 -07:00
7c19c35d9f
Global instead of local rank.
2021-06-09 19:11:58 +02:00
e15e68d881
Fix #566 , summary.csv writing to pwd on local_rank != 0. Tweak benchmark mem handling to see if it reduces likelihood of 'bad' exceptions on OOM.
2021-04-15 23:03:56 -07:00
e685618f45
Merge pull request #550 from amaarora/wandb
...
Wandb Support
2021-04-15 09:26:35 -07:00
7c97e66f7c
Remove commented code, add more consistent seed fn
2021-04-12 09:51:36 -07:00
5772c55c57
Make wandb optional
2021-04-10 01:34:20 -04:00
f54897cc0b
make wandb not required but rather optional as huggingface_hub
2021-04-10 01:27:23 -04:00
f13f7508a9
Keep changes to minimal and use args.experiment as wandb project name if it exists
2021-04-10 00:50:52 -04:00
f8bb13f640
Default project name to None
2021-04-10 00:44:05 -04:00
3f028ebc0f
import wandb in summary.py
2021-04-08 03:48:51 -04:00
a9e5d9e5ad
log loss as before
2021-04-08 03:41:40 -04:00
624c9b6949
log to wandb only if using using wandb
2021-04-08 03:40:22 -04:00
00c8e0b8bd
Make use of wandb configurable
2021-04-08 03:35:59 -04:00
8e6fb861e4
Add wandb support
2021-04-08 03:22:29 -04:00
37c71a5609
Some further create_optimizer_v2 tweaks, remove some redudnant code, add back safe model str. Benchmark step times per batch.
2021-04-01 22:34:55 -07:00
288682796f
Update benchmark script to add precision arg. Fix some downstream (DeiT) compat issues with latest changes. Bump version to 0.4.7
2021-04-01 16:40:12 -07:00
a5310a3451
Merge remote-tracking branch 'origin/benchmark-fixes-vit_hybrids' into pit_and_vit_update
2021-04-01 12:15:34 -07:00
e2e3290fbf
Add '--experiment' to train args for fixed exp name if desired, 'train' not added to output folder if specified.
2021-03-20 12:02:17 -07:00
d584e7f617
Support for huggingface hub via create_model and default_cfgs.
...
* improve consistency of model creation helper fns
* add comments to some of the model helpers
* support passing external default_cfgs so they can be sourced from hub
2021-03-16 22:48:26 -07:00
2db2d87ff7
Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only.
2021-02-23 17:31:42 -08:00
0e16d4e9fb
Add benchmark.py script, and update optimizer factory to be more friendly to use outside of argparse interface.
2021-02-23 15:38:12 -08:00
01653db104
Missed clip-mode arg for repo train script
2021-02-15 23:27:16 -08:00
4f49b94311
Initial AGC impl. Still testing.
2021-02-15 23:22:44 -08:00
d8e69206be
Merge pull request #419 from rwightman/byob_vgg_models
...
More models, GPU-Efficient Nets, RepVGG, classic VGG, and flexible Byob backbone.
2021-02-10 15:44:09 -08:00
0356e773f5
Default to native PyTorch AMP instead of APEX amp. Too many APEX issues cropping up lately.
2021-02-10 14:31:18 -08:00
5114c214fc
Change the Python interpreter to Python 3.x in the scripts
2021-02-09 21:20:28 +02:00
4203efa36d
Fix #387 so that checkpoint saver works with max history of 1. Add checkpoint-hist arg to train.py.
2021-01-31 20:14:51 -08:00
38d8f67570
Fix potential issue with change to num_classes arg in train/validate.py defaulting to None (rely on model def / default_cfg)
2021-01-25 11:53:34 -08:00
5d4c3d0af3
Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup.
2021-01-22 10:52:04 -08:00
9d5d4b8df6
Fix silly train.py typo during dataset work
2021-01-16 16:32:21 -08:00
855d6cc217
More dataset work including factories and a tensorflow datasets (TFDS) wrapper
...
* Add parser/dataset factory methods for more flexible dataset & parser creation
* Add dataset parser that wraps TFDS image classification datasets
* Tweak num_classes handling bug for 21k models
* Add initial deit models so they can be benchmarked in next csv results runs
2021-01-15 17:26:20 -08:00
fd9061dbf7
Remove debug print from train.py
2021-01-04 12:16:06 -08:00
59ec7e6a53
Merge branch 'master' into imagenet21k_datasets_more
2021-01-04 12:11:05 -08:00
e42b140ade
Add --input-size option to scripts to specify full input dimensions from command-line
2021-01-04 00:25:29 +02:00
231d04e91a
ResNetV2 pre-act and non-preact model, w/ BiT pretrained weights and support for ViT R50 model. Tweaks for in21k num_classes passing. More to do... tests failing.
2020-12-28 16:59:15 -08:00
de6046e213
Initial commit for dataset / parser reorg to support additional datasets / types
2020-12-28 16:59:15 -08:00
2ed8f24715
A few more changes for 0.3.2 maint release. Linear layer change for mobilenetv3 and inception_v3, support no bias for linear wrapper.
2020-11-30 16:19:52 -08:00
460eba7f24
Work around casting issue with combination of native torch AMP and torchscript for Linear layers
2020-11-30 13:30:51 -08:00
27bbc70d71
Add back old ModelEma and rename new one to ModelEmaV2 to avoid compat breaks in dependant code. Shuffle train script, add a few comments, remove DataParallel support, support experimental torchscript training.
2020-11-29 16:22:19 -08:00
9214ca0716
Simplifying EMA...
2020-11-16 12:51:52 -08:00
80078c47bb
Add Adafactor and Adahessian optimizers, cleanup optimizer arg passing, add gradient clipping support.
2020-10-09 17:24:43 -07:00
47a7b3b5b1
More flexible mixup mode, add 'half' mode.
2020-09-07 20:03:06 -07:00
532e3b417d
Reorg of utils into separate modules
2020-09-07 13:58:09 -07:00
751b0bba98
Add global_pool (--gp) arg changes to allow passing 'fast' easily for train/validate to avoid channels_last issue with AdaptiveAvgPool
2020-09-02 16:13:47 -07:00
9c297ec67d
Cleanup Apex vs native AMP scaler state save/load. Cleanup CheckpointSaver a bit.
2020-09-02 15:12:59 -07:00
c2cd1a332e
Improve torch amp support and add channels_last support for train/validate scripts
2020-08-31 17:58:16 -07:00
5f563ca4df
fix save_checkpoint bug with native amp
2020-08-22 11:31:50 +09:00
d98967ed5d
add support for native torch AMP in torch 1.6
2020-08-22 09:44:23 +09:00
8c9814e3f5
Final cleanup of mixup/cutmix. Element/batch modes working with both collate (prefetcher active) and without prefetcher.
2020-08-12 17:01:32 -07:00
f471c17c9d
More cutmix/mixup overhaul, ready to kick-off some trials.
2020-08-11 00:10:33 -07:00
92f2d0d65d
Merge branch 'master' into cutmix. Fixup a few issues.
2020-08-07 15:59:52 -07:00
fa28067704
Add more augmentation arguments, including a no_aug disable flag. Fix #209
2020-08-05 13:16:44 -07:00
7995295968
Merge branch 'logger' into features. Change 'logger' to '_logger'.
2020-07-27 18:00:46 -07:00
1998bd3180
Merge branch 'feature/AB/logger' of https://github.com/antoinebrl/pytorch-image-models into logger
2020-07-27 16:06:01 -07:00
6c17d57a2c
Fix some attributions, add copyrights to some file docstrings
2020-07-27 13:44:56 -07:00
78fa0772cc
Leverage python hierachical logger
...
with this update one can tune the kind of logs generated by timm but
training and inference traces are unchanged
2020-06-09 18:28:48 +01:00
6441e9cc1b
Fix memory_efficient mode for DenseNets. Add AntiAliasing (Blur) support for DenseNets and create one test model. Add lr cycle/mul params to train args.
2020-05-22 16:16:45 -07:00
a7f570c9b7
added MultiEpochsDataLoader
2020-05-05 14:47:02 +03:00
13cf68850b
Remove poorly named metrics from torch imagenet example origins. Use top1/top5 in csv output for consistency with existing validation results files, acc elsewhere. Fixes #111
2020-04-10 14:41:08 -07:00
27b3680d49
Revamp LR noise, move logic to scheduler base. Fixup PlateauLRScheduler and add it as an option.
2020-02-22 16:23:15 -08:00
514b0938c4
Experimenting with per-epoch learning rate noise w/ step scheduler
2020-02-21 11:51:05 -08:00
43225d110c
Unify drop connect vs drop path under 'drop path' name, switch all EfficientNet/MobilenetV3 refs to 'drop_path'. Update factory to handle new drop args.
2020-02-18 14:00:26 -08:00
b3cb5f3275
Working on CutMix impl as per #8 , integrating with Mixup, currently experimenting...
2020-02-16 20:09:11 -08:00
b72013def8
Added commandline argument validation-batch-size-multiplier with default set to 1.
2020-02-04 01:02:27 -08:00
5b7cc16ac9
Add warning about using sync-bn with zero initialized BN layers. Fixes #54
2020-01-31 11:44:24 -08:00
d9a6a9d0af
Merge pull request #74 from rwightman/augmix-jsd
...
AugMix, JSD loss, SplitBatchNorm (Auxiliary BN), and more
2020-01-11 12:04:29 -08:00
3eb4a96eda
Update AugMix, JSD, etc comments and references
2020-01-11 12:02:05 -08:00
7547119891
Add SplitBatchNorm. AugMix, Rand/AutoAugment, Split (Aux) BatchNorm, Jensen-Shannon Divergence, RandomErasing all working together
2020-01-05 19:58:59 -08:00
40fea63ebe
Add checkpoint averaging script. Add headers, shebangs, exec perms to all scripts
2020-01-03 14:57:46 -08:00
4666cc9aed
Add --pin-mem arg to enable dataloader pin_memory (showing more benefit in some scenarios now), also add --torchscript arg to validate.py for testing models with jit.script
2020-01-02 16:22:06 -08:00
232ab7fb12
Working on an implementation of AugMix with JensenShannonDivergence loss that's compatible with my AutoAugment and RandAugment impl
2019-12-20 23:04:11 -08:00
5719b493ad
Missed update dist-bn logic for EMA model
2019-12-19 23:03:04 -08:00
a435ea1327
Change reduce_bn to distribute_bn, add ability to choose between broadcast and reduce (mean). Add crop_pct arg to allow selecting validation crop while training.
2019-12-19 22:56:54 -08:00
3bff2b21dc
Add support for keeping running bn stats the same across distributed training nodes before eval/save
2019-12-05 22:35:40 -08:00
1f39d15f15
Allow float decay epochs arg for training, works out with step lr math
2019-11-22 13:28:58 -08:00
7b83e67f77
Pass drop connect arg through to EfficientNet models
2019-11-22 13:27:43 -08:00
4748c6dff2
Fix non-prefetch variant of Mixup. Fixes #50
2019-11-02 17:42:55 -07:00
187ecbafbe
Add support for loading args from yaml file (and saving them with each experiment)
2019-09-09 17:56:45 -07:00
b750b76f67
More AutoAugment work. Ready to roll...
2019-09-01 16:55:42 -07:00
3d9c8a6489
Add support for new AMP checkpointing support w/ amp.state_dict
2019-08-29 15:19:18 -07:00
fac58f609a
Add RAdam, NovoGrad, Lookahead, and AdamW optimizers, a few ResNet tweaks and scheduler factory tweak.
...
* Add some of the trendy new optimizers. Decent results but not clearly better than the standards.
* Can create a None scheduler for constant LR
* ResNet defaults to zero_init of last BN in residual
* add resnet50d config
2019-08-28 00:14:10 -07:00
66634d2200
Add support to split random erasing blocks into randomly selected number with --recount arg. Fix random selection of aspect ratios.
2019-08-12 16:01:58 -07:00
e7c8a37334
Make min-lr and cooldown-epochs cmdline args, change dash in color_jitter arg for consistency
2019-07-26 09:35:31 -07:00
c6b32cbe73
A number of tweaks to arguments, epoch handling, config
...
* reorganize train args
* allow resolve_data_config to be used with dict args, not just arparse
* stop incrementing epoch before save, more consistent naming vs csv, etc
* update resume and start epoch handling to match above
* stop auto-incrementing epoch in scheduler
2019-06-28 13:49:20 -07:00
b20bb58284
Distributed tweaks
...
* Support PyTorch native DDP as fallback if APEX not present
* Support SyncBN for both APEX and Torch native (if torch >= 1.1)
* EMA model does not appear to need DDP wrapper, no gradients, updated from wrapped model
2019-06-20 22:30:23 -07:00
6fc886acaf
Remove all prints, change most to logging calls, tweak alignment of batch logs, improve setup.py
2019-06-20 17:29:25 -07:00
aa4354f466
Big re-org, working towards making pip/module as 'timm'
2019-06-19 17:20:51 -07:00
7dab6d1ec7
Default to img_size in model default_cfg, defer output folder creation until later in the init sequence
2019-06-10 13:34:42 -07:00
9bcd65181b
Add exponential moving average for model weights + few other additions and cleanup
...
* ModelEma class added to track an EMA set of weights for the model being trained
* EMA handling added to train, validation and clean_checkpoint scripts
* Add multi checkpoint or multi-model validation support to validate.py
* Add syncbn option (APEX) to train script for experimentation
* Cleanup interface of CheckpointSaver while adding ema functionality
2019-06-07 15:39:36 -07:00
e6c14427c0
More appropriate/correct loss name
2019-05-28 21:41:10 -07:00
127487369f
Fix bug for prefetcher
...
Set input and target to Cuda when without using prefetcher.
2019-05-26 15:37:19 +10:00
4d2056722a
Mixup and prefetcher improvements
...
* Do mixup in custom collate fn if prefetcher enabled, reduces performance impact
* Move mixup code to own file
* Add arg to disable prefetcher
* Fix no cuda transfer when prefetcher off
* Random erasing when prefetcher off wasn't changed to match new args, fixed
* Default random erasing to off (prob = 0.) for train
2019-05-18 22:17:51 -07:00
780c0a96a4
Change args for RandomErasing so only one required for pixel/color mode
2019-05-18 12:29:30 -07:00
76539d905e
Some transform/data/loader refactoring, hopefully didn't break things
...
* factor out data related constants to own file
* move data related config helpers to own file
* add a variant of RandomResizeCrop that randomizes interpolation method
* remove old Numpy version of RandomErasing
* cleanup torch version of RandomErasing and use it in either GPU loader batch mode or single image cpu Transform
2019-05-16 22:52:17 -07:00
fee607edf6
Mixup implemention in progress
...
* initial impl w/ label smoothing converging, but needs more testing
2019-05-13 19:05:40 -07:00
8fbd62a169
Exclude batchnorm and bias params from weight_decay by default
2019-04-22 17:33:22 -07:00
bc264269c9
Morph mnasnet impl into a generic mobilenet that covers Mnasnet, MobileNetV1/V2, ChamNet, FBNet, and related
...
* add an alternate RMSprop opt that applies eps like TF
* add bn params for passing through alternates and changing defaults to TF style
2019-04-21 15:54:28 -07:00
e9c7961efc
Fix pooling in mnasnet, more sensible default for AMP opt level
2019-04-17 18:06:37 -07:00
Ross Wightman
SamuelGabriel
Ross Wightman
Aman Arora
Csaba Kertesz
datamining99
Antoine Broyelle
AFLALO, Jonathan Isaac
Andrew Lavin
Zhun Zhong