Commit Graph

89 Commits (87939e6fab4bf40bec7eeac7c6fbcd0a82294f17)

Author SHA1 Message Date
Ross Wightman 87939e6fab Refactor device handling in scripts, distributed init to be less 'cuda' centric. More device args passed through where needed. 2022-09-23 16:08:59 -07:00
Ross Wightman c88947ad3d Add initial Hugging Face Datasets parser impl. 2022-09-23 16:08:19 -07:00
Ross Wightman e069249a2d Add hf hub entries for laion2b clip models, add huggingface_hub dependency, update some setup/reqs, torch >= 1.7 2022-09-16 21:39:05 -07:00
Ross Wightman 9be0c84715 Change set -> dict w/ None keys for dataset split synonym search, so always consistent if more than 1 exists. Fix #1224 2022-07-07 15:33:53 -07:00
Ross Wightman bfc0dccb0e Improve image extension handling, add methods to modify / get defaults. Fix #1335 fix #1274. 2022-07-07 14:23:20 -07:00
Ross Wightman 70d6d2c484 support test_crop_size in data config resolve 2022-07-02 15:17:05 -07:00
Ross Wightman 7cedc8d474 Follow up to #1256, fix interpolation warning in auto_autoaugment as well 2022-06-21 14:56:53 -07:00
Jakub Kaczmarzyk db64393c0d
use `Image.Resampling` namespace for PIL mapping (#1256)
* use `Image.Resampling` namespace for PIL mapping

PIL shows a deprecation warning when accessing resampling constants via the `Image` namespace. The suggested namespace is `Image.Resampling`. This commit updates `_pil_interpolation_to_str` to use the `Image.Resampling` namespace.

```
/tmp/ipykernel_11959/698124036.py:2: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
  Image.NEAREST: 'nearest',
/tmp/ipykernel_11959/698124036.py:3: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  Image.BILINEAR: 'bilinear',
/tmp/ipykernel_11959/698124036.py:4: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
  Image.BICUBIC: 'bicubic',
/tmp/ipykernel_11959/698124036.py:5: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
  Image.BOX: 'box',
/tmp/ipykernel_11959/698124036.py:6: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
  Image.HAMMING: 'hamming',
/tmp/ipykernel_11959/698124036.py:7: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
  Image.LANCZOS: 'lanczos',
```

* use new pillow resampling enum only if it exists
2022-06-12 22:30:57 -07:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 95cfc9b3e8 Merge remote-tracking branch 'origin/master' into norm_norm_norm 2022-01-25 22:20:45 -08:00
Ross Wightman 656757d26b Fix MobileNetV2 head conv size for multiplier < 1.0. Add some missing modification copyrights, fix starting date of some old ones. 2022-01-14 16:28:27 -08:00
Ross Wightman 02ae11e526 Leaving repeat aug sampler indices as tensor thrashes worker shared process memory 2022-01-06 22:33:09 -08:00
Ross Wightman de5fa791c6 Merge branch 'master' into norm_norm_norm 2022-01-03 11:37:00 -08:00
Hyeongchan Kim a0b2657497
Use `torch.repeat_interleave()` to generate repeated indices faster (#1058)
* update: use numpy to generate repeated indices faster

* update: use torch.repeat_interleave() instead of np.repeat()

* refactor: remove unused import, numpy

* refactor: torch.range to torch.arange

* update: tensor to list before appending the extra samples

* update: concatenate the paddings with torch.cat
2022-01-02 14:01:06 -08:00
Ross Wightman 55adfbeb8d Add commented code to increase open file limit via Python (for TFDS dataset building) 2021-11-30 17:02:27 -08:00
Ross Wightman 78b36bf46c Places365 doesn't exist in some still used torchvision version 2021-11-18 14:59:51 -08:00
Ross Wightman cfa414cad2 Matching two bits_and_tpu changes for TFDs wrapper
* change 'samples' -> 'examples' for tfds wrapper to match tfds naming
* add class_to_idx for image classification datasets in tfds wrapper
2021-11-14 12:52:19 -08:00
Ross Wightman 9ec3210c2d More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently). 2021-11-10 15:52:09 -08:00
Ross Wightman ba65dfe2c6 Dataset work
* support some torchvision datasets
* improvements to TFDS wrapper for subsplit handling (fix #942), shuffle seed
* add class-map support to train (fix #957)
2021-11-09 22:34:15 -08:00
Ross Wightman a41de1f666 Add interpolation mode handling to transforms. Removes InterpolationMode warning. Works for torchvision versions w/ and w/o InterpolationMode enum. Fix #738. 2021-10-28 17:35:01 -07:00
Ross Wightman ed41d32637 Add repr to auto_augment and random_erasing impl 2021-10-28 17:33:36 -07:00
masafumi 047a5ec05f Fix bugs that Mixup does not work device=cpu 2021-10-12 23:51:46 +09:00
Ross Wightman 80075b0b8a Add worker_seeding arg to allow selecting old vs updated data loader worker seed for (old) experiment repeatability 2021-09-28 16:37:45 -07:00
Ross Wightman f8a63a3b71 Add worker_init_fn to loader for numpy seed per worker 2021-09-23 15:44:38 -07:00
Ross Wightman fb94350896 Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss 2021-09-01 17:46:40 -07:00
Ross Wightman f262137ff2 Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments. 2021-09-01 17:40:53 -07:00
Ross Wightman 3cdaf5ed56 Add `mmax` config key to auto_augment for increasing upper bound of RandAugment magnitude beyond 10. Make AugMix uniform sampling default not override config setting. 2021-08-12 15:39:05 -07:00
Ross Wightman d53e91218e Fix tf.data options setting for newer TF versions 2021-05-15 22:56:12 -07:00
Norman Mu 79640fcc1f Enable uniform augmentation magnitude sampling and set AugMix default 2021-04-19 14:21:12 -07:00
Ross Wightman a5310a3451 Merge remote-tracking branch 'origin/benchmark-fixes-vit_hybrids' into pit_and_vit_update 2021-04-01 12:15:34 -07:00
Ross Wightman f42f1df26c Improve evenness of per-worker split for validation set with TFDS 2021-03-18 23:16:14 -07:00
Ross Wightman d584e7f617 Support for huggingface hub via create_model and default_cfgs.
* improve consistency of model creation helper fns
* add comments to some of the model helpers
* support passing external default_cfgs so they can be sourced from hub
2021-03-16 22:48:26 -07:00
Ross Wightman 2db2d87ff7 Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only. 2021-02-23 17:31:42 -08:00
Ross Wightman 68a4144882 Add new weights for ecaresnet26t/50t/269d models. Remove distinction between 't' and 'tn' (tiered models), tn is now t. Add test time img size spec to default cfg. 2021-02-06 16:30:02 -08:00
Ross Wightman 22748f1a2d Convert samples/targets in ParserImageInTar to numpy arrays, slightly less mem usage for massive datasets. Add a few more se/eca model defs to resnet.py 2021-01-22 16:54:33 -08:00
Ross Wightman 5d4c3d0af3 Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup. 2021-01-22 10:52:04 -08:00
Ross Wightman 855d6cc217 More dataset work including factories and a tensorflow datasets (TFDS) wrapper
* Add parser/dataset factory methods for more flexible dataset & parser creation
* Add dataset parser that wraps TFDS image classification datasets
* Tweak num_classes handling bug for 21k models
* Add initial deit models so they can be benchmarked in next csv results runs
2021-01-15 17:26:20 -08:00
Ross Wightman 59ec7e6a53 Merge branch 'master' into imagenet21k_datasets_more 2021-01-04 12:11:05 -08:00
Ross Wightman e7a9ddf982
Merge pull request #334 from kecsap/links
Follow symbolic links during dataset scanning
2021-01-04 10:30:58 -08:00
Csaba Kertesz 7cae7e7035 Follow links during dataset scanning 2021-01-04 00:16:45 +02:00
Ross Wightman 0167f749d3 Remove some old __future__ imports 2021-01-03 11:24:16 -08:00
Ross Wightman e35e9760a6 More work on dataset / parser split and imagenet21k (tar) support 2020-12-28 16:59:15 -08:00
Ross Wightman de6046e213 Initial commit for dataset / parser reorg to support additional datasets / types 2020-12-28 16:59:15 -08:00
Tymoteusz Wiśniewski de15b43865 Fix a bug with accuracy retrieving from RealLabels 2020-12-04 16:12:50 +01:00
Ross Wightman e8ca45854c More models in sotabench, more control over sotabench run, dataset filename extraction consistency 2020-09-24 15:56:57 -07:00
Ross Wightman 47a7b3b5b1 More flexible mixup mode, add 'half' mode. 2020-09-07 20:03:06 -07:00
Ross Wightman 8c9814e3f5 Final cleanup of mixup/cutmix. Element/batch modes working with both collate (prefetcher active) and without prefetcher. 2020-08-12 17:01:32 -07:00
Ross Wightman cd23f55397 Fix mixed prec issues with new mixup code 2020-08-11 12:17:43 -07:00
Ross Wightman f471c17c9d More cutmix/mixup overhaul, ready to kick-off some trials. 2020-08-11 00:10:33 -07:00
Ross Wightman 92f2d0d65d Merge branch 'master' into cutmix. Fixup a few issues. 2020-08-07 15:59:52 -07:00