Commit Graph

5 Commits (bb2b6b5f09737059ccca004c35f3e2e611d341db)

Author SHA1 Message Date
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 02ae11e526 Leaving repeat aug sampler indices as tensor thrashes worker shared process memory 2022-01-06 22:33:09 -08:00
Hyeongchan Kim a0b2657497
Use `torch.repeat_interleave()` to generate repeated indices faster (#1058)
* update: use numpy to generate repeated indices faster

* update: use torch.repeat_interleave() instead of np.repeat()

* refactor: remove unused import, numpy

* refactor: torch.range to torch.arange

* update: tensor to list before appending the extra samples

* update: concatenate the paddings with torch.cat
2022-01-02 14:01:06 -08:00
Ross Wightman f262137ff2 Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments. 2021-09-01 17:40:53 -07:00
Ross Wightman aa4354f466 Big re-org, working towards making pip/module as 'timm' 2019-06-19 17:20:51 -07:00