11 Commits

Author SHA1 Message Date
SeeFun
8f0bc0591e fix convnext args 2022-04-05 20:00:57 +08:00
SeeFun
ec4e9aa5a0
Add ConvNeXt tiny and small pretrain in22k
Add ConvNeXt tiny and small pretrain  in22k from ConvNeXt  repo:
06f7b05f92
2022-03-24 15:18:08 +08:00
Ross Wightman
474ac906a2 Add 'head norm first' convnext_tiny_hnf weights 2022-03-23 16:06:00 -07:00
Ross Wightman
372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman
5f81d4de23 Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details... 2022-01-26 22:53:57 -08:00
Ross Wightman
738a9cd635 unbiased=False for torch.var_mean path of ConvNeXt LN. Fix #1090 2022-01-17 09:25:06 -08:00
Ross Wightman
e0c4eec4b6 Default conv_mlp to False across the board for ConvNeXt, causing issues on more setups than it's improving right now... 2022-01-16 14:20:08 -08:00
Ross Wightman
b669f4a588 Add ConvNeXt 22k->1k fine-tuned and 384 22k-1k fine-tuned weights after testing 2022-01-15 15:44:36 -08:00
Ross Wightman
edd3d73695 Add missing dropout for head reset in ConvNeXt default head 2022-01-14 16:28:27 -08:00
Ross Wightman
b093dcb46d Some convnext cleanup, remove in place mul_ for gamma, breaking symbolic trace, cleanup head a bit... 2022-01-14 16:28:27 -08:00
Ross Wightman
18934debc5 Add initial ConvNeXt impl (mods of official code) 2022-01-14 16:28:27 -08:00