Commit Graph

15 Commits (c5e0d1c700de2e39441af9b93f745aadf34be878)

Author SHA1 Message Date
Ross Wightman c5e0d1c700 Add dilation support to convnext, allows output_stride=8 and 16 use. Fix #1341 2022-07-19 17:52:10 -07:00
Ross Wightman 06307b8b41 Remove experimental downsample in block support in ConvNeXt. Experiment further before keeping it in. 2022-07-07 14:37:58 -07:00
Ross Wightman 188c194b0f Left some experiment stem code in convnext by mistake 2022-07-02 15:17:28 -07:00
Ross Wightman 6064d16a2d Add initial EdgeNeXt import. Significant cleanup / reorg (like ConvNeXt). Fix #1320
* edgenext refactored for torchscript compat, stage base organization
* slight refactor of ConvNeXt to match some EdgeNeXt additions
* remove use of funky LayerNorm layer in ConvNeXt and just use nn.LayerNorm and LayerNorm2d (permute)
2022-07-01 15:18:42 -07:00
SeeFun 8f0bc0591e fix convnext args 2022-04-05 20:00:57 +08:00
SeeFun ec4e9aa5a0
Add ConvNeXt tiny and small pretrain in22k
Add ConvNeXt tiny and small pretrain  in22k from ConvNeXt  repo:
06f7b05f92
2022-03-24 15:18:08 +08:00
Ross Wightman 474ac906a2 Add 'head norm first' convnext_tiny_hnf weights 2022-03-23 16:06:00 -07:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 5f81d4de23 Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details... 2022-01-26 22:53:57 -08:00
Ross Wightman 738a9cd635 unbiased=False for torch.var_mean path of ConvNeXt LN. Fix #1090 2022-01-17 09:25:06 -08:00
Ross Wightman e0c4eec4b6 Default conv_mlp to False across the board for ConvNeXt, causing issues on more setups than it's improving right now... 2022-01-16 14:20:08 -08:00
Ross Wightman b669f4a588 Add ConvNeXt 22k->1k fine-tuned and 384 22k-1k fine-tuned weights after testing 2022-01-15 15:44:36 -08:00
Ross Wightman edd3d73695 Add missing dropout for head reset in ConvNeXt default head 2022-01-14 16:28:27 -08:00
Ross Wightman b093dcb46d Some convnext cleanup, remove in place mul_ for gamma, breaking symbolic trace, cleanup head a bit... 2022-01-14 16:28:27 -08:00
Ross Wightman 18934debc5 Add initial ConvNeXt impl (mods of official code) 2022-01-14 16:28:27 -08:00