Commit Graph

22 Commits (3e7ab12af9838103aeac9fe1cf28cf43296c5402)

Author SHA1 Message Date
Ross Wightman c719f7eb86 More forward_intermediates() updates
* add convnext, resnet, efficientformer, levit support
* remove kwargs only for fn so that torchscript isn't broken for all :(
* use reset_classifier() consistently in prune
2024-05-03 16:22:32 -07:00
Ross Wightman 67332fce24 Add features_intermediate() support to coatnet, maxvit, swin* models. Refine feature interface. Start prep of new vit weights. 2024-04-30 16:56:33 -07:00
Ross Wightman 4b2565e4cb More forward_intermediates() / FeatureGetterNet work
* include relpos vit
* refactor reduction / size calcs so hybrid vits work and dynamic_img_size works
* fix -ve feature indices when pruning
* fix mvitv2 w/ class token
* refine naming
* add tests
2024-04-10 15:11:34 -07:00
Ross Wightman 679daef76a More forward_intermediates() & features_only work
* forward_intermediates() added to beit, deit, eva, mvitv2, twins, vit, vit_sam
* add features_only to forward intermediates to allow just intermediate features
* fix #2060
* fix #1374
* fix #657
2024-04-09 21:29:16 -07:00
Yassine 884ef88818 fix all SDPA dropouts 2023-10-05 08:58:41 -07:00
Ross Wightman e4e43190ce Add typing to all model entrypoint fns, add old cache check env var to builder 2023-05-08 08:52:38 -07:00
Ross Wightman 3386af8c86 Final push to get remaining models using multi-weight pretrained configs, almost all weights on HF hub 2023-04-26 15:52:13 -07:00
Ross Wightman 965d0a2d36 fast_attn -> fused_attn, implement global config for enable/disable fused_attn, add to more models. vit clip openai 336 weights. 2023-04-10 12:04:33 -07:00
Ross Wightman 4d135421a3 Implement patch dropout for eva / vision_transformer, refactor / improve consistency of dropout args across all vit based models 2023-04-07 20:27:23 -07:00
Ross Wightman 927f031293 Major module / path restructure, timm.models.layers -> timm.layers, add _ prefix to all non model modules in timm.models 2022-12-06 15:00:06 -08:00
Ross Wightman 0862e6ebae Fix correctness of some group matching regex (no impact on result), some formatting, missed forward_head for resnet 2022-03-19 14:58:54 -07:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 5f81d4de23 Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details... 2022-01-26 22:53:57 -08:00
Ross Wightman abc9ba2544 Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks. 2022-01-25 21:54:13 -08:00
Alexander Soare 65d827c7a6 rename notrace registration and standardize trace_utils imports 2021-11-15 21:03:21 +00:00
Alexander Soare b25ff96768 wip - pre-rebase 2021-11-12 20:45:05 +00:00
Alexander Soare bc3d4eb403 wip -rebase 2021-11-12 20:45:05 +00:00
Ross Wightman 8880f696b6 Refactoring, cleanup, improved test coverage.
* Add eca_nfnet_l2 weights, 84.7 @ 384x384
* All 'non-std' (ie transformer / mlp) models have classifier / default_cfg test added
* Fix #694 reset_classifer / num_features / forward_features / num_classes=0 consistency for transformer / mlp models
* Add direct loading of npz to vision transformer (pure transformer so far, hybrid to come)
* Rename vit_deit* to deit_*
* Remove some deprecated vit hybrid model defs
* Clean up classifier flatten for conv classifiers and unusual cases (mobilenetv3/ghostnet)
* Remove explicit model fns for levit conv, just pass in arg
2021-06-12 16:40:02 -07:00
Ross Wightman bfc72f75d3 Expand scope of testing for non-std vision transformer / mlp models. Some related cleanup and create fn cleanup for all vision transformer and mlp models. More CoaT weights. 2021-05-24 21:13:26 -07:00
Ross Wightman a569635045 Update twin weights to a copy in GitHub releases for faster dl. Tweak model class comment. 2021-05-21 16:23:14 -07:00
Ross Wightman be99eef9c1 Remove redundant code, cleanup, fix torchscript. 2021-05-20 23:38:35 -07:00
李鑫杰 00548b8427 Add Twins 2021-05-18 19:21:53 +08:00