Commit Graph

24 Commits (965d0a2d363668b7f8d1794e45c52d525bdb6278)

Author SHA1 Message Date
Ross Wightman 965d0a2d36 fast_attn -> fused_attn, implement global config for enable/disable fused_attn, add to more models. vit clip openai 336 weights. 2023-04-10 12:04:33 -07:00
Ross Wightman 4d135421a3 Implement patch dropout for eva / vision_transformer, refactor / improve consistency of dropout args across all vit based models 2023-04-07 20:27:23 -07:00
Ross Wightman 1bb3989b61 Improve kwarg passthrough for swin, vit, deit, beit, eva 2023-04-05 21:37:16 -07:00
Ross Wightman 572f05096a Swin and FocalNet weights on HF hub. Add model deprecation functionality w/ some registry tweaks. 2023-03-18 14:55:09 -07:00
Ross Wightman acfd85ad68 All swin models support spatial output, add output_fmt to v1/v2 and use ClassifierHead.
* update ClassifierHead to allow different input format
* add output format support to patch embed
* fix some flatten issues for a few conv head models
* add Format enum and helpers for tensor format (layout) choices
2023-03-15 23:21:51 -07:00
Ross Wightman 7d9e321b76 Improve tracing of window attn models with simpler reshape logic 2023-02-17 07:59:06 -08:00
Ross Wightman 927f031293 Major module / path restructure, timm.models.layers -> timm.layers, add _ prefix to all non model modules in timm.models 2022-12-06 15:00:06 -08:00
Ross Wightman e11efa872d Update a bunch of weights with external links to timm release assets. Fixes issue with *aliyuncs.com returning forbidden. Did pickle scan / verify and re-hash. Add TresNet-V2-L weights. 2022-09-13 16:35:26 -07:00
Ross Wightman c5a8e929fb Add initial swinv2 tiny / small weights 2022-04-03 15:22:55 -07:00
Ross Wightman c42be74621 Add attrib / comments about Swin-S3 (AutoFormerV2) weights 2022-03-23 16:07:09 -07:00
Ross Wightman 0862e6ebae Fix correctness of some group matching regex (no impact on result), some formatting, missed forward_head for resnet 2022-03-19 14:58:54 -07:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 5f81d4de23 Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details... 2022-01-26 22:53:57 -08:00
Ross Wightman 95cfc9b3e8 Merge remote-tracking branch 'origin/master' into norm_norm_norm 2022-01-25 22:20:45 -08:00
Ross Wightman abc9ba2544 Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks. 2022-01-25 21:54:13 -08:00
Ross Wightman 656757d26b Fix MobileNetV2 head conv size for multiplier < 1.0. Add some missing modification copyrights, fix starting date of some old ones. 2022-01-14 16:28:27 -08:00
Alexander Soare 65d827c7a6 rename notrace registration and standardize trace_utils imports 2021-11-15 21:03:21 +00:00
Alexander Soare b25ff96768 wip - pre-rebase 2021-11-12 20:45:05 +00:00
Alexander Soare bc3d4eb403 wip -rebase 2021-11-12 20:45:05 +00:00
Thomas Viehmann f805ba86d9 use .unbind instead of explicitly listing the indices 2021-10-24 21:08:47 +02:00
Ross Wightman 8880f696b6 Refactoring, cleanup, improved test coverage.
* Add eca_nfnet_l2 weights, 84.7 @ 384x384
* All 'non-std' (ie transformer / mlp) models have classifier / default_cfg test added
* Fix #694 reset_classifer / num_features / forward_features / num_classes=0 consistency for transformer / mlp models
* Add direct loading of npz to vision transformer (pure transformer so far, hybrid to come)
* Rename vit_deit* to deit_*
* Remove some deprecated vit hybrid model defs
* Clean up classifier flatten for conv classifiers and unusual cases (mobilenetv3/ghostnet)
* Remove explicit model fns for levit conv, just pass in arg
2021-06-12 16:40:02 -07:00
Ross Wightman 715519a5ef Rethink name of patch embed grid info 2021-05-06 14:08:20 -07:00
Ross Wightman b2c305c2aa Move Mlp and PatchEmbed modules into layers. Being used in lots of models now... 2021-05-06 14:03:23 -07:00
Ross Wightman f606c45c38 Add Swin Transformer models from https://github.com/microsoft/Swin-Transformer 2021-04-13 12:17:21 -07:00