Commit Graph

10 Commits (3785c234d7e67b881cf68f1f8e1ceb0c53c4da53)

Author SHA1 Message Date
Ross Wightman c8ab747bf4 BEiT-V2 checkpoints didn't remove 'module' from weights, adapt checkpoint filter 2022-09-13 17:56:49 -07:00
Ross Wightman 914544fc81 Add beitv2 224x224 checkpoints from https://github.com/microsoft/unilm/tree/master/beit2 2022-09-06 20:25:18 -07:00
Ross Wightman 909705e7ff Remove some redundant requires_grad=True from nn.Parameter in third party code 2022-07-20 12:37:41 -07:00
Li Dong 09e9f3defb
migrate azure blob for beit checkpoints
## Motivation

We are going to use a new blob account to store the checkpoints.

## Modification

Modify the azure blob storage URLs for BEiT checkpoints.
2022-04-23 13:02:29 +08:00
Ross Wightman 372ad5fa0d Significant model refactor and additions:
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
Ross Wightman 5f81d4de23 Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details... 2022-01-26 22:53:57 -08:00
Ross Wightman abc9ba2544 Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks. 2022-01-25 21:54:13 -08:00
Ross Wightman f2006b2437 Cleanup qkv_bias cat in beit model so it can be traced 2021-11-18 21:25:00 -08:00
Thomas Viehmann f805ba86d9 use .unbind instead of explicitly listing the indices 2021-10-24 21:08:47 +02:00
Ross Wightman 1c9284c640 Add BeiT 'finetuned' 1k weights and pretrained 22k weights, pretraining specific (masked) model excluded for now 2021-09-13 16:38:23 -07:00