58621723bd
Add CrossStage3 DarkNet (cs3) weights
2022-07-07 17:43:38 -07:00
9be0c84715
Change set -> dict w/ None keys for dataset split synonym search, so always consistent if more than 1 exists. Fix #1224
2022-07-07 15:33:53 -07:00
db0cee9910
Refactor cspnet configuration using dataclasses, update feature extraction for new cs3 variants.
2022-07-07 14:43:27 -07:00
eca09b8642
Add MobileVitV2 support. Fix #1332 . Move GroupNorm1 to common layers (used in poolformer + mobilevitv2). Keep ol custom ConvNeXt LayerNorm2d impl as LayerNormExp2d for reference.
2022-07-07 14:41:01 -07:00
06307b8b41
Remove experimental downsample in block support in ConvNeXt. Experiment further before keeping it in.
2022-07-07 14:37:58 -07:00
bfc0dccb0e
Improve image extension handling, add methods to modify / get defaults. Fix #1335 fix #1274 .
2022-07-07 14:23:20 -07:00
7d4b3807d5
Support DeiT-3 (Revenge of the ViT) checkpoints. Add non-overlapping (w/ class token) pos-embed support to vit.
2022-07-04 22:25:22 -07:00
d0c5bd5722
Rename cs2->cs3 for darknets. Fix features_only for cs3 darknets.
2022-07-03 08:32:41 -07:00
d765305821
Remove first_conv for resnetaa50 def
2022-07-02 15:56:17 -07:00
dd9b8f57c4
Add feature_info to edgenext for features_only support, hopefully fix some fx / test errors
2022-07-02 15:20:45 -07:00
377e9bfa21
Add TPU trained darknet53 weights. Add mising pretrain_cfg for some csp/darknet models.
2022-07-02 15:18:52 -07:00
c170ba3173
Add weights for resnet10t, resnet14t, and resnetaa50 models. Fix #1314
2022-07-02 15:18:06 -07:00
188c194b0f
Left some experiment stem code in convnext by mistake
2022-07-02 15:17:28 -07:00
70d6d2c484
support test_crop_size in data config resolve
2022-07-02 15:17:05 -07:00
6064d16a2d
Add initial EdgeNeXt import. Significant cleanup / reorg (like ConvNeXt). Fix #1320
...
* edgenext refactored for torchscript compat, stage base organization
* slight refactor of ConvNeXt to match some EdgeNeXt additions
* remove use of funky LayerNorm layer in ConvNeXt and just use nn.LayerNorm and LayerNorm2d (permute)
2022-07-01 15:18:42 -07:00
7a9c6811c9
Add eps arg to LayerNorm2d, add 'tf' (tensorflow) variant of trunc_normal_ that applies scale/shift after sampling (instead of needing to move a/b)
2022-07-01 15:15:39 -07:00
82c311d082
Add more experimental darknet and 'cs2' darknet variants (different cross stage setup, closer to newer YOLO backbones) for train trials.
2022-07-01 15:14:01 -07:00
a050fde5cd
Add resnet10t (basic block) and resnet14t (bottleneck) with 1,1,1,1 repeats
2022-07-01 15:03:28 -07:00
e6d7df40ec
no longer a point using kwargs for pretrain_cfg resolve, just pass explicit arg
2022-06-24 21:36:23 -07:00
07d0c4ae96
Improve repr for DropPath module
2022-06-24 14:58:15 -07:00
e27c16b8a0
Remove unecessary code for synbn guard
2022-06-24 14:57:42 -07:00
0da3c9ebbf
Remove SiLU layer in default args that breaks import on old old PyTorch
2022-06-24 14:56:58 -07:00
7d657d2ef4
Improve resolve_pretrained_cfg behaviour when no cfg exists, warn instead of crash. Improve usability ex #1311
2022-06-24 14:55:25 -07:00
879df47c0a
Support BatchNormAct2d for sync-bn use. Fix #1254
2022-06-24 14:51:26 -07:00
7cedc8d474
Follow up to #1256 , fix interpolation warning in auto_autoaugment as well
2022-06-21 14:56:53 -07:00
db64393c0d
use `Image.Resampling` namespace for PIL mapping ( #1256 )
...
* use `Image.Resampling` namespace for PIL mapping
PIL shows a deprecation warning when accessing resampling constants via the `Image` namespace. The suggested namespace is `Image.Resampling`. This commit updates `_pil_interpolation_to_str` to use the `Image.Resampling` namespace.
```
/tmp/ipykernel_11959/698124036.py:2: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
Image.NEAREST: 'nearest',
/tmp/ipykernel_11959/698124036.py:3: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
Image.BILINEAR: 'bilinear',
/tmp/ipykernel_11959/698124036.py:4: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
Image.BICUBIC: 'bicubic',
/tmp/ipykernel_11959/698124036.py:5: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.
Image.BOX: 'box',
/tmp/ipykernel_11959/698124036.py:6: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
Image.HAMMING: 'hamming',
/tmp/ipykernel_11959/698124036.py:7: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
Image.LANCZOS: 'lanczos',
```
* use new pillow resampling enum only if it exists
2022-06-12 22:30:57 -07:00
20a1fa63f8
Make dev version 0.6.2.dev0 for pypi pre
2022-05-15 14:29:57 -07:00
347308faad
Update README.md, version to 0.6.2
2022-05-13 13:54:41 -07:00
4b30bae67b
Add updated vit_relpos weights, and impl w/ support for official swin-v2 differences for relpos. Add bias control support for MLP layers
2022-05-13 13:53:57 -07:00
d4c0588012
Remove persistent buffers from Swin-V2. Change SwinV2Cr cos attn + tau/logit_scale to match official, add ckpt convert, init_value zeros resid LN weight by default
2022-05-13 10:50:59 -07:00
27c42f0830
Fix torchscript use for offician Swin-V2, add support for non-square window/shift to WindowAttn/Block
2022-05-13 09:29:33 -07:00
2f2b22d8c7
Disable nvfuser fma / opt level overrides per #1244
2022-05-13 09:27:13 -07:00
c0211b0bf7
Swin-V2 test fixes, typo
2022-05-12 22:31:55 -07:00
9a86b900fa
Official SwinV2 models
2022-05-12 15:05:10 -07:00
d07d015173
Merge pull request #1249 from okojoalg/sequencer
...
Add Sequencer
2022-05-09 20:42:43 -07:00
d30685c283
Merge pull request #1251 from hankyul2/fix-multistep-scheduler
...
fix: multistep lr decay epoch bugs
2022-05-09 16:07:46 -07:00
a16171335b
fix: change milestones to decay-milestones
...
- change argparser option `milestone` to `decay-milestone`
2022-05-10 07:57:19 +09:00
39b725e1c9
Fix tests for rank-4 output where feature channels dim is -1 (3) and not 1
2022-05-09 15:20:24 -07:00
78a32655fa
Fix poolformer group_matcher to merge proj downsample with previous block, support coarse
2022-05-09 12:20:04 -07:00
d79f3d9d1e
Fix torchscript use for sequencer, add group_matcher, forward_head support, minor formatting
2022-05-09 12:09:39 -07:00
37b6920df3
Fix group_matcher regex for regnet.py
2022-05-09 10:40:40 -07:00
93a79a3dd9
Fix num_features in Sequencer
2022-05-06 23:16:32 +09:00
57a988df30
fix: multistep lr decay epoch bugs
...
- add milestones arguments
- change decay_epochs to milestones variable
2022-05-06 13:14:43 +09:00
578d52e752
Add Sequencer
2022-05-06 00:36:01 +09:00
f5ca4141f7
Adjust arg order for recent vit model args, add a few comments
2022-05-02 22:41:38 -07:00
41dc49a337
Vision Transformer refactoring and Rel Pos impl
2022-05-02 15:37:39 -07:00
b7cb8d0337
Add Swin-V2 Small-NS weights (83.5 @ 224). Add layer scale like 'init_values' via post-norm LN weight scaling
2022-04-26 17:32:49 -07:00
f88c606fcf
fixing channels_last on cond_conv2d; update nvfuser debug env variable
2022-04-25 12:41:46 -07:00
09e9f3defb
migrate azure blob for beit checkpoints
...
## Motivation
We are going to use a new blob account to store the checkpoints.
## Modification
Modify the azure blob storage URLs for BEiT checkpoints.
2022-04-23 13:02:29 +08:00
52ac881402
Missed first_conv in latest seresnext 'D' default_cfgs
2022-04-22 20:55:52 -07:00
7629d8264d
Add two new SE-ResNeXt101-D 32x8d weights, one anti-aliased and one not. Reshuffle default_cfgs vs model entrypoints for resnet.py so they are better aligned.
2022-04-22 16:54:53 -07:00
569d114ef7
Fix device problem
...
Before, the one_hot could only run in device='cuda'. Now it will run on input device automatically.
2022-04-19 11:53:18 +08:00
8f0bc0591e
fix convnext args
2022-04-05 20:00:57 +08:00
c5a8e929fb
Add initial swinv2 tiny / small weights
2022-04-03 15:22:55 -07:00
f670d98cb8
Make a few more layers symbolically traceable (remove from FX leaf modules)
...
* remove dtype kwarg from .to() calls in EvoNorm as it messed up script + trace combo
* BatchNormAct2d always uses custom forward (cut & paste from original) instead of super().forward. Fixes #1176
* BlurPool groups==channels, no need to use input.dim[1]
2022-03-24 21:43:56 -07:00
ec4e9aa5a0
Add ConvNeXt tiny and small pretrain in22k
...
Add ConvNeXt tiny and small pretrain in22k from ConvNeXt repo:
06f7b05f92
2022-03-24 15:18:08 +08:00
575924ed60
Update test crop for new RegNet-V weights to match Y
2022-03-23 21:40:53 -07:00
1618527098
Add layer scale and parallel blocks to vision_transformer
2022-03-23 16:09:07 -07:00
c42be74621
Add attrib / comments about Swin-S3 (AutoFormerV2) weights
2022-03-23 16:07:09 -07:00
474ac906a2
Add 'head norm first' convnext_tiny_hnf weights
2022-03-23 16:06:00 -07:00
dc51334cdc
Fix pruned adapt for EfficientNet models that are now using BatchNormAct layers
2022-03-22 20:33:01 -07:00
024fc4d9ab
version 0.6.1 for master
2022-03-21 22:03:13 -07:00
e1e037ba52
Fix bad tuple typing fix that was on XLA branch bust missed on master merge
2022-03-21 22:00:33 -07:00
341b464a5a
Remove redundant noise attr from Plateau scheduler (use parent)
2022-03-21 22:00:03 -07:00
fe457c1996
Update SwinTransformerV2Cr post-merge, update with grad checkpointing / grad matcher
...
* weight compat break, activate norm3 for final block of final stage (equivalent to pre-head norm, but while still in BLC shape)
* remove fold/unfold for TPU compat, add commented out roll code for TPU
* add option for end of stage norm in all stages
* allow weight_init to be selected between pytorch default inits and xavier / moco style vit variant
2022-03-21 14:50:28 -07:00
b049a5c5c6
Merge remote-tracking branch 'origin/master' into norm_norm_norm
2022-03-21 13:41:43 -07:00
7cdd164d77
Fix #1184 , scheduler noise bug during merge madness
2022-03-21 13:35:45 -07:00
9440a50c95
Merge branch 'mrT23-master'
2022-03-21 12:30:02 -07:00
d98aa47d12
Revert ml-decoder changes to model factory and train script
2022-03-21 12:29:02 -07:00
b20665d379
Merge pull request #1007 from qwertyforce/patch-1
...
update arxiv link
2022-03-21 12:12:58 -07:00
7a0994f581
Merge pull request #1150 from ChristophReich1996/master
...
Swin Transformer V2
2022-03-21 11:56:57 -07:00
61d3493f87
Fix hf-hub handling when hf-hub is config source
2022-03-21 11:12:55 -07:00
5f47518f27
Fix pit implementation to be clsoer to deit/levit re distillation head handling
2022-03-21 11:12:14 -07:00
0862e6ebae
Fix correctness of some group matching regex (no impact on result), some formatting, missed forward_head for resnet
2022-03-19 14:58:54 -07:00
94bcdebd73
Add latest weights trained on TPU-v3 VM instances
2022-03-18 21:35:41 -07:00
0557c8257d
Fix bug introduced in non layer_decay weight_decay application. Remove debug print, fix arg desc.
2022-02-28 17:06:32 -08:00
372ad5fa0d
Significant model refactor and additions:
...
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2022-02-28 13:56:23 -08:00
1420c118df
Missed comitting outstanding changes to default_cfg keys and test exclusions for swin v2
2022-02-23 19:50:26 -08:00
c6e4b7895a
Swin V2 CR impl refactor.
...
* reformat and change some naming so closer to existing timm vision transformers
* remove typing that wasn't adding clarity (or causing torchscript issues)
* support non-square windows
* auto window size adjust from image size
* post-norm + main-branch no
2022-02-23 17:28:52 -08:00
67d140446b
Fix bug in classification head
2022-02-20 22:28:05 +01:00
29add820ac
Refactor (back to relative imports)
2022-02-20 00:46:48 +01:00
74a04e0016
Add parameter to change normalization type
2022-02-20 00:46:00 +01:00
2a4f6c13dd
Create model functions
2022-02-20 00:40:22 +01:00
87b4d7a29a
Add get and reset classifier method
2022-02-19 22:47:02 +01:00
ff5f6bcd6c
Check input resolution
2022-02-19 22:42:02 +01:00
81bf0b4033
Change parameter names to match Swin V1
2022-02-19 22:37:22 +01:00
f227b88831
Add initials (CR) to model and file
2022-02-19 22:14:38 +01:00
90dc74c450
Add code from https://github.com/ChristophReich1996/Swin-Transformer-V2 and change docstring style to match timm
2022-02-19 22:12:11 +01:00
2c3870e107
semobilevit_s for good measure
2022-01-31 22:36:09 -08:00
bcaeb91b03
Version to 0.6.0, possible interface incompatibilities vs 0.5.x
2022-01-31 15:42:14 -08:00
58ba49c8ef
Add MobileViT models (w/ ByobNet base). Close #1038 .
2022-01-31 15:39:34 -08:00
5f81d4de23
Move DeiT to own file, vit getting crowded. Working towards fixing #1029 , make pooling interface for transformers and mlp closer to convnets. Still working through some details...
2022-01-26 22:53:57 -08:00
cf57695938
sched noise dup code remove
2022-01-26 11:53:08 +03:00
95cfc9b3e8
Merge remote-tracking branch 'origin/master' into norm_norm_norm
2022-01-25 22:20:45 -08:00
abc9ba2544
Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks.
2022-01-25 21:54:13 -08:00
07379c6d5d
Add vit_base2_patch32_256 for a model between base_patch16 and patch32 with a slightly larger img size and width
2022-01-24 14:46:47 -08:00
447677616f
version 0.5.5
2022-01-20 21:18:30 -08:00
83b40c5a58
Last batch of small model weights (for now). mobilenetv3_small 050/075/100 and updated mnasnet_small with lambc/lamb optimizer.
2022-01-19 10:02:02 -08:00
cdcd0a92ca
fix lars
2022-01-19 17:49:43 +08:00
1aa617cb3b
Add AvgPool2d anti-aliasing support to ResNet arch (as per OpenAI CLIP models), add a few blur aa models as well
2022-01-18 21:57:24 -08:00
f0f9eccda8
Add --fuser arg to train/validate/benchmark scripts to select jit fuser type
2022-01-17 13:54:25 -08:00
010b486590
Add Dino pretrained weights (no head) for vit models. Add support to tests and helpers for models w/ no classifier (num_classes=0 in pretrained cfg)
2022-01-17 12:20:02 -08:00
738a9cd635
unbiased=False for torch.var_mean path of ConvNeXt LN. Fix #1090
2022-01-17 09:25:06 -08:00
e0c4eec4b6
Default conv_mlp to False across the board for ConvNeXt, causing issues on more setups than it's improving right now...
2022-01-16 14:20:08 -08:00
b669f4a588
Add ConvNeXt 22k->1k fine-tuned and 384 22k-1k fine-tuned weights after testing
2022-01-15 15:44:36 -08:00
e967c72875
Update REAMDE.md. Sneak in g/G (giant / gigantic?) ViT defs from scaling paper
2022-01-14 16:28:27 -08:00
9ca3437178
Add some more small model weights lcnet, mnas, mnv2
2022-01-14 16:28:27 -08:00
fa6463c936
Version 0.5.4
2022-01-14 16:28:27 -08:00
fa81164378
Fix stem width for really small mobilenetv3 arch defs
2022-01-14 16:28:27 -08:00
edd3d73695
Add missing dropout for head reset in ConvNeXt default head
2022-01-14 16:28:27 -08:00
b093dcb46d
Some convnext cleanup, remove in place mul_ for gamma, breaking symbolic trace, cleanup head a bit...
2022-01-14 16:28:27 -08:00
18934debc5
Add initial ConvNeXt impl (mods of official code)
2022-01-14 16:28:27 -08:00
656757d26b
Fix MobileNetV2 head conv size for multiplier < 1.0. Add some missing modification copyrights, fix starting date of some old ones.
2022-01-14 16:28:27 -08:00
ccfeb06936
Fix out_indices handling breakage, should have left as per vgg approach.
2022-01-07 19:30:51 -08:00
a9f91483a6
Fix #1078 , DarkNet has 6 feature maps. Make vgg and darknet out_indices handling/comments equivalent
2022-01-07 15:08:32 -08:00
c21b21660d
visformer supports spatial feat map, update pool_size in pretrained cfg to match
2022-01-07 14:31:43 -08:00
9c11dfd9cb
Fix fbnetv3 pretrained cfg changes
2022-01-07 14:09:50 -08:00
1406cddc2e
FBNetV3 timm trained weights added for b/d/g variants. Update version to 0.5.2 for pypi release.
2022-01-07 12:05:08 -08:00
02ae11e526
Leaving repeat aug sampler indices as tensor thrashes worker shared process memory
2022-01-06 22:33:09 -08:00
4df51f3932
Add lcnet_100 and mnasnet_small weights
2022-01-06 22:21:05 -08:00
5ccf682a8f
Remove deprecated bn-tf train arg and create_model handler. Add evos/evob models back into fx test filter until norm_norm_norm branch merged.
2022-01-06 18:08:39 -08:00
b9a715c86a
Add more small model defs for MobileNetV3/V2/LCNet
2022-01-06 16:06:43 -08:00
b27c21b09a
Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster
2022-01-06 16:04:58 -08:00
214c84a235
Disable use of timm nn.Linear wrapper since AMP autocast + torchscript use appears fixed
2022-01-06 16:01:51 -08:00
72b57163d1
Merge branch 'master' of https://github.com/mrT23/pytorch-image-models into mrT23-master
2022-01-06 13:57:16 -08:00
de5fa791c6
Merge branch 'master' into norm_norm_norm
2022-01-03 11:37:00 -08:00
26ff57f953
Add more small model defs for MobileNetV3/V2/LCNet
2022-01-03 11:30:54 -08:00
a0b2657497
Use `torch.repeat_interleave()` to generate repeated indices faster ( #1058 )
...
* update: use numpy to generate repeated indices faster
* update: use torch.repeat_interleave() instead of np.repeat()
* refactor: remove unused import, numpy
* refactor: torch.range to torch.arange
* update: tensor to list before appending the extra samples
* update: concatenate the paddings with torch.cat
2022-01-02 14:01:06 -08:00
450ac6a0f5
Post merge tinynet fixes for pool_size, feature extraction
2021-12-21 23:51:54 -08:00
a04164cd75
Merge branch 'tinynet' of https://github.com/rsomani95/pytorch-image-models into rsomani95-tinynet
2021-12-21 22:45:56 -08:00
8a93ce6ee3
Fix regnetv/w tests, refactor regnet generator code a bit
2021-12-16 17:08:25 -08:00
4dec8c8087
Fix skip path regression for updated EfficientNet and RegNet def. Add Pre-Act RegNet support (experimental). Remove BN-TF flag. Add efficientnet_b0_g8_gn model.
2021-12-16 14:53:57 -08:00
a52a614475
Remove layer experiment which should not have been added
2021-12-14 14:29:32 -08:00
ab49d275de
Significant norm update
...
* ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased
* Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise)
* Update RegNet to add Z variant
* Add Pre variant of XceptionAligned that works with NormAct layers
* EvoNorm matches bits_and_tpu branch for merge
2021-12-14 13:48:30 -08:00
31bcd36e46
add tinynet models
2021-12-14 19:34:04 +05:30
b4b8d1ec18
fix hard-coded strides
2021-12-14 17:22:54 +08:00
d04f2f1377
Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster
2021-12-05 15:36:56 -08:00
834a9ec721
Disable use of timm nn.Linear wrapper since AMP autocast + torchscript use appears fixed
2021-12-01 14:58:09 -08:00
78912b6375
Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials.
2021-12-01 12:09:01 -08:00
55adfbeb8d
Add commented code to increase open file limit via Python (for TFDS dataset building)
2021-11-30 17:02:27 -08:00
c11f4c3218
support CNNs
2021-11-30 08:48:08 +02:00
d6701d8a81
Merge branch 'rwightman:master' into master
2021-11-30 08:07:44 +02:00
ccb3815360
update arxiv link
2021-11-29 21:41:00 +03:00
3dc71695bf
Merge pull request #989 from martinsbruveris/feat/resmlp-dino
...
Added DINO pretrained ResMLP models.
2021-11-24 09:26:07 -08:00
480c676ffa
Fix FX breaking assert in evonorm
2021-11-24 09:24:47 -08:00
85c5ff26d7
Added DINO pretrained ResMLP models.
2021-11-24 15:02:46 +02:00
d633a014e6
Post merge cleanup. Fix potential security issue passing kwargs directly through to serialized web data.
2021-11-23 16:54:01 -08:00
b18c9e323b
Update helpers.py
2021-11-22 23:43:44 -05:00
308d0b9554
Merge branch 'master' into hf-save-and-push
2021-11-22 23:39:27 -05:00
f0507f6da6
Fix k_decay default arg != 1.0 in poly scheduler
2021-11-22 09:37:36 -08:00
41559247e9
use_ml_decoder_head
2021-11-22 17:50:39 +02:00
1f53db2ece
Updated lamhalobotnet weights, 81.5 top-1
2021-11-21 19:49:51 -08:00
15ef108eb4
Add better halo2botnet50ts weights, 82 top-1 @ 256
2021-11-21 14:09:12 -08:00
734b2244fe
Add RegNetZ-D8 (83.5 @ 256, 84 @ 320) and RegNetZ-E8 (84.5 @ 256, 85 @ 320) weights. Update names of existing RegZ models to include group size.
2021-11-20 15:52:04 -08:00
93cc08fdc5
Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).
2021-11-20 15:50:51 -08:00
af607b75cc
Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU
2021-11-19 17:37:00 -08:00
c976a410d9
Add ResNet-50 w/ GN (resnet50_gn) and SEBotNet-33-TS (sebotnet33ts_256) model defs and weights. Update halonet50ts weights w/ slightly better variant in1k val, more robust to test sets.
2021-11-19 14:24:43 -08:00
f2006b2437
Cleanup qkv_bias cat in beit model so it can be traced
2021-11-18 21:25:00 -08:00
1076a65df1
Minor post FX merge cleanup
2021-11-18 19:47:07 -08:00
32c9937dec
Merge branch 'fx-feature-extract-new' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-fx-feature-extract-new
2021-11-18 16:31:29 -08:00
78b36bf46c
Places365 doesn't exist in some still used torchvision version
2021-11-18 14:59:51 -08:00
65d827c7a6
rename notrace registration and standardize trace_utils imports
2021-11-15 21:03:21 +00:00
9b2daf2a35
Add ResNeXt-50 weights 81.1 top-1 @ 224, 82 @ 288 with A1 'high aug' recipe
2021-11-14 13:17:27 -08:00
9b5d6dc7e2
Merge branch 'add-vit-b8' of https://github.com/martinsbruveris/pytorch-image-models into martinsbruveris-add-vit-b8
2021-11-14 12:54:28 -08:00
cfa414cad2
Matching two bits_and_tpu changes for TFDs wrapper
...
* change 'samples' -> 'examples' for tfds wrapper to match tfds naming
* add class_to_idx for image classification datasets in tfds wrapper
2021-11-14 12:52:19 -08:00
5220711d87
Added B/8 models to ViT.
2021-11-14 11:01:48 +00:00
0262a0e8e1
fx ready for review
2021-11-13 00:06:33 +00:00
d2994016e9
Add try/except guards
2021-11-12 21:16:53 +00:00
b25ff96768
wip - pre-rebase
2021-11-12 20:45:05 +00:00
e051dce354
Make all models FX traceable
2021-11-12 20:45:05 +00:00
cf4561ca72
Add FX based FeatureGraphNet capability
2021-11-12 20:45:05 +00:00
0149ec30d7
wip - attempting to rebase
2021-11-12 20:45:05 +00:00
02c3a75a45
wip - make it possible to use fx graph in train and eval mode
2021-11-12 20:45:05 +00:00
bc3d4eb403
wip -rebase
2021-11-12 20:45:05 +00:00
ab3ac3f25b
Add FX based FeatureGraphNet capability
2021-11-12 20:45:05 +00:00
9ec3210c2d
More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).
2021-11-10 15:52:09 -08:00
ba65dfe2c6
Dataset work
...
* support some torchvision datasets
* improvements to TFDS wrapper for subsplit handling (fix #942 ), shuffle seed
* add class-map support to train (fix #957 )
2021-11-09 22:34:15 -08:00
ddc29da974
Add ResNet101 and ResNet152 weights from higher aug RSB recipes. 81.93 and 82.82 top-1 at 224x224.
2021-11-02 17:59:16 -07:00
b328e56f49
Update eca_halonext26ts weights to a better set
2021-11-02 16:52:53 -07:00
2ddef942b9
Better fix for #954 that doesn't break torchscript, pull torch._assert into timm namespace when it exists
2021-11-02 11:22:33 -07:00
4f0f9cb348
Fix #954 by bringing traceable _assert into timm to allow compat w/ PyTorch < 1.8
2021-11-02 09:21:40 -07:00
a41de1f666
Add interpolation mode handling to transforms. Removes InterpolationMode warning. Works for torchvision versions w/ and w/o InterpolationMode enum. Fix #738 .
2021-10-28 17:35:01 -07:00
ed41d32637
Add repr to auto_augment and random_erasing impl
2021-10-28 17:33:36 -07:00
ae72d009fa
Add weights for lambda_resnet50ts, halo2botnet50ts, lamhalobotnet50ts, updated halonet50ts
2021-10-27 22:08:54 -07:00
b745d30a3e
Fix formatting of last commit
2021-10-25 15:15:14 -07:00
3478f1d7f1
Traceability fix for vit models for some experiments
2021-10-25 15:13:08 -07:00
f658a72e72
Cleanup re-use of Dropout modules in Mlp modules after some twitter feedback :p
2021-10-25 00:40:59 -07:00
f805ba86d9
use .unbind instead of explicitly listing the indices
2021-10-24 21:08:47 +02:00
57992509f9
Fix some formatting in utils/model.py
2021-10-23 20:35:36 -07:00
0fe4fd3f1f
add d8 and e8 regnetz models with group size 8
2021-10-23 20:34:21 -07:00
25e7c8c5e5
Update broken resnetv2_50 weight url, add resnetv1_101 a1h recipe weights for 224x224 train
2021-10-20 22:14:12 -07:00
b6caa356d2
Fixed eca_botnext26ts_256 weights added, 79.27
2021-10-19 12:44:28 -07:00
c02334d9fa
Add weights for regnetz_d and haloregnetz_c, update regnetz_c weights. Add commented PyTorch XLA code for halo attention
2021-10-19 12:32:09 -07:00
02daf2ab94
Add option to include relative pos embedding in the attention scaling as per references. See discussion #912
2021-10-12 15:37:01 -07:00
047a5ec05f
Fix bugs that Mixup does not work device=cpu
2021-10-12 23:51:46 +09:00
cd34913278
Remove some outdated comments, botnet networks working great now.
2021-10-11 22:43:41 -07:00
6ed4cdccca
Update lambda_resnet26t weights with better set
2021-10-10 16:32:54 -07:00
44d6d51668
Add ConvMixer
2021-10-09 21:09:51 -04:00
a85df34993
Update lambda_resnet26rpt weights to 78.9, add better halonet26t weights at 79.1 with tweak to attention dim
2021-10-08 17:44:13 -07:00
b544ad4d3f
regnetz model default cfg tweaks
2021-10-06 21:14:59 -07:00
e5da481073
Small post-merge tweak for freeze/unfreeze, add to __init__ for utils
2021-10-06 17:00:27 -07:00
5ca72dcc75
Merge branch 'freeze-functionality' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-freeze-functionality
2021-10-06 16:51:03 -07:00
e2b8d44ff0
Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs
...
* align interfaces of halo, bottleneck attn and lambda layer
* add qk_ratio to all of above, control q/k dim relative to output dim
* add experimental haloregnetz, and trionet (lambda + halo + bottle) models
2021-10-06 16:32:48 -07:00
431e60c83f
Add acknowledgements for freeze_batch_norm inspiration
2021-10-06 14:28:49 +01:00
fbf59c04ee
Change crop ratio on correct resnet50 variant.
2021-10-04 22:31:08 -07:00
ae1ff5792f
Clean a1/a2/3 rsb _0 checkpoints properly, fix v2 loading.
2021-10-04 16:46:00 -07:00
93901e992f
Version bump to 0.5.0 for pending release post RSB and ATTN updates
2021-10-03 17:34:57 -07:00
da0d39bedd
Update default crop_pct for byoanet
2021-10-03 17:33:16 -07:00
cc9bedf373
Add initial ResNet Strikes Back weights for ResNet50 and ResNetV2-50 models
2021-10-03 17:32:02 -07:00
64495505b7
Add updated lambda resnet26 and botnet26 checkpoints with fixes applied
2021-10-03 17:31:39 -07:00
b2094f4ee8
support bits checkpoints in avg/load
2021-10-03 17:31:22 -07:00
007bc39323
Some halo and bottleneck attn code cleanup, add halonet50ts weights, use optimal crop ratios
2021-10-02 15:51:42 -07:00
65c3d78b96
Freeze unfreeze functionality finalized. Tests added
2021-10-02 15:55:08 +01:00
0cb8ea432c
wip
2021-10-02 15:55:08 +01:00
b1c2e3eb92
Match rel_pos_indices attr rename in conv branch
2021-09-30 23:19:05 -07:00
b49630a138
Add relative pos embed option to LambdaLayer, fix last transpose/reshape.
2021-09-30 22:45:09 -07:00
d657e2cc0b
Remove dead code line from efficientnet
2021-09-30 21:54:42 -07:00
0ca687f224
Make 'regnetz' model experiments closer to actual RegNetZ, bottleneck expansion, expand from in_chs, no shortcut on stride 2, tweak model sizes
2021-09-30 21:49:38 -07:00
51eaf9360d
Remove a duplicate layer creation in byobnet.py
...
`self.conv2_kxk` is repeated in `byobnet.py`. Remove the duplicate code.
2021-09-30 18:30:48 +08:00
b81e79aae9
Fix bottleneck attn transpose typo, hopefully these train better now..
2021-09-28 16:38:41 -07:00
80075b0b8a
Add worker_seeding arg to allow selecting old vs updated data loader worker seed for (old) experiment repeatability
2021-09-28 16:37:45 -07:00
6478bcd02c
Fix regnetz_d conv layer name, use inception mean/std
2021-09-26 14:54:17 -07:00
0387e6057e
Update binary cross ent impl to use thresholding as an option (convert soft targets from mixup/cutmix to 0, 1)
2021-09-23 15:45:39 -07:00
f8a63a3b71
Add worker_init_fn to loader for numpy seed per worker
2021-09-23 15:44:38 -07:00
515121cca1
Use reshape instead of view in std_conv, causing issues in recent PyTorch in channels_last
2021-09-23 15:43:48 -07:00
da06cc61d4
ResNetV2 seems to work best without zero_init residual
2021-09-23 15:43:22 -07:00
8e11da0ce3
Add experimental RegNetZ(ish) models for training / perf trials.
2021-09-23 15:42:57 -07:00
6bbc50beb4
make it possible to provide norm_layer via create_model
2021-09-21 10:19:04 +01:00
adcb74f87f
🎨 Import load_state_dict_from_url directly
2021-09-14 01:11:40 -04:00
e65a2cba3d
🎨 cleanup and add a couple comments
2021-09-14 01:07:04 -04:00
2b6ade24b3
🎨 write model card to enable inference
2021-09-13 23:31:28 -04:00
cf5ac2800c
BotNet models were still off, remove weights for bad configs. Add good SE-HaloNet33-TS weights.
2021-09-13 17:18:59 -07:00
24720abe3b
Merge branch 'master' into attn_update
2021-09-13 16:51:10 -07:00
1c9284c640
Add BeiT 'finetuned' 1k weights and pretrained 22k weights, pretraining specific (masked) model excluded for now
2021-09-13 16:38:23 -07:00
f8a215cfe6
A few more crossvit tweaks, fix training w/ no_weight_decay names, add crop option for scaling, adjust default crop_pct for large img size to 1.0 for better results
2021-09-13 14:17:34 -07:00
7ab2491ab7
Better handling of crossvit for tests / forward_features, fix torchscript regression in my changes
2021-09-13 13:01:05 -07:00
f1808e0970
Post crossvit merge cleanup, change model names to reflect input size, cleanup img size vs scale handling, fix tests
2021-09-13 11:49:54 -07:00
4027412757
Add resnet33ts weights, update resnext26ts baseline weights
2021-09-09 14:46:41 -07:00
9fe5798bee
fix bug for reset classifier and fix for validating the dimension
2021-09-08 21:58:17 -04:00
3718c5a5bd
fix loading pretrained model
2021-09-08 11:53:05 -04:00
bb50b69a57
fix for torch script
2021-09-08 11:20:59 -04:00
abf9d51bc3
🚧 wip
2021-09-07 18:39:26 -06:00
5bd04714e4
Cleanup weight init for byob/byoanet and related
2021-09-05 15:34:05 -07:00
8642401e88
Swap botnet 26/50 weights/models after realizing a mistake in arch def, now figuring out why they were so low...
2021-09-05 15:17:19 -07:00
5f12de4875
Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.
2021-09-05 12:41:14 -07:00
76881d207b
Add baseline resnet26t @ 256x256 weights. Add 33ts variant of halonet with at least one halo in stage 2,3,4
2021-09-04 14:52:54 -07:00
484e61648d
Adding the attn series weights, tweaking model names, comments...
2021-09-03 18:09:42 -07:00
fb94350896
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
2021-09-01 17:46:40 -07:00
f262137ff2
Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments.
2021-09-01 17:40:53 -07:00
ba9c1108a1
Add a BCE loss impl that converts dense targets to sparse /w smoothing as an alternate to CE w/ smoothing. For training experiments.
2021-09-01 17:39:28 -07:00
29a37e23ee
LR scheduler update:
...
* add polynomial decay 'poly'
* cleanup cycle specific args for cosine, poly, and tanh sched, t_mul -> cycle_mul, decay -> cycle_decay, default cycle_limit to 1 in each opt
* add k-decay for cosine and poly sched as per https://arxiv.org/abs/2004.05909
* change default tanh ub/lb to push inflection to later epochs
2021-09-01 17:33:11 -07:00
28d2841acf
💄 apply isort
2021-09-01 18:15:08 -06:00
492c0a4e20
Update HaloAttn comment
2021-09-01 17:14:31 -07:00
e72c989973
✨ add ability to push to hf hub
2021-09-01 18:14:28 -06:00
7ab9d4555c
add crossvit
2021-09-01 17:13:12 -04:00
3b9032ea48
Use Tensor.unfold().unfold() for HaloAttn, fast like as_strided but more clarity
2021-08-27 12:45:53 -07:00
78933122c9
Fix silly typo
2021-08-27 09:22:20 -07:00
2568ffc5ef
Merge branch 'master' into attn_update
2021-08-27 09:21:22 -07:00
708d87a813
Fix ViT SAM weight compat as weights at URL changed to not use repr layer. Fix #825 . Tweak optim test.
2021-08-27 09:20:13 -07:00
8449ba210c
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
2021-08-26 21:56:44 -07:00
a8b65695f1
Add resnet26ts and resnext26ts models for non-attn baselines
2021-08-21 12:42:10 -07:00
a5a542f17d
Fix typo
2021-08-20 17:47:23 -07:00
925e102982
Update attention / self-attn based models from a series of experiments:
...
* remove dud attention, involution + my swin attention adaptation don't seem worth keeping
* add or update several new 26/50 layer ResNe(X)t variants that were used in experiments
* remove models associated with dead-end or uninteresting experiment results
* weights coming soon...
2021-08-20 16:13:11 -07:00
d667351eac
Tweak accuracy topk safety. Fix #807
2021-08-19 14:18:53 -07:00
35c9740826
Fix accuracy when topk > num_classes
2021-08-19 11:58:59 +02:00
a16a753852
Add lamb/lars to optim init imports, remove stray comment
2021-08-18 22:55:02 -07:00
c207e02782
MOAR optimizer changes. Woo!
2021-08-18 22:20:35 -07:00
a426511c95
More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.
2021-08-18 17:21:56 -07:00
9541f4963b
One more scalar -> tensor fix for lamb optimizer
2021-08-18 11:20:25 -07:00
8f68193c91
Update lamp.py comment
2021-08-18 09:27:40 -07:00
4d284017b8
Merge pull request #813 from rwightman/opt_cleanup
...
Optimizer cleanup and additions
2021-08-18 09:12:00 -07:00
a6af48be64
add madgradw optimizer
2021-08-17 22:19:27 -07:00
55fb5eedf6
Remove experiment from lamb impl
2021-08-17 21:48:26 -07:00
8a9eca5157
A few optimizer comments, dead import, missing import
2021-08-17 18:01:33 -07:00
ac469b50da
Optimizer improvements, additions, cleanup
...
* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad
2021-08-17 17:51:20 -07:00
abf3e044bb
Update scheduler_factory.py
...
remove duplicate code from create_scheduler()
2021-08-14 22:53:17 +02:00
3cdaf5ed56
Add `mmax` config key to auto_augment for increasing upper bound of RandAugment magnitude beyond 10. Make AugMix uniform sampling default not override config setting.
2021-08-12 15:39:05 -07:00
1042b8a146
Add non fused LAMB optimizer option
2021-08-09 13:13:43 -07:00
01cb46a9a5
Add gc_efficientnetv2_rw_t weights (global context instead of SE attn). Add TF XL weights even though the fine-tuned ones don't validate that well. Change default arg for GlobalContext to use scal (mul) mode.
2021-08-07 16:45:29 -07:00
d3f7440650
Add EfficientNetV2 XL model defs
2021-07-22 13:15:24 -07:00
72b227dcf5
Merge pull request #750 from drjinying/master
...
Specify "interpolation" mode in vision_transformer's resize_pos_embed
2021-07-13 11:01:20 -07:00
2907c1f967
Merge pull request #746 from samarth4149/master
...
Adding a Multi Step LR Scheduler
2021-07-13 10:55:54 -07:00
748ab852ca
Allow act_layer switch for xcit, fix in_chans for some variants
2021-07-12 13:27:29 -07:00
20b2d4b69d
Use bicubic interpolation in resize_pos_embed()
2021-07-12 10:38:31 -07:00
d3255adf8e
Merge branch 'xcit' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-xcit
2021-07-12 08:30:30 -07:00
f8039c7492
Fix gc effv2 model cfg name
2021-07-11 12:14:31 -07:00
3a55a30ed1
add notes from author
2021-07-11 14:25:58 +01:00
899cf84ccc
bug fix - missing _dist postfix for many of the 224_dist models
2021-07-11 12:41:51 +01:00
623e8b8eb8
wip xcit
2021-07-11 09:39:38 +01:00
392368e210
Add efficientnetv2_rw_t defs w/ weights, and gc variant, as well as gcresnet26ts for experiments. Version 0.4.13
2021-07-09 16:46:52 -07:00
daab57a6d9
1. Added a simple multi step LR scheduler
2021-07-09 16:18:27 -04:00
6d8272e92c
Add SAM pretrained model defs/weights for ViT B16 and B32 models.
2021-07-08 11:51:12 -07:00
ee4d8fc69a
Remove unecessary line from nest post refactor
2021-07-05 21:22:46 -07:00
8165cacd82
Realized LayerNorm2d won't work in all cases as is, fixed.
2021-07-05 18:21:34 -07:00
81cd6863c8
Move aggregation (convpool) for nest into NestLevel, cleanup and enable features_only use. Finalize weight url.
2021-07-05 18:20:49 -07:00
6ae0ac6420
Merge branch 'nested_transformer' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-nested_transformer
2021-07-03 12:45:26 -07:00
7b8a0017f1
wip to review
2021-07-03 12:10:12 +01:00
b11d949a06
wip checkpoint with some feature extraction work
2021-07-03 11:45:19 +01:00
23bb72ce5e
nested_transformer wip
2021-07-02 20:12:29 +01:00
766b4d3262
Fix features for resnetv2_50t
2021-06-28 15:56:24 -07:00
e8045e712f
Fix BatchNorm for ResNetV2 non GN models, add more ResNetV2 model defs for future experimentation, fix zero_init of last residual for pre-act.
2021-06-28 10:52:45 -07:00
20a2be14c3
Add gMLP-S weights, 79.6 top-1
2021-06-23 10:40:30 -07:00
85f894e03d
Fix ViT in21k representation (pre_logits) layer handling across old and new npz checkpoints
2021-06-23 10:38:34 -07:00
b41cffaa93
Fix a few issues loading pretrained vit/bit npz weights w/ num_classes=0 __init__ arg. Missed a few other small classifier handling detail on Mlp, GhostNet, Levit. Should fix #713
2021-06-22 23:16:05 -07:00
9c9755a808
AugReg release
2021-06-20 17:46:06 -07:00
381b279785
Add hybrid model fwds back
2021-06-19 22:28:44 -07:00
26f04a8e3e
Fix a weight link
2021-06-19 16:39:36 -07:00
8f4a0222ed
Add GMixer-24 MLP model weights, trained w/ TPU + PyTorch XLA
2021-06-18 16:49:28 -07:00
4c09a2f169
Bump version 0.4.12
2021-06-18 16:17:34 -07:00
b319eb5b5d
Update ViT weights, more details to be added before merge.
2021-06-18 16:16:49 -07:00
8257b86550
Fix up resnetv2 bit/bitm model default res
2021-06-18 16:16:06 -07:00
1228f5a3d8
Add BiT distilled 50x1 and teacher 152x2 models from 'A good teacher is patient and consistent' paper.
2021-06-18 11:40:33 -07:00
511a8e8c96
Add official ResMLP weights.
2021-06-14 17:03:16 -07:00
b9cfb64412
Support npz custom load for vision transformer hybrid models. Add posembed rescale for npz load.
2021-06-14 12:31:44 -07:00
8319e0c373
Add file docstring to std_conv.py
2021-06-13 12:31:06 -07:00
4d96165989
Merge branch 'master' into cleanup_xla_model_fixes
2021-06-12 23:19:25 -07:00
8880f696b6
Refactoring, cleanup, improved test coverage.
...
* Add eca_nfnet_l2 weights, 84.7 @ 384x384
* All 'non-std' (ie transformer / mlp) models have classifier / default_cfg test added
* Fix #694 reset_classifer / num_features / forward_features / num_classes=0 consistency for transformer / mlp models
* Add direct loading of npz to vision transformer (pure transformer so far, hybrid to come)
* Rename vit_deit* to deit_*
* Remove some deprecated vit hybrid model defs
* Clean up classifier flatten for conv classifiers and unusual cases (mobilenetv3/ghostnet)
* Remove explicit model fns for levit conv, just pass in arg
2021-06-12 16:40:02 -07:00
ba2ca4b464
One codepath for stdconv, switch layernorm to batchnorm so gain included. Tweak epsilon values for nfnet, resnetv2, vit hybrid.
2021-06-12 12:27:43 -07:00
b7a568f065
Fix torchscript issue in bat
2021-06-08 23:19:51 -07:00
d17b374f0f
Minimum input_size needed to be higher
2021-06-08 21:31:39 -07:00
b3b90d944d
Add min_input_size to bat_resnext to prevent test breakage.
2021-06-08 17:32:08 -07:00
d413eef1bf
Add ResMLP-24 model weights that I trained in PyTorch XLA on TPU-VM. 79.2 top-1.
2021-06-08 14:22:05 -07:00
10d8fa4620
Add gc and bat attention resnext26ts variants to byob for test.
2021-06-08 14:21:07 -07:00
2f5ed2dec1
Update `init_values` const for 24 and 36 layer ResMLP models
2021-06-07 17:15:04 -07:00
8e4ac3549f
All ScaledStdConv and StdConv uses default to using F.layernorm so that they work with PyTorch XLA. eps value tweaking is a WIP.
2021-06-07 17:14:19 -07:00
2a63d0246b
Post merge cleanup
2021-06-07 14:38:30 -07:00
45dec179e5
Merge pull request #681 from lmk123568/master
...
Update convit.py
2021-06-07 14:10:53 -07:00
ded1671483
Fix stochastic depth working only with a shortcut
2021-06-07 23:08:55 +09:00
b87d98b238
Update convit.py
...
Cut out the duplicates
2021-06-06 17:58:31 +08:00
02320c3e3d
Bump version to 0.4.11
2021-05-31 15:41:51 -07:00
bda8ab015a
Remove min channels for SelectiveKernel, divisor should cover cases well enough.
2021-05-31 15:38:56 -07:00
a27f4aec4a
Missed args for skresnext w/ refactoring.
2021-05-31 14:06:34 -07:00
307a935b79
Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.
2021-05-31 13:18:11 -07:00
8bf63b6c6c
Able to use other attn layer in EfficientNet now. Create test ECA + GC B0 configs. Make ECA more configurable.
2021-05-30 12:47:02 -07:00
bcec14d3b5
Bring EfficientNet SE layer in line with others, pull se_ratio outside of blocks. Allows swapping w/ other attn layers.
2021-05-29 23:41:38 -07:00
9611458e19
Throw in some FBNetV3 code I had lying around, some refactoring of SE reduction channel calcs for all EffNet archs.
2021-05-28 20:47:24 -07:00
01b9108619
Merge branch 'master' into more_attn
2021-05-28 11:09:37 -07:00
d7bab8a6c5
Fix strict flag change for checkpoint load.
2021-05-28 09:54:50 -07:00
02f9d4bc34
Add weights for resnet51q model, add 61q def.
2021-05-28 09:53:16 -07:00
f615474be3
Fix broken test, repvgg block doesn't have attn_last attr.
2021-05-27 18:12:22 -07:00
742c2d5247
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
2021-05-27 18:03:29 -07:00
9c78de8c02
Fix #661 , move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present.
2021-05-26 15:28:42 -07:00
5db7452173
Fix visformer in_chans stem handling
2021-05-25 14:11:36 -07:00
318360c3f9
Update README.md before merge. Bump version to 0.4.10
2021-05-25 12:26:16 -07:00
11ae795e99
Redo LeViT attention bias caching in a way that works with both torchscript and DataParallel
2021-05-25 10:15:32 -07:00
d400f1dbdd
Filter test models before creation for backward/torchscript tests
2021-05-25 10:14:45 -07:00
c4572cc5aa
Add Visformer-small weighs, tweak torchscript jit test img size.
2021-05-24 22:50:12 -07:00
bfc72f75d3
Expand scope of testing for non-std vision transformer / mlp models. Some related cleanup and create fn cleanup for all vision transformer and mlp models. More CoaT weights.
2021-05-24 21:13:26 -07:00
18bf520ad1
Add eca_nfnet_l2/l3 defs for future training
2021-05-22 21:55:37 -07:00
f45de37690
Merge branch 'master' into levit_visformer_rednet
2021-05-22 16:34:31 -07:00
Ross Wightman
Jakub Kaczmarzyk
Ross Wightman
han
okojoalg
jjsjann123
Li Dong
Junming Chen
SeeFun
Christoph Reich
ayasyrev
Mi-Peng
Hyeongchan Kim
Rahul Somani
KAI ZHAO
talrid
mrT23
qwertyforce
Martins Bruveris
Nathan Raw
Alexander Soare
Thomas Viehmann
masafumi
ICLR Author
leondgarse
Richard Chen
Yohann Lereclus
Sepehr Sameni
Ying Jin
samarth
Dongyoon Han
Mike