d633a014e6
Post merge cleanup. Fix potential security issue passing kwargs directly through to serialized web data.
2021-11-23 16:54:01 -08:00
b18c9e323b
Update helpers.py
2021-11-22 23:43:44 -05:00
308d0b9554
Merge branch 'master' into hf-save-and-push
2021-11-22 23:39:27 -05:00
f0507f6da6
Fix k_decay default arg != 1.0 in poly scheduler
2021-11-22 09:37:36 -08:00
41559247e9
use_ml_decoder_head
2021-11-22 17:50:39 +02:00
1f53db2ece
Updated lamhalobotnet weights, 81.5 top-1
2021-11-21 19:49:51 -08:00
15ef108eb4
Add better halo2botnet50ts weights, 82 top-1 @ 256
2021-11-21 14:09:12 -08:00
734b2244fe
Add RegNetZ-D8 (83.5 @ 256, 84 @ 320) and RegNetZ-E8 (84.5 @ 256, 85 @ 320) weights. Update names of existing RegZ models to include group size.
2021-11-20 15:52:04 -08:00
93cc08fdc5
Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).
2021-11-20 15:50:51 -08:00
af607b75cc
Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU
2021-11-19 17:37:00 -08:00
c976a410d9
Add ResNet-50 w/ GN (resnet50_gn) and SEBotNet-33-TS (sebotnet33ts_256) model defs and weights. Update halonet50ts weights w/ slightly better variant in1k val, more robust to test sets.
2021-11-19 14:24:43 -08:00
f2006b2437
Cleanup qkv_bias cat in beit model so it can be traced
2021-11-18 21:25:00 -08:00
1076a65df1
Minor post FX merge cleanup
2021-11-18 19:47:07 -08:00
32c9937dec
Merge branch 'fx-feature-extract-new' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-fx-feature-extract-new
2021-11-18 16:31:29 -08:00
78b36bf46c
Places365 doesn't exist in some still used torchvision version
2021-11-18 14:59:51 -08:00
65d827c7a6
rename notrace registration and standardize trace_utils imports
2021-11-15 21:03:21 +00:00
9b2daf2a35
Add ResNeXt-50 weights 81.1 top-1 @ 224, 82 @ 288 with A1 'high aug' recipe
2021-11-14 13:17:27 -08:00
9b5d6dc7e2
Merge branch 'add-vit-b8' of https://github.com/martinsbruveris/pytorch-image-models into martinsbruveris-add-vit-b8
2021-11-14 12:54:28 -08:00
cfa414cad2
Matching two bits_and_tpu changes for TFDs wrapper
...
* change 'samples' -> 'examples' for tfds wrapper to match tfds naming
* add class_to_idx for image classification datasets in tfds wrapper
2021-11-14 12:52:19 -08:00
5220711d87
Added B/8 models to ViT.
2021-11-14 11:01:48 +00:00
0262a0e8e1
fx ready for review
2021-11-13 00:06:33 +00:00
d2994016e9
Add try/except guards
2021-11-12 21:16:53 +00:00
b25ff96768
wip - pre-rebase
2021-11-12 20:45:05 +00:00
e051dce354
Make all models FX traceable
2021-11-12 20:45:05 +00:00
cf4561ca72
Add FX based FeatureGraphNet capability
2021-11-12 20:45:05 +00:00
0149ec30d7
wip - attempting to rebase
2021-11-12 20:45:05 +00:00
02c3a75a45
wip - make it possible to use fx graph in train and eval mode
2021-11-12 20:45:05 +00:00
bc3d4eb403
wip -rebase
2021-11-12 20:45:05 +00:00
ab3ac3f25b
Add FX based FeatureGraphNet capability
2021-11-12 20:45:05 +00:00
9ec3210c2d
More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).
2021-11-10 15:52:09 -08:00
ba65dfe2c6
Dataset work
...
* support some torchvision datasets
* improvements to TFDS wrapper for subsplit handling (fix #942 ), shuffle seed
* add class-map support to train (fix #957 )
2021-11-09 22:34:15 -08:00
ddc29da974
Add ResNet101 and ResNet152 weights from higher aug RSB recipes. 81.93 and 82.82 top-1 at 224x224.
2021-11-02 17:59:16 -07:00
b328e56f49
Update eca_halonext26ts weights to a better set
2021-11-02 16:52:53 -07:00
2ddef942b9
Better fix for #954 that doesn't break torchscript, pull torch._assert into timm namespace when it exists
2021-11-02 11:22:33 -07:00
4f0f9cb348
Fix #954 by bringing traceable _assert into timm to allow compat w/ PyTorch < 1.8
2021-11-02 09:21:40 -07:00
a41de1f666
Add interpolation mode handling to transforms. Removes InterpolationMode warning. Works for torchvision versions w/ and w/o InterpolationMode enum. Fix #738 .
2021-10-28 17:35:01 -07:00
ed41d32637
Add repr to auto_augment and random_erasing impl
2021-10-28 17:33:36 -07:00
ae72d009fa
Add weights for lambda_resnet50ts, halo2botnet50ts, lamhalobotnet50ts, updated halonet50ts
2021-10-27 22:08:54 -07:00
b745d30a3e
Fix formatting of last commit
2021-10-25 15:15:14 -07:00
3478f1d7f1
Traceability fix for vit models for some experiments
2021-10-25 15:13:08 -07:00
f658a72e72
Cleanup re-use of Dropout modules in Mlp modules after some twitter feedback :p
2021-10-25 00:40:59 -07:00
f805ba86d9
use .unbind instead of explicitly listing the indices
2021-10-24 21:08:47 +02:00
57992509f9
Fix some formatting in utils/model.py
2021-10-23 20:35:36 -07:00
0fe4fd3f1f
add d8 and e8 regnetz models with group size 8
2021-10-23 20:34:21 -07:00
25e7c8c5e5
Update broken resnetv2_50 weight url, add resnetv1_101 a1h recipe weights for 224x224 train
2021-10-20 22:14:12 -07:00
b6caa356d2
Fixed eca_botnext26ts_256 weights added, 79.27
2021-10-19 12:44:28 -07:00
c02334d9fa
Add weights for regnetz_d and haloregnetz_c, update regnetz_c weights. Add commented PyTorch XLA code for halo attention
2021-10-19 12:32:09 -07:00
02daf2ab94
Add option to include relative pos embedding in the attention scaling as per references. See discussion #912
2021-10-12 15:37:01 -07:00
047a5ec05f
Fix bugs that Mixup does not work device=cpu
2021-10-12 23:51:46 +09:00
cd34913278
Remove some outdated comments, botnet networks working great now.
2021-10-11 22:43:41 -07:00
6ed4cdccca
Update lambda_resnet26t weights with better set
2021-10-10 16:32:54 -07:00
44d6d51668
Add ConvMixer
2021-10-09 21:09:51 -04:00
a85df34993
Update lambda_resnet26rpt weights to 78.9, add better halonet26t weights at 79.1 with tweak to attention dim
2021-10-08 17:44:13 -07:00
b544ad4d3f
regnetz model default cfg tweaks
2021-10-06 21:14:59 -07:00
e5da481073
Small post-merge tweak for freeze/unfreeze, add to __init__ for utils
2021-10-06 17:00:27 -07:00
5ca72dcc75
Merge branch 'freeze-functionality' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-freeze-functionality
2021-10-06 16:51:03 -07:00
e2b8d44ff0
Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs
...
* align interfaces of halo, bottleneck attn and lambda layer
* add qk_ratio to all of above, control q/k dim relative to output dim
* add experimental haloregnetz, and trionet (lambda + halo + bottle) models
2021-10-06 16:32:48 -07:00
431e60c83f
Add acknowledgements for freeze_batch_norm inspiration
2021-10-06 14:28:49 +01:00
fbf59c04ee
Change crop ratio on correct resnet50 variant.
2021-10-04 22:31:08 -07:00
ae1ff5792f
Clean a1/a2/3 rsb _0 checkpoints properly, fix v2 loading.
2021-10-04 16:46:00 -07:00
93901e992f
Version bump to 0.5.0 for pending release post RSB and ATTN updates
2021-10-03 17:34:57 -07:00
da0d39bedd
Update default crop_pct for byoanet
2021-10-03 17:33:16 -07:00
cc9bedf373
Add initial ResNet Strikes Back weights for ResNet50 and ResNetV2-50 models
2021-10-03 17:32:02 -07:00
64495505b7
Add updated lambda resnet26 and botnet26 checkpoints with fixes applied
2021-10-03 17:31:39 -07:00
b2094f4ee8
support bits checkpoints in avg/load
2021-10-03 17:31:22 -07:00
007bc39323
Some halo and bottleneck attn code cleanup, add halonet50ts weights, use optimal crop ratios
2021-10-02 15:51:42 -07:00
65c3d78b96
Freeze unfreeze functionality finalized. Tests added
2021-10-02 15:55:08 +01:00
0cb8ea432c
wip
2021-10-02 15:55:08 +01:00
b1c2e3eb92
Match rel_pos_indices attr rename in conv branch
2021-09-30 23:19:05 -07:00
b49630a138
Add relative pos embed option to LambdaLayer, fix last transpose/reshape.
2021-09-30 22:45:09 -07:00
d657e2cc0b
Remove dead code line from efficientnet
2021-09-30 21:54:42 -07:00
0ca687f224
Make 'regnetz' model experiments closer to actual RegNetZ, bottleneck expansion, expand from in_chs, no shortcut on stride 2, tweak model sizes
2021-09-30 21:49:38 -07:00
51eaf9360d
Remove a duplicate layer creation in byobnet.py
...
`self.conv2_kxk` is repeated in `byobnet.py`. Remove the duplicate code.
2021-09-30 18:30:48 +08:00
b81e79aae9
Fix bottleneck attn transpose typo, hopefully these train better now..
2021-09-28 16:38:41 -07:00
80075b0b8a
Add worker_seeding arg to allow selecting old vs updated data loader worker seed for (old) experiment repeatability
2021-09-28 16:37:45 -07:00
6478bcd02c
Fix regnetz_d conv layer name, use inception mean/std
2021-09-26 14:54:17 -07:00
0387e6057e
Update binary cross ent impl to use thresholding as an option (convert soft targets from mixup/cutmix to 0, 1)
2021-09-23 15:45:39 -07:00
f8a63a3b71
Add worker_init_fn to loader for numpy seed per worker
2021-09-23 15:44:38 -07:00
515121cca1
Use reshape instead of view in std_conv, causing issues in recent PyTorch in channels_last
2021-09-23 15:43:48 -07:00
da06cc61d4
ResNetV2 seems to work best without zero_init residual
2021-09-23 15:43:22 -07:00
8e11da0ce3
Add experimental RegNetZ(ish) models for training / perf trials.
2021-09-23 15:42:57 -07:00
6bbc50beb4
make it possible to provide norm_layer via create_model
2021-09-21 10:19:04 +01:00
adcb74f87f
🎨 Import load_state_dict_from_url directly
2021-09-14 01:11:40 -04:00
e65a2cba3d
🎨 cleanup and add a couple comments
2021-09-14 01:07:04 -04:00
2b6ade24b3
🎨 write model card to enable inference
2021-09-13 23:31:28 -04:00
cf5ac2800c
BotNet models were still off, remove weights for bad configs. Add good SE-HaloNet33-TS weights.
2021-09-13 17:18:59 -07:00
24720abe3b
Merge branch 'master' into attn_update
2021-09-13 16:51:10 -07:00
1c9284c640
Add BeiT 'finetuned' 1k weights and pretrained 22k weights, pretraining specific (masked) model excluded for now
2021-09-13 16:38:23 -07:00
f8a215cfe6
A few more crossvit tweaks, fix training w/ no_weight_decay names, add crop option for scaling, adjust default crop_pct for large img size to 1.0 for better results
2021-09-13 14:17:34 -07:00
7ab2491ab7
Better handling of crossvit for tests / forward_features, fix torchscript regression in my changes
2021-09-13 13:01:05 -07:00
f1808e0970
Post crossvit merge cleanup, change model names to reflect input size, cleanup img size vs scale handling, fix tests
2021-09-13 11:49:54 -07:00
4027412757
Add resnet33ts weights, update resnext26ts baseline weights
2021-09-09 14:46:41 -07:00
9fe5798bee
fix bug for reset classifier and fix for validating the dimension
2021-09-08 21:58:17 -04:00
3718c5a5bd
fix loading pretrained model
2021-09-08 11:53:05 -04:00
bb50b69a57
fix for torch script
2021-09-08 11:20:59 -04:00
abf9d51bc3
🚧 wip
2021-09-07 18:39:26 -06:00
5bd04714e4
Cleanup weight init for byob/byoanet and related
2021-09-05 15:34:05 -07:00
8642401e88
Swap botnet 26/50 weights/models after realizing a mistake in arch def, now figuring out why they were so low...
2021-09-05 15:17:19 -07:00
5f12de4875
Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.
2021-09-05 12:41:14 -07:00
76881d207b
Add baseline resnet26t @ 256x256 weights. Add 33ts variant of halonet with at least one halo in stage 2,3,4
2021-09-04 14:52:54 -07:00
484e61648d
Adding the attn series weights, tweaking model names, comments...
2021-09-03 18:09:42 -07:00
fb94350896
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
2021-09-01 17:46:40 -07:00
f262137ff2
Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments.
2021-09-01 17:40:53 -07:00
ba9c1108a1
Add a BCE loss impl that converts dense targets to sparse /w smoothing as an alternate to CE w/ smoothing. For training experiments.
2021-09-01 17:39:28 -07:00
29a37e23ee
LR scheduler update:
...
* add polynomial decay 'poly'
* cleanup cycle specific args for cosine, poly, and tanh sched, t_mul -> cycle_mul, decay -> cycle_decay, default cycle_limit to 1 in each opt
* add k-decay for cosine and poly sched as per https://arxiv.org/abs/2004.05909
* change default tanh ub/lb to push inflection to later epochs
2021-09-01 17:33:11 -07:00
28d2841acf
💄 apply isort
2021-09-01 18:15:08 -06:00
492c0a4e20
Update HaloAttn comment
2021-09-01 17:14:31 -07:00
e72c989973
✨ add ability to push to hf hub
2021-09-01 18:14:28 -06:00
7ab9d4555c
add crossvit
2021-09-01 17:13:12 -04:00
3b9032ea48
Use Tensor.unfold().unfold() for HaloAttn, fast like as_strided but more clarity
2021-08-27 12:45:53 -07:00
78933122c9
Fix silly typo
2021-08-27 09:22:20 -07:00
2568ffc5ef
Merge branch 'master' into attn_update
2021-08-27 09:21:22 -07:00
708d87a813
Fix ViT SAM weight compat as weights at URL changed to not use repr layer. Fix #825 . Tweak optim test.
2021-08-27 09:20:13 -07:00
8449ba210c
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
2021-08-26 21:56:44 -07:00
a8b65695f1
Add resnet26ts and resnext26ts models for non-attn baselines
2021-08-21 12:42:10 -07:00
a5a542f17d
Fix typo
2021-08-20 17:47:23 -07:00
925e102982
Update attention / self-attn based models from a series of experiments:
...
* remove dud attention, involution + my swin attention adaptation don't seem worth keeping
* add or update several new 26/50 layer ResNe(X)t variants that were used in experiments
* remove models associated with dead-end or uninteresting experiment results
* weights coming soon...
2021-08-20 16:13:11 -07:00
d667351eac
Tweak accuracy topk safety. Fix #807
2021-08-19 14:18:53 -07:00
35c9740826
Fix accuracy when topk > num_classes
2021-08-19 11:58:59 +02:00
a16a753852
Add lamb/lars to optim init imports, remove stray comment
2021-08-18 22:55:02 -07:00
c207e02782
MOAR optimizer changes. Woo!
2021-08-18 22:20:35 -07:00
a426511c95
More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.
2021-08-18 17:21:56 -07:00
9541f4963b
One more scalar -> tensor fix for lamb optimizer
2021-08-18 11:20:25 -07:00
8f68193c91
Update lamp.py comment
2021-08-18 09:27:40 -07:00
4d284017b8
Merge pull request #813 from rwightman/opt_cleanup
...
Optimizer cleanup and additions
2021-08-18 09:12:00 -07:00
a6af48be64
add madgradw optimizer
2021-08-17 22:19:27 -07:00
55fb5eedf6
Remove experiment from lamb impl
2021-08-17 21:48:26 -07:00
8a9eca5157
A few optimizer comments, dead import, missing import
2021-08-17 18:01:33 -07:00
ac469b50da
Optimizer improvements, additions, cleanup
...
* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad
2021-08-17 17:51:20 -07:00
abf3e044bb
Update scheduler_factory.py
...
remove duplicate code from create_scheduler()
2021-08-14 22:53:17 +02:00
3cdaf5ed56
Add `mmax` config key to auto_augment for increasing upper bound of RandAugment magnitude beyond 10. Make AugMix uniform sampling default not override config setting.
2021-08-12 15:39:05 -07:00
1042b8a146
Add non fused LAMB optimizer option
2021-08-09 13:13:43 -07:00
01cb46a9a5
Add gc_efficientnetv2_rw_t weights (global context instead of SE attn). Add TF XL weights even though the fine-tuned ones don't validate that well. Change default arg for GlobalContext to use scal (mul) mode.
2021-08-07 16:45:29 -07:00
d3f7440650
Add EfficientNetV2 XL model defs
2021-07-22 13:15:24 -07:00
72b227dcf5
Merge pull request #750 from drjinying/master
...
Specify "interpolation" mode in vision_transformer's resize_pos_embed
2021-07-13 11:01:20 -07:00
2907c1f967
Merge pull request #746 from samarth4149/master
...
Adding a Multi Step LR Scheduler
2021-07-13 10:55:54 -07:00
748ab852ca
Allow act_layer switch for xcit, fix in_chans for some variants
2021-07-12 13:27:29 -07:00
20b2d4b69d
Use bicubic interpolation in resize_pos_embed()
2021-07-12 10:38:31 -07:00
d3255adf8e
Merge branch 'xcit' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-xcit
2021-07-12 08:30:30 -07:00
f8039c7492
Fix gc effv2 model cfg name
2021-07-11 12:14:31 -07:00
3a55a30ed1
add notes from author
2021-07-11 14:25:58 +01:00
899cf84ccc
bug fix - missing _dist postfix for many of the 224_dist models
2021-07-11 12:41:51 +01:00
623e8b8eb8
wip xcit
2021-07-11 09:39:38 +01:00
392368e210
Add efficientnetv2_rw_t defs w/ weights, and gc variant, as well as gcresnet26ts for experiments. Version 0.4.13
2021-07-09 16:46:52 -07:00
daab57a6d9
1. Added a simple multi step LR scheduler
2021-07-09 16:18:27 -04:00
6d8272e92c
Add SAM pretrained model defs/weights for ViT B16 and B32 models.
2021-07-08 11:51:12 -07:00
ee4d8fc69a
Remove unecessary line from nest post refactor
2021-07-05 21:22:46 -07:00
8165cacd82
Realized LayerNorm2d won't work in all cases as is, fixed.
2021-07-05 18:21:34 -07:00
81cd6863c8
Move aggregation (convpool) for nest into NestLevel, cleanup and enable features_only use. Finalize weight url.
2021-07-05 18:20:49 -07:00
6ae0ac6420
Merge branch 'nested_transformer' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-nested_transformer
2021-07-03 12:45:26 -07:00
7b8a0017f1
wip to review
2021-07-03 12:10:12 +01:00
b11d949a06
wip checkpoint with some feature extraction work
2021-07-03 11:45:19 +01:00
23bb72ce5e
nested_transformer wip
2021-07-02 20:12:29 +01:00
766b4d3262
Fix features for resnetv2_50t
2021-06-28 15:56:24 -07:00
e8045e712f
Fix BatchNorm for ResNetV2 non GN models, add more ResNetV2 model defs for future experimentation, fix zero_init of last residual for pre-act.
2021-06-28 10:52:45 -07:00
20a2be14c3
Add gMLP-S weights, 79.6 top-1
2021-06-23 10:40:30 -07:00
85f894e03d
Fix ViT in21k representation (pre_logits) layer handling across old and new npz checkpoints
2021-06-23 10:38:34 -07:00
b41cffaa93
Fix a few issues loading pretrained vit/bit npz weights w/ num_classes=0 __init__ arg. Missed a few other small classifier handling detail on Mlp, GhostNet, Levit. Should fix #713
2021-06-22 23:16:05 -07:00
9c9755a808
AugReg release
2021-06-20 17:46:06 -07:00
381b279785
Add hybrid model fwds back
2021-06-19 22:28:44 -07:00
26f04a8e3e
Fix a weight link
2021-06-19 16:39:36 -07:00
8f4a0222ed
Add GMixer-24 MLP model weights, trained w/ TPU + PyTorch XLA
2021-06-18 16:49:28 -07:00
4c09a2f169
Bump version 0.4.12
2021-06-18 16:17:34 -07:00
b319eb5b5d
Update ViT weights, more details to be added before merge.
2021-06-18 16:16:49 -07:00
8257b86550
Fix up resnetv2 bit/bitm model default res
2021-06-18 16:16:06 -07:00
1228f5a3d8
Add BiT distilled 50x1 and teacher 152x2 models from 'A good teacher is patient and consistent' paper.
2021-06-18 11:40:33 -07:00
511a8e8c96
Add official ResMLP weights.
2021-06-14 17:03:16 -07:00
b9cfb64412
Support npz custom load for vision transformer hybrid models. Add posembed rescale for npz load.
2021-06-14 12:31:44 -07:00
8319e0c373
Add file docstring to std_conv.py
2021-06-13 12:31:06 -07:00
4d96165989
Merge branch 'master' into cleanup_xla_model_fixes
2021-06-12 23:19:25 -07:00
8880f696b6
Refactoring, cleanup, improved test coverage.
...
* Add eca_nfnet_l2 weights, 84.7 @ 384x384
* All 'non-std' (ie transformer / mlp) models have classifier / default_cfg test added
* Fix #694 reset_classifer / num_features / forward_features / num_classes=0 consistency for transformer / mlp models
* Add direct loading of npz to vision transformer (pure transformer so far, hybrid to come)
* Rename vit_deit* to deit_*
* Remove some deprecated vit hybrid model defs
* Clean up classifier flatten for conv classifiers and unusual cases (mobilenetv3/ghostnet)
* Remove explicit model fns for levit conv, just pass in arg
2021-06-12 16:40:02 -07:00
ba2ca4b464
One codepath for stdconv, switch layernorm to batchnorm so gain included. Tweak epsilon values for nfnet, resnetv2, vit hybrid.
2021-06-12 12:27:43 -07:00
b7a568f065
Fix torchscript issue in bat
2021-06-08 23:19:51 -07:00
d17b374f0f
Minimum input_size needed to be higher
2021-06-08 21:31:39 -07:00
b3b90d944d
Add min_input_size to bat_resnext to prevent test breakage.
2021-06-08 17:32:08 -07:00
d413eef1bf
Add ResMLP-24 model weights that I trained in PyTorch XLA on TPU-VM. 79.2 top-1.
2021-06-08 14:22:05 -07:00
10d8fa4620
Add gc and bat attention resnext26ts variants to byob for test.
2021-06-08 14:21:07 -07:00
2f5ed2dec1
Update `init_values` const for 24 and 36 layer ResMLP models
2021-06-07 17:15:04 -07:00
8e4ac3549f
All ScaledStdConv and StdConv uses default to using F.layernorm so that they work with PyTorch XLA. eps value tweaking is a WIP.
2021-06-07 17:14:19 -07:00
2a63d0246b
Post merge cleanup
2021-06-07 14:38:30 -07:00
45dec179e5
Merge pull request #681 from lmk123568/master
...
Update convit.py
2021-06-07 14:10:53 -07:00
ded1671483
Fix stochastic depth working only with a shortcut
2021-06-07 23:08:55 +09:00
b87d98b238
Update convit.py
...
Cut out the duplicates
2021-06-06 17:58:31 +08:00
02320c3e3d
Bump version to 0.4.11
2021-05-31 15:41:51 -07:00
bda8ab015a
Remove min channels for SelectiveKernel, divisor should cover cases well enough.
2021-05-31 15:38:56 -07:00
a27f4aec4a
Missed args for skresnext w/ refactoring.
2021-05-31 14:06:34 -07:00
307a935b79
Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.
2021-05-31 13:18:11 -07:00
8bf63b6c6c
Able to use other attn layer in EfficientNet now. Create test ECA + GC B0 configs. Make ECA more configurable.
2021-05-30 12:47:02 -07:00
bcec14d3b5
Bring EfficientNet SE layer in line with others, pull se_ratio outside of blocks. Allows swapping w/ other attn layers.
2021-05-29 23:41:38 -07:00
9611458e19
Throw in some FBNetV3 code I had lying around, some refactoring of SE reduction channel calcs for all EffNet archs.
2021-05-28 20:47:24 -07:00
01b9108619
Merge branch 'master' into more_attn
2021-05-28 11:09:37 -07:00
d7bab8a6c5
Fix strict flag change for checkpoint load.
2021-05-28 09:54:50 -07:00
02f9d4bc34
Add weights for resnet51q model, add 61q def.
2021-05-28 09:53:16 -07:00
f615474be3
Fix broken test, repvgg block doesn't have attn_last attr.
2021-05-27 18:12:22 -07:00
742c2d5247
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
2021-05-27 18:03:29 -07:00
9c78de8c02
Fix #661 , move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present.
2021-05-26 15:28:42 -07:00
5db7452173
Fix visformer in_chans stem handling
2021-05-25 14:11:36 -07:00
318360c3f9
Update README.md before merge. Bump version to 0.4.10
2021-05-25 12:26:16 -07:00
11ae795e99
Redo LeViT attention bias caching in a way that works with both torchscript and DataParallel
2021-05-25 10:15:32 -07:00
d400f1dbdd
Filter test models before creation for backward/torchscript tests
2021-05-25 10:14:45 -07:00
c4572cc5aa
Add Visformer-small weighs, tweak torchscript jit test img size.
2021-05-24 22:50:12 -07:00
bfc72f75d3
Expand scope of testing for non-std vision transformer / mlp models. Some related cleanup and create fn cleanup for all vision transformer and mlp models. More CoaT weights.
2021-05-24 21:13:26 -07:00
18bf520ad1
Add eca_nfnet_l2/l3 defs for future training
2021-05-22 21:55:37 -07:00
f45de37690
Merge branch 'master' into levit_visformer_rednet
2021-05-22 16:34:31 -07:00
23c18a33e4
Add efficientnetv2_rw_m weights trained in PyTorch. 84.8 top-1 @ 416 test. 53M params.
2021-05-21 21:16:25 -07:00
c2ba229d99
Prep for effcientnetv2_rw_m model weights that started training before official release..
2021-05-21 17:47:49 -07:00
30b9880d06
Minor adjustment, mutable default arg, extra check of valid len...
2021-05-21 17:20:51 -07:00
be0abfbcce
Merge branch 'master' of https://github.com/alexander-soare/pytorch-image-models into alexander-soare-master
2021-05-21 17:10:11 -07:00
b7de82e835
ConViT cleanup, fix torchscript, bit of reformatting, reuse existing layers.
2021-05-21 17:04:23 -07:00
306c86b668
Merge branch 'convit' of https://github.com/amaarora/pytorch-image-models into amaarora-convit
2021-05-21 16:27:10 -07:00
a569635045
Update twin weights to a copy in GitHub releases for faster dl. Tweak model class comment.
2021-05-21 16:23:14 -07:00
be99eef9c1
Remove redundant code, cleanup, fix torchscript.
2021-05-20 23:38:35 -07:00
5ab372a3ec
Merge branch 'master' of https://github.com/abcdvzz/pytorch-image-models into abcdvzz-master
2021-05-20 23:37:50 -07:00
5db1eb6ba5
Add defaults
2021-05-21 02:11:20 +00:00
8b1f2e8e1f
remote unused matplotlib import
2021-05-20 23:42:42 +00:00
40c506ba1e
Add ConViT
2021-05-20 23:17:28 +00:00
7976019864
extend positional embedding resizing functionality to tnt
2021-05-20 11:55:48 +01:00
8086943b6f
allow resize positional embeddings to non-square grid
2021-05-20 11:27:58 +01:00
dc1a4efd28
mixer_b16_224_miil, mixer_b16_224_miil_in21k models
2021-05-20 10:35:50 +03:00
7b799c4e79
add latest code
2021-05-20 11:15:49 +08:00
d5af752117
Add preliminary gMLP and ResMLP impl to Mlp-Mixer
2021-05-19 09:55:05 -07:00
00548b8427
Add Twins
2021-05-18 19:21:53 +08:00
e7f0db8664
Fix drop/drop_path arg on MLP-Mixer model. Fix #641
2021-05-16 08:31:52 -07:00
9a3ae97311
Another set of byoanet models w/ ECA channel + SA + groups
2021-05-15 22:56:51 -07:00
d53e91218e
Fix tf.data options setting for newer TF versions
2021-05-15 22:56:12 -07:00
7077f16c6a
Change 21k model naming from _21k to _in21k for consistency with existing 21k models.
2021-05-15 12:42:26 -07:00
94d4b53352
Add temporary default_cfgs to visformer models so they pass tests
2021-05-15 08:41:31 -07:00
3bffc701f1
Merge branch 'master' into levit_visformer_rednet
2021-05-14 23:02:12 -07:00
ecc7552c5c
Add levit, levit_c, and visformer model defs. Largely untested and not finished cleanup.
2021-05-14 17:16:34 -07:00
165fb354b2
Add initial RedNet model / Involution layer impl for testing
2021-05-14 17:16:34 -07:00
328249f11a
Update README, tweak fine-tune effv2 model names.
2021-05-14 16:37:43 -07:00
c4f482a08b
EfficientNetV2 official impl w/ weights ported from TF. Cleanup/refactor of related EfficientNet classes and models.
2021-05-14 15:50:00 -07:00
4fbc32d3d0
Fix crop_pct for cait models.
2021-05-11 10:50:15 -07:00
715519a5ef
Rethink name of patch embed grid info
2021-05-06 14:08:20 -07:00
b2c305c2aa
Move Mlp and PatchEmbed modules into layers. Being used in lots of models now...
2021-05-06 14:03:23 -07:00
3ba6b55cb2
More adjustments to ByoaNet models for further experiments.
2021-05-06 13:59:44 -07:00
5fcddb96a8
Merge branch 'master' into cait
2021-05-05 17:29:38 -07:00
3db12b4b6a
Finish CaiT cleanup
2021-05-05 17:28:19 -07:00
2d8b09fe8b
Add official pretrained weights to MLP-Mixer, complete model cfgs.
2021-05-05 15:59:40 -07:00
12efffa6b1
Initial MLP-Mixer attempt...
2021-05-05 00:59:45 -07:00
0721559511
Improved (hopefully) init for SA/SA-like layers used in ByoaNets
2021-05-04 21:40:39 -07:00
d5473c17f7
Fix incorrect name of shortcut/identity paths in many residual nets. Inherited from naming in old old torchvision, long fixed there.
2021-05-04 21:27:15 -07:00
0d87650fea
Remove filter hack from BlurPool w/ non-persistent buffer. Use BlurPool2d instead of AntiAliasing.. for TResNet. Breaks PyTorch < 1.6.
2021-05-04 16:56:28 -07:00
ddc743fdf8
Update ResNet-RS models to EMA weights
2021-05-04 16:16:55 -07:00
08d60f4a9a
resnetrs50 pool sizing wrong
2021-05-04 12:41:09 -07:00
1daa15ecc3
Initial Cait commit. Still some cleanup to do.
2021-05-04 11:19:27 -07:00
67d0665b46
Post ResNet-RS merge cleanup. Add weight urls, adjust train/test/crop pct.
2021-05-04 11:04:23 -07:00
560eae38f5
[WIP] Add ResNet-RS models ( #554 )
...
* Add ResNet-RS models
* Only include resnet-rs changes
* remove whitespace diff
* EOF newline
* Update time
* increase time
* Add first conv
* Try running only resnetv2_101x1_bitm on Linux runner
* Add to exclude filter
* Run test_model_forward_features for all
* Add to exclude ftrs
* back to defaults
* only run test_forward_features
* run all tests
* Run all tests
* Add bigger resnetrs to model filters to fix Github CLI
* Remove resnetv2_101x1_bitm from exclude feat features
* Remove hardcoded values
* Make sure reduction ratio in resnetrs is 0.25
* There is no bias in replaced maxpool so remove it
2021-05-04 10:59:44 -07:00
9cc7dda6e5
Fixup byoanet configs to pass unit tests. Add swin_attn and swinnet26t model for testing.
2021-04-29 21:08:37 -07:00
e15c3886ba
Defaul lambda r=7. Define '26t' stage 4/5 256x256 variants for all of bot/halo/lambda nets for experiment. Add resnet50t for exp. Fix a few comments.
2021-04-29 10:58:49 -07:00
Ross Wightman
Nathan Raw
talrid
Alexander Soare
Martins Bruveris
Thomas Viehmann
masafumi
ICLR Author
leondgarse
Richard Chen
Yohann Lereclus
Ross Wightman
Sepehr Sameni
Ying Jin
samarth
Dongyoon Han
Mike
Aman Arora
李鑫杰
Aman Arora