cc9bedf373
Add initial ResNet Strikes Back weights for ResNet50 and ResNetV2-50 models
2021-10-03 17:32:02 -07:00
64495505b7
Add updated lambda resnet26 and botnet26 checkpoints with fixes applied
2021-10-03 17:31:39 -07:00
b2094f4ee8
support bits checkpoints in avg/load
2021-10-03 17:31:22 -07:00
007bc39323
Some halo and bottleneck attn code cleanup, add halonet50ts weights, use optimal crop ratios
2021-10-02 15:51:42 -07:00
b1c2e3eb92
Match rel_pos_indices attr rename in conv branch
2021-09-30 23:19:05 -07:00
b49630a138
Add relative pos embed option to LambdaLayer, fix last transpose/reshape.
2021-09-30 22:45:09 -07:00
d657e2cc0b
Remove dead code line from efficientnet
2021-09-30 21:54:42 -07:00
0ca687f224
Make 'regnetz' model experiments closer to actual RegNetZ, bottleneck expansion, expand from in_chs, no shortcut on stride 2, tweak model sizes
2021-09-30 21:49:38 -07:00
b81e79aae9
Fix bottleneck attn transpose typo, hopefully these train better now..
2021-09-28 16:38:41 -07:00
80075b0b8a
Add worker_seeding arg to allow selecting old vs updated data loader worker seed for (old) experiment repeatability
2021-09-28 16:37:45 -07:00
6478bcd02c
Fix regnetz_d conv layer name, use inception mean/std
2021-09-26 14:54:17 -07:00
0387e6057e
Update binary cross ent impl to use thresholding as an option (convert soft targets from mixup/cutmix to 0, 1)
2021-09-23 15:45:39 -07:00
f8a63a3b71
Add worker_init_fn to loader for numpy seed per worker
2021-09-23 15:44:38 -07:00
515121cca1
Use reshape instead of view in std_conv, causing issues in recent PyTorch in channels_last
2021-09-23 15:43:48 -07:00
da06cc61d4
ResNetV2 seems to work best without zero_init residual
2021-09-23 15:43:22 -07:00
8e11da0ce3
Add experimental RegNetZ(ish) models for training / perf trials.
2021-09-23 15:42:57 -07:00
6bbc50beb4
make it possible to provide norm_layer via create_model
2021-09-21 10:19:04 +01:00
cf5ac2800c
BotNet models were still off, remove weights for bad configs. Add good SE-HaloNet33-TS weights.
2021-09-13 17:18:59 -07:00
24720abe3b
Merge branch 'master' into attn_update
2021-09-13 16:51:10 -07:00
1c9284c640
Add BeiT 'finetuned' 1k weights and pretrained 22k weights, pretraining specific (masked) model excluded for now
2021-09-13 16:38:23 -07:00
f8a215cfe6
A few more crossvit tweaks, fix training w/ no_weight_decay names, add crop option for scaling, adjust default crop_pct for large img size to 1.0 for better results
2021-09-13 14:17:34 -07:00
7ab2491ab7
Better handling of crossvit for tests / forward_features, fix torchscript regression in my changes
2021-09-13 13:01:05 -07:00
f1808e0970
Post crossvit merge cleanup, change model names to reflect input size, cleanup img size vs scale handling, fix tests
2021-09-13 11:49:54 -07:00
4027412757
Add resnet33ts weights, update resnext26ts baseline weights
2021-09-09 14:46:41 -07:00
9fe5798bee
fix bug for reset classifier and fix for validating the dimension
2021-09-08 21:58:17 -04:00
3718c5a5bd
fix loading pretrained model
2021-09-08 11:53:05 -04:00
bb50b69a57
fix for torch script
2021-09-08 11:20:59 -04:00
5bd04714e4
Cleanup weight init for byob/byoanet and related
2021-09-05 15:34:05 -07:00
8642401e88
Swap botnet 26/50 weights/models after realizing a mistake in arch def, now figuring out why they were so low...
2021-09-05 15:17:19 -07:00
5f12de4875
Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.
2021-09-05 12:41:14 -07:00
76881d207b
Add baseline resnet26t @ 256x256 weights. Add 33ts variant of halonet with at least one halo in stage 2,3,4
2021-09-04 14:52:54 -07:00
484e61648d
Adding the attn series weights, tweaking model names, comments...
2021-09-03 18:09:42 -07:00
fb94350896
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
2021-09-01 17:46:40 -07:00
f262137ff2
Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments.
2021-09-01 17:40:53 -07:00
ba9c1108a1
Add a BCE loss impl that converts dense targets to sparse /w smoothing as an alternate to CE w/ smoothing. For training experiments.
2021-09-01 17:39:28 -07:00
29a37e23ee
LR scheduler update:
...
* add polynomial decay 'poly'
* cleanup cycle specific args for cosine, poly, and tanh sched, t_mul -> cycle_mul, decay -> cycle_decay, default cycle_limit to 1 in each opt
* add k-decay for cosine and poly sched as per https://arxiv.org/abs/2004.05909
* change default tanh ub/lb to push inflection to later epochs
2021-09-01 17:33:11 -07:00
492c0a4e20
Update HaloAttn comment
2021-09-01 17:14:31 -07:00
7ab9d4555c
add crossvit
2021-09-01 17:13:12 -04:00
3b9032ea48
Use Tensor.unfold().unfold() for HaloAttn, fast like as_strided but more clarity
2021-08-27 12:45:53 -07:00
78933122c9
Fix silly typo
2021-08-27 09:22:20 -07:00
2568ffc5ef
Merge branch 'master' into attn_update
2021-08-27 09:21:22 -07:00
708d87a813
Fix ViT SAM weight compat as weights at URL changed to not use repr layer. Fix #825 . Tweak optim test.
2021-08-27 09:20:13 -07:00
8449ba210c
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
2021-08-26 21:56:44 -07:00
a8b65695f1
Add resnet26ts and resnext26ts models for non-attn baselines
2021-08-21 12:42:10 -07:00
a5a542f17d
Fix typo
2021-08-20 17:47:23 -07:00
925e102982
Update attention / self-attn based models from a series of experiments:
...
* remove dud attention, involution + my swin attention adaptation don't seem worth keeping
* add or update several new 26/50 layer ResNe(X)t variants that were used in experiments
* remove models associated with dead-end or uninteresting experiment results
* weights coming soon...
2021-08-20 16:13:11 -07:00
d667351eac
Tweak accuracy topk safety. Fix #807
2021-08-19 14:18:53 -07:00
35c9740826
Fix accuracy when topk > num_classes
2021-08-19 11:58:59 +02:00
a16a753852
Add lamb/lars to optim init imports, remove stray comment
2021-08-18 22:55:02 -07:00
c207e02782
MOAR optimizer changes. Woo!
2021-08-18 22:20:35 -07:00
Ross Wightman
Alexander Soare
Richard Chen
Yohann Lereclus