76881d207b
Add baseline resnet26t @ 256x256 weights. Add 33ts variant of halonet with at least one halo in stage 2,3,4
2021-09-04 14:52:54 -07:00
484e61648d
Adding the attn series weights, tweaking model names, comments...
2021-09-03 18:09:42 -07:00
8449ba210c
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
2021-08-26 21:56:44 -07:00
925e102982
Update attention / self-attn based models from a series of experiments:
...
* remove dud attention, involution + my swin attention adaptation don't seem worth keeping
* add or update several new 26/50 layer ResNe(X)t variants that were used in experiments
* remove models associated with dead-end or uninteresting experiment results
* weights coming soon...
2021-08-20 16:13:11 -07:00
742c2d5247
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
2021-05-27 18:03:29 -07:00
9a3ae97311
Another set of byoanet models w/ ECA channel + SA + groups
2021-05-15 22:56:51 -07:00
165fb354b2
Add initial RedNet model / Involution layer impl for testing
2021-05-14 17:16:34 -07:00
3ba6b55cb2
More adjustments to ByoaNet models for further experiments.
2021-05-06 13:59:44 -07:00
0721559511
Improved (hopefully) init for SA/SA-like layers used in ByoaNets
2021-05-04 21:40:39 -07:00
9cc7dda6e5
Fixup byoanet configs to pass unit tests. Add swin_attn and swinnet26t model for testing.
2021-04-29 21:08:37 -07:00
e15c3886ba
Defaul lambda r=7. Define '26t' stage 4/5 256x256 variants for all of bot/halo/lambda nets for experiment. Add resnet50t for exp. Fix a few comments.
2021-04-29 10:58:49 -07:00
b3d7580df1
Update ByoaNet comments. Fix first Steam feat chs for ByobNet.
2021-04-12 12:11:35 -07:00
16f7aa9f54
Add default_cfg options for min_input_size / fixed_input_size, queries in model registry, and use for testing self-attn models
2021-04-12 11:54:22 -07:00
ce62f96d4d
ByoaNet with bottleneck transformer, lambda resnet, and halo net experiments
2021-04-12 09:38:02 -07:00
Ross Wightman