Ross Wightman
|
492c0a4e20
|
Update HaloAttn comment
|
2021-09-01 17:14:31 -07:00 |
|
Ross Wightman
|
3b9032ea48
|
Use Tensor.unfold().unfold() for HaloAttn, fast like as_strided but more clarity
|
2021-08-27 12:45:53 -07:00 |
|
Ross Wightman
|
8449ba210c
|
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
|
2021-08-26 21:56:44 -07:00 |
|
Ross Wightman
|
0721559511
|
Improved (hopefully) init for SA/SA-like layers used in ByoaNets
|
2021-05-04 21:40:39 -07:00 |
|
Ross Wightman
|
e15c3886ba
|
Defaul lambda r=7. Define '26t' stage 4/5 256x256 variants for all of bot/halo/lambda nets for experiment. Add resnet50t for exp. Fix a few comments.
|
2021-04-29 10:58:49 -07:00 |
|
Ross Wightman
|
ce62f96d4d
|
ByoaNet with bottleneck transformer, lambda resnet, and halo net experiments
|
2021-04-12 09:38:02 -07:00 |