Ross Wightman e2b8d44ff0 Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs
* align interfaces of halo, bottleneck attn and lambda layer
* add qk_ratio to all of above, control q/k dim relative to output dim
* add experimental haloregnetz, and trionet (lambda + halo + bottle) models
2021-10-06 16:32:48 -07:00
..
2021-06-23 10:40:30 -07:00
2021-06-07 14:38:30 -07:00
2021-06-20 17:46:06 -07:00
2021-08-27 09:22:20 -07:00