Ross Wightman
|
b293dfa595
|
Add CL SE module
|
2022-09-23 16:06:09 -07:00 |
|
Ross Wightman
|
43aa84e861
|
Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)
|
2022-08-17 14:32:58 -07:00 |
|
Ross Wightman
|
307a935b79
|
Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.
|
2021-05-31 13:18:11 -07:00 |
|
Ross Wightman
|
742c2d5247
|
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
|
2021-05-27 18:03:29 -07:00 |