Ross Wightman
|
803254bb40
|
Fix spacing misalignment for fast norm path in LayerNorm modules
|
2022-10-24 21:43:49 -07:00 |
|
Ross Wightman
|
48e1df8b37
|
Add norm/norm_act header comments
|
2022-08-25 16:29:34 -07:00 |
|
Ross Wightman
|
43aa84e861
|
Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)
|
2022-08-17 14:32:58 -07:00 |
|
Ross Wightman
|
eca09b8642
|
Add MobileVitV2 support. Fix #1332. Move GroupNorm1 to common layers (used in poolformer + mobilevitv2). Keep ol custom ConvNeXt LayerNorm2d impl as LayerNormExp2d for reference.
|
2022-07-07 14:41:01 -07:00 |
|
Ross Wightman
|
7a9c6811c9
|
Add eps arg to LayerNorm2d, add 'tf' (tensorflow) variant of trunc_normal_ that applies scale/shift after sampling (instead of needing to move a/b)
|
2022-07-01 15:15:39 -07:00 |
|
Ross Wightman
|
c976a410d9
|
Add ResNet-50 w/ GN (resnet50_gn) and SEBotNet-33-TS (sebotnet33ts_256) model defs and weights. Update halonet50ts weights w/ slightly better variant in1k val, more robust to test sets.
|
2021-11-19 14:24:43 -08:00 |
|
Ross Wightman
|
8165cacd82
|
Realized LayerNorm2d won't work in all cases as is, fixed.
|
2021-07-05 18:21:34 -07:00 |
|
Ross Wightman
|
742c2d5247
|
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
|
2021-05-27 18:03:29 -07:00 |
|
Ross Wightman
|
4e4b863b15
|
Missed norm.py
|
2021-04-12 09:57:56 -07:00 |