Ross Wightman a4146b79d1 Add SimpleNorm to create_norm factory

2024-12-30 14:22:42 -08:00

..

__init__.py

Fixing RmsNorm to fix #2380 and noticed with aimv2 when comparing outputs. Still some work to do, need to look at AMP / fast mode behaviour, dispatch to torch when possible. Add SimpleNorm for 'LayerNorm w/o centering and bias'

2024-12-28 21:03:49 -08:00

activations_me.py

…

activations.py

…

adaptive_avgmax_pool.py

…

attention2d.py

…

attention_pool2d.py

…

attention_pool.py

…

blur_pool.py

…

bottleneck_attn.py

…

cbam.py

…

classifier.py

…

cond_conv2d.py

…

config.py

…

conv2d_same.py

…

conv_bn_act.py

…

create_act.py

…

create_attn.py

…

create_conv2d.py

…

create_norm_act.py

…

create_norm.py

Add SimpleNorm to create_norm factory

2024-12-30 14:22:42 -08:00

drop.py

…

eca.py

…

evo_norm.py

…

fast_norm.py

Use torch F.rms_norm when possible, select fast vs normal paths appropriately and test with torchscript

2024-12-29 14:05:07 -08:00

filter_response_norm.py

…

format.py

…

gather_excite.py

…

global_context.py

…

grid.py

…

grn.py

…

halo_attn.py

…

helpers.py

…

hybrid_embed.py

…

inplace_abn.py

…

interpolate.py

…

lambda_layer.py

…

layer_scale.py

…

linear.py

…

median_pool.py

…

mixed_conv2d.py

…

ml_decoder.py

…

mlp.py

Switch aimv2 to used packed SwiGLU

2024-12-28 21:05:38 -08:00

non_local_attn.py

…

norm_act.py

…

norm.py

Use torch F.rms_norm when possible, select fast vs normal paths appropriately and test with torchscript

2024-12-29 14:05:07 -08:00

padding.py

…

patch_dropout.py

…

patch_embed.py

…

pool2d_same.py

…

pos_embed_rel.py

…

pos_embed_sincos.py

…

pos_embed.py

…

selective_kernel.py

…

separable_conv.py

…

space_to_depth.py

…

split_attn.py

…

split_batchnorm.py

…

squeeze_excite.py

…

std_conv.py

…

test_time_pool.py

…

trace_utils.py

…

typing.py

…

weight_init.py

…