Ross Wightman c02334d9fa Add weights for regnetz_d and haloregnetz_c, update regnetz_c weights. Add commented PyTorch XLA code for halo attention

2021-10-19 12:32:09 -07:00

..

__init__.py

Update attention / self-attn based models from a series of experiments:

2021-08-20 16:13:11 -07:00

activations_jit.py

…

activations_me.py

…

activations.py

…

adaptive_avgmax_pool.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

attention_pool2d.py

Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.

2021-09-05 12:41:14 -07:00

blur_pool.py

…

bottleneck_attn.py

Add option to include relative pos embedding in the attention scaling as per references. See discussion #912

2021-10-12 15:37:01 -07:00

cbam.py

…

classifier.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

cond_conv2d.py

…

config.py

…

conv2d_same.py

…

conv_bn_act.py

…

create_act.py

…

create_attn.py

Update attention / self-attn based models from a series of experiments:

2021-08-20 16:13:11 -07:00

create_conv2d.py

…

create_norm_act.py

…

drop.py

…

eca.py

…

evo_norm.py

…

gather_excite.py

…

global_context.py

Add gc_efficientnetv2_rw_t weights (global context instead of SE attn). Add TF XL weights even though the fine-tuned ones don't validate that well. Change default arg for GlobalContext to use scal (mul) mode.

2021-08-07 16:45:29 -07:00

halo_attn.py

Add weights for regnetz_d and haloregnetz_c, update regnetz_c weights. Add commented PyTorch XLA code for halo attention

2021-10-19 12:32:09 -07:00

helpers.py

…

inplace_abn.py

…

lambda_layer.py

Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs

2021-10-06 16:32:48 -07:00

linear.py

…

median_pool.py

…

mixed_conv2d.py

…

mlp.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

non_local_attn.py

…

norm_act.py

…

norm.py

Realized LayerNorm2d won't work in all cases as is, fixed.

2021-07-05 18:21:34 -07:00

padding.py

…

patch_embed.py

…

pool2d_same.py

Support npz custom load for vision transformer hybrid models. Add posembed rescale for npz load.

2021-06-14 12:31:44 -07:00

selective_kernel.py

…

separable_conv.py

…

space_to_depth.py

…

split_attn.py

…

split_batchnorm.py

…

squeeze_excite.py

…

std_conv.py

Use reshape instead of view in std_conv, causing issues in recent PyTorch in channels_last

2021-09-23 15:43:48 -07:00

test_time_pool.py

…

weight_init.py

…