Ross Wightman e2b8d44ff0 Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs ...

* align interfaces of halo, bottleneck attn and lambda layer
* add qk_ratio to all of above, control q/k dim relative to output dim
* add experimental haloregnetz, and trionet (lambda + halo + bottle) models

2021-10-06 16:32:48 -07:00

..

__init__.py

Update attention / self-attn based models from a series of experiments:

2021-08-20 16:13:11 -07:00

activations_jit.py

Fix some attributions, add copyrights to some file docstrings

2020-07-27 13:44:56 -07:00

activations_me.py

Merge pull request #282 from tigert1998/patch-1

2021-02-04 12:18:40 -08:00

activations.py

Fix inplace arg compat for GELU and PreLU via activation factory

2020-11-30 13:27:40 -08:00

adaptive_avgmax_pool.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

attention_pool2d.py

Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.

2021-09-05 12:41:14 -07:00

blur_pool.py

Remove filter hack from BlurPool w/ non-persistent buffer. Use BlurPool2d instead of AntiAliasing.. for TResNet. Breaks PyTorch < 1.6.

2021-05-04 16:56:28 -07:00

bottleneck_attn.py

Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs

2021-10-06 16:32:48 -07:00

cbam.py

Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.

2021-05-27 18:03:29 -07:00

classifier.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

cond_conv2d.py

Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers.

2020-10-13 13:33:44 -07:00

config.py

Add set_layer_config contextmgr to adjust all layer configs at once, use in create_module with new args. Remove a few old warning causing constant annotations for jit.

2020-06-02 21:06:10 -07:00

conv2d_same.py

Fix some attributions, add copyrights to some file docstrings

2020-07-27 13:44:56 -07:00

conv_bn_act.py

Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.

2021-02-09 16:22:52 -08:00

create_act.py

Fix #661, move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present.

2021-05-26 15:28:42 -07:00

create_attn.py

Update attention / self-attn based models from a series of experiments:

2021-08-20 16:13:11 -07:00

create_conv2d.py

Use in_channels for depthwise groups, allows using out_channels=N * in_channels (does not impact existing models). Fix #354.

2021-02-09 16:22:52 -08:00

create_norm_act.py

Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.

2021-02-09 16:22:52 -08:00

drop.py

Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers.

2020-10-13 13:33:44 -07:00

eca.py

Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.

2021-05-31 13:18:11 -07:00

evo_norm.py

Fix a silly bug in Sample version of EvoNorm missing x* part of swish, update EvoNormBatch to accumulated unbiased variance.

2020-08-13 18:25:01 -07:00

gather_excite.py

Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.

2021-05-27 18:03:29 -07:00

global_context.py

Add gc_efficientnetv2_rw_t weights (global context instead of SE attn). Add TF XL weights even though the fine-tuned ones don't validate that well. Change default arg for GlobalContext to use scal (mul) mode.

2021-08-07 16:45:29 -07:00

halo_attn.py

Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs

2021-10-06 16:32:48 -07:00

helpers.py

Throw in some FBNetV3 code I had lying around, some refactoring of SE reduction channel calcs for all EffNet archs.

2021-05-28 20:47:24 -07:00

inplace_abn.py

Update README, fix iabn pip version print.

2021-03-07 16:17:06 -08:00

lambda_layer.py

Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs

2021-10-06 16:32:48 -07:00

linear.py

A few more changes for 0.3.2 maint release. Linear layer change for mobilenetv3 and inception_v3, support no bias for linear wrapper.

2020-11-30 16:19:52 -08:00

median_pool.py

Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers.

2020-10-13 13:33:44 -07:00

mixed_conv2d.py

Use in_channels for depthwise groups, allows using out_channels=N * in_channels (does not impact existing models). Fix #354.

2021-02-09 16:22:52 -08:00

mlp.py

Refactoring, cleanup, improved test coverage.

2021-06-12 16:40:02 -07:00

non_local_attn.py

Fix torchscript issue in bat

2021-06-08 23:19:51 -07:00

norm_act.py

Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.

2021-02-09 16:22:52 -08:00

norm.py

Realized LayerNorm2d won't work in all cases as is, fixed.

2021-07-05 18:21:34 -07:00

padding.py

Fix some attributions, add copyrights to some file docstrings

2020-07-27 13:44:56 -07:00

patch_embed.py

Add levit, levit_c, and visformer model defs. Largely untested and not finished cleanup.

2021-05-14 17:16:34 -07:00

pool2d_same.py

Support npz custom load for vision transformer hybrid models. Add posembed rescale for npz load.

2021-06-14 12:31:44 -07:00

selective_kernel.py

Remove min channels for SelectiveKernel, divisor should cover cases well enough.

2021-05-31 15:38:56 -07:00

separable_conv.py

Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.

2021-02-09 16:22:52 -08:00

space_to_depth.py

Remove some CRLF that made it into TResNet merge

2020-04-27 13:23:41 -07:00

split_attn.py

Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.

2021-05-31 13:18:11 -07:00

split_batchnorm.py

Fix some attributions, add copyrights to some file docstrings

2020-07-27 13:44:56 -07:00

squeeze_excite.py

Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.

2021-05-31 13:18:11 -07:00

std_conv.py

Use reshape instead of view in std_conv, causing issues in recent PyTorch in channels_last

2021-09-23 15:43:48 -07:00

test_time_pool.py

Add new weights for ecaresnet26t/50t/269d models. Remove distinction between 't' and 'tn' (tiered models), tn is now t. Add test time img size spec to default cfg.

2021-02-06 16:30:02 -08:00

weight_init.py

Cleanup experimental vit weight init a bit

2021-03-20 09:44:24 -07:00