Ross Wightman
|
8e4ac3549f
|
All ScaledStdConv and StdConv uses default to using F.layernorm so that they work with PyTorch XLA. eps value tweaking is a WIP.
|
2021-06-07 17:14:19 -07:00 |
|
Ross Wightman
|
bda8ab015a
|
Remove min channels for SelectiveKernel, divisor should cover cases well enough.
|
2021-05-31 15:38:56 -07:00 |
|
Ross Wightman
|
a27f4aec4a
|
Missed args for skresnext w/ refactoring.
|
2021-05-31 14:06:34 -07:00 |
|
Ross Wightman
|
307a935b79
|
Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.
|
2021-05-31 13:18:11 -07:00 |
|
Ross Wightman
|
8bf63b6c6c
|
Able to use other attn layer in EfficientNet now. Create test ECA + GC B0 configs. Make ECA more configurable.
|
2021-05-30 12:47:02 -07:00 |
|
Ross Wightman
|
9611458e19
|
Throw in some FBNetV3 code I had lying around, some refactoring of SE reduction channel calcs for all EffNet archs.
|
2021-05-28 20:47:24 -07:00 |
|
Ross Wightman
|
f615474be3
|
Fix broken test, repvgg block doesn't have attn_last attr.
|
2021-05-27 18:12:22 -07:00 |
|
Ross Wightman
|
742c2d5247
|
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
|
2021-05-27 18:03:29 -07:00 |
|
Ross Wightman
|
9c78de8c02
|
Fix #661, move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present.
|
2021-05-26 15:28:42 -07:00 |
|
Ross Wightman
|
f45de37690
|
Merge branch 'master' into levit_visformer_rednet
|
2021-05-22 16:34:31 -07:00 |
|
Ross Wightman
|
d5af752117
|
Add preliminary gMLP and ResMLP impl to Mlp-Mixer
|
2021-05-19 09:55:05 -07:00 |
|
Ross Wightman
|
3bffc701f1
|
Merge branch 'master' into levit_visformer_rednet
|
2021-05-14 23:02:12 -07:00 |
|
Ross Wightman
|
ecc7552c5c
|
Add levit, levit_c, and visformer model defs. Largely untested and not finished cleanup.
|
2021-05-14 17:16:34 -07:00 |
|
Ross Wightman
|
165fb354b2
|
Add initial RedNet model / Involution layer impl for testing
|
2021-05-14 17:16:34 -07:00 |
|
Ross Wightman
|
c4f482a08b
|
EfficientNetV2 official impl w/ weights ported from TF. Cleanup/refactor of related EfficientNet classes and models.
|
2021-05-14 15:50:00 -07:00 |
|
Ross Wightman
|
715519a5ef
|
Rethink name of patch embed grid info
|
2021-05-06 14:08:20 -07:00 |
|
Ross Wightman
|
b2c305c2aa
|
Move Mlp and PatchEmbed modules into layers. Being used in lots of models now...
|
2021-05-06 14:03:23 -07:00 |
|
Ross Wightman
|
0721559511
|
Improved (hopefully) init for SA/SA-like layers used in ByoaNets
|
2021-05-04 21:40:39 -07:00 |
|
Ross Wightman
|
0d87650fea
|
Remove filter hack from BlurPool w/ non-persistent buffer. Use BlurPool2d instead of AntiAliasing.. for TResNet. Breaks PyTorch < 1.6.
|
2021-05-04 16:56:28 -07:00 |
|
Ross Wightman
|
9cc7dda6e5
|
Fixup byoanet configs to pass unit tests. Add swin_attn and swinnet26t model for testing.
|
2021-04-29 21:08:37 -07:00 |
|
Ross Wightman
|
e15c3886ba
|
Defaul lambda r=7. Define '26t' stage 4/5 256x256 variants for all of bot/halo/lambda nets for experiment. Add resnet50t for exp. Fix a few comments.
|
2021-04-29 10:58:49 -07:00 |
|
Ross Wightman
|
4e4b863b15
|
Missed norm.py
|
2021-04-12 09:57:56 -07:00 |
|
Ross Wightman
|
ce62f96d4d
|
ByoaNet with bottleneck transformer, lambda resnet, and halo net experiments
|
2021-04-12 09:38:02 -07:00 |
|
Ross Wightman
|
a5310a3451
|
Merge remote-tracking branch 'origin/benchmark-fixes-vit_hybrids' into pit_and_vit_update
|
2021-04-01 12:15:34 -07:00 |
|
Ross Wightman
|
cf5fec5047
|
Cleanup experimental vit weight init a bit
|
2021-03-20 09:44:24 -07:00 |
|
Ross Wightman
|
740f32c96a
|
Add ECA-NFNet-L0 weights and update model name. Update README and bump version to 0.4.6
|
2021-03-17 13:55:32 -07:00 |
|
Ross Wightman
|
f57db99101
|
Update README, fix iabn pip version print.
|
2021-03-07 16:17:06 -08:00 |
|
Ross Wightman
|
8563609b28
|
Update notes in ScaledStdConv impl
|
2021-02-18 12:44:08 -08:00 |
|
Ross Wightman
|
678ba4e0a2
|
Add NFNet-F model weights ported from DeepMind Haiku impl and new set of models w/ compatible config.
|
2021-02-18 12:28:46 -08:00 |
Ross Wightman
|
d8e69206be
|
Merge pull request #419 from rwightman/byob_vgg_models
More models, GPU-Efficient Nets, RepVGG, classic VGG, and flexible Byob backbone.
|
2021-02-10 15:44:09 -08:00 |
Reuben
|
94ca140b67
|
update collections.abc import
|
2021-02-10 23:54:35 +11:00 |
|
Ross Wightman
|
1bcc69e0ad
|
Use in_channels for depthwise groups, allows using `out_channels=N * in_channels` (does not impact existing models). Fix #354.
|
2021-02-09 16:22:52 -08:00 |
|
Ross Wightman
|
9811e229f7
|
Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.
|
2021-02-09 16:22:52 -08:00 |
|
Ross Wightman
|
a39c3ee216
|
Merge branch 'master' into eca-weights
|
2021-02-08 11:52:31 -08:00 |
|
Ross Wightman
|
68a4144882
|
Add new weights for ecaresnet26t/50t/269d models. Remove distinction between 't' and 'tn' (tiered models), tn is now t. Add test time img size spec to default cfg.
|
2021-02-06 16:30:02 -08:00 |
|
Ross Wightman
|
b9843f954b
|
Merge pull request #282 from tigert1998/patch-1
Add symbolic for SwishJitAutoFn to support onnx
|
2021-02-04 12:18:40 -08:00 |
hwangdeyu
|
7a4be5c035
|
add operator HardSwishJitAutoFn export to onnx
|
2021-02-03 09:06:53 +08:00 |
|
Ross Wightman
|
90980de4a9
|
Fix up a few details in NFResNet models, managed stable training. Add support for gamma gain to be applied in activation or ScaleStdConv. Some tweaks to ScaledStdConv.
|
2021-01-30 16:32:07 -08:00 |
|
Ross Wightman
|
5a8e1e643e
|
Initial Normalizer-Free Reg/ResNet impl. A bit of related layer refactoring.
|
2021-01-27 22:06:57 -08:00 |
|
Ross Wightman
|
231d04e91a
|
ResNetV2 pre-act and non-preact model, w/ BiT pretrained weights and support for ViT R50 model. Tweaks for in21k num_classes passing. More to do... tests failing.
|
2020-12-28 16:59:15 -08:00 |
|
Ross Wightman
|
2ed8f24715
|
A few more changes for 0.3.2 maint release. Linear layer change for mobilenetv3 and inception_v3, support no bias for linear wrapper.
|
2020-11-30 16:19:52 -08:00 |
|
Ross Wightman
|
460eba7f24
|
Work around casting issue with combination of native torch AMP and torchscript for Linear layers
|
2020-11-30 13:30:51 -08:00 |
|
Ross Wightman
|
5f4b6076d8
|
Fix inplace arg compat for GELU and PreLU via activation factory
|
2020-11-30 13:27:40 -08:00 |
|
Ross Wightman
|
fd962c4b4a
|
Native SiLU (Swish) op doesn't export to ONNX
|
2020-11-29 21:56:55 -08:00 |
tigertang
|
43f2500c26
|
Add symbolic for SwishJitAutoFn to support onnx
|
2020-11-18 14:36:12 +08:00 |
|
Ross Wightman
|
e90edce438
|
Support native silu activation (aka swish). An optimized ver is available in PyTorch 1.7.
|
2020-10-29 15:45:17 -07:00 |
|
Ross Wightman
|
f31933cb37
|
Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers.
|
2020-10-13 13:33:44 -07:00 |
|
Ross Wightman
|
fcb6258877
|
Add missing leaky_relu layer factory defn, update Apex/Native loss scaler interfaces to support unscaled grad clipping. Bump ver to 0.2.2 for pending release.
|
2020-10-02 16:19:39 -07:00 |
|
Ross Wightman
|
e8ca45854c
|
More models in sotabench, more control over sotabench run, dataset filename extraction consistency
|
2020-09-24 15:56:57 -07:00 |
|
Ross Wightman
|
80c9d9cc72
|
Add 'fast' global pool option, remove redundant SEModule from tresnet, normal one is now 'fast'
|
2020-09-02 09:11:48 -07:00 |