Commit Graph

15 Commits (c486aa71f815188d9d86aa6711dd175a0bb7f955)

Author SHA1 Message Date
Ross Wightman 07d0c4ae96 Improve repr for DropPath module 2022-06-24 14:58:15 -07:00
Ross Wightman f670d98cb8 Make a few more layers symbolically traceable (remove from FX leaf modules)
* remove dtype kwarg from .to() calls in EvoNorm as it messed up script + trace combo
* BatchNormAct2d always uses custom forward (cut & paste from original) instead of super().forward. Fixes 
* BlurPool groups==channels, no need to use input.dim[1]
2022-03-24 21:43:56 -07:00
Ross Wightman 95cfc9b3e8 Merge remote-tracking branch 'origin/master' into norm_norm_norm 2022-01-25 22:20:45 -08:00
Ross Wightman b27c21b09a Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster 2022-01-06 16:04:58 -08:00
Ross Wightman ab49d275de Significant norm update
* ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased
* Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise)
* Update RegNet to add Z variant
* Add Pre variant of XceptionAligned that works with NormAct layers
* EvoNorm matches bits_and_tpu branch for merge
2021-12-14 13:48:30 -08:00
Ross Wightman d04f2f1377 Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster 2021-12-05 15:36:56 -08:00
Ross Wightman f31933cb37 Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers. 2020-10-13 13:33:44 -07:00
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings 2020-07-27 13:44:56 -07:00
Ross Wightman f122f0274b Significant ResNet refactor:
* stage creation + make_layer moved to separate fn with more sensible dilation/output_stride calc
* drop path rate decay easy to impl with refactored block creation loops
* fix dilation + blur pool combo
2020-07-05 00:48:12 -07:00
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
* refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto
* implement hard-mish, better grad for hard-swish
* add initial VovNet V1/V2 impl, fix 
* VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN)
* Wrap IABN for any models that use it
* make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests
2020-06-01 17:16:52 -07:00
Ross Wightman 1904ed8fec Improve dropblock impl, add fast variant, and better AMP speed, inplace, batchwise... few ResNeSt cleanups 2020-05-13 15:17:08 -07:00
Ross Wightman c60069c1eb Annotate types on drop fns to avoid torchscript error 2020-02-27 09:30:23 -08:00
Ross Wightman f1d5f8a6c4 Update comments for Selective Kernel and DropBlock/Path impl, add skresnet34 weights 2020-02-18 13:58:30 -08:00
Ross Wightman f902bcd54c Layer refactoring continues, ResNet downsample rewrite for proper dilation in 3x3 and avg_pool cases
* select_conv2d -> create_conv2d
* added create_attn to create attention module from string/bool/module
* factor padding helpers into own file, use in both conv2d_same and avg_pool2d_same
* add some more test eca resnet variants
* minor tweaks, naming, comments, consistency
2020-02-10 11:55:03 -08:00
Ross Wightman a99ec4e7d1 A bunch more layer reorg, splitting many layers into own files. Improve torchscript compatibility. 2020-02-09 14:46:28 -08:00