Commit Graph

11 Commits (d04f2f137795017fc110ecf1084ffbfdf5a25747)

Author SHA1 Message Date
Ross Wightman 78912b6375 Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials. 2021-12-01 12:09:01 -08:00
Ross Wightman 9c78de8c02 Fix #661, move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present. 2021-05-26 15:28:42 -07:00
Ross Wightman 5f4b6076d8 Fix inplace arg compat for GELU and PreLU via activation factory 2020-11-30 13:27:40 -08:00
Ross Wightman fd962c4b4a Native SiLU (Swish) op doesn't export to ONNX 2020-11-29 21:56:55 -08:00
Ross Wightman e90edce438 Support native silu activation (aka swish). An optimized ver is available in PyTorch 1.7. 2020-10-29 15:45:17 -07:00
Ross Wightman fcb6258877 Add missing leaky_relu layer factory defn, update Apex/Native loss scaler interfaces to support unscaled grad clipping. Bump ver to 0.2.2 for pending release. 2020-10-02 16:19:39 -07:00
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings 2020-07-27 13:44:56 -07:00
Ross Wightman 298fba09ac Back out some activation hacks trialing upcoming pytorch changes 2020-07-17 18:41:37 -07:00
Ross Wightman 3b9004bef9 Lots of changes to model creation helpers, close to finalizing feature extraction / interfaces 2020-07-17 17:54:26 -07:00
Ross Wightman 151679c2f1 Add custom grad tests, fix cut & paste error with hard_mish ME, add a few more pytorch act fns to factory 2020-06-11 14:49:23 -07:00
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
* refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto
* implement hard-mish, better grad for hard-swish
* add initial VovNet V1/V2 impl, fix #151
* VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN)
* Wrap IABN for any models that use it
* make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests
2020-06-01 17:16:52 -07:00