Commit Graph

10 Commits (81bf0b403384b45680ecc303b50d482c3d1dcd5f)

Author SHA1 Message Date
Ross Wightman 480c676ffa Fix FX breaking assert in evonorm 2021-11-24 09:24:47 -08:00
Ross Wightman 93cc08fdc5 Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix). 2021-11-20 15:50:51 -08:00
Ross Wightman af607b75cc Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU 2021-11-19 17:37:00 -08:00
Alexander Soare b25ff96768 wip - pre-rebase 2021-11-12 20:45:05 +00:00
Alexander Soare bc3d4eb403 wip -rebase 2021-11-12 20:45:05 +00:00
Ross Wightman fc8b8afb6f Fix a silly bug in Sample version of EvoNorm missing x* part of swish, update EvoNormBatch to accumulated unbiased variance. 2020-08-13 18:25:01 -07:00
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings 2020-07-27 13:44:56 -07:00
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
* refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto
* implement hard-mish, better grad for hard-swish
* add initial VovNet V1/V2 impl, fix #151
* VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN)
* Wrap IABN for any models that use it
* make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests
2020-06-01 17:16:52 -07:00
Ross Wightman 780860d140 Add norm_act factory method, move JIT of norm layers to factory 2020-05-09 22:09:21 -07:00
Ross Wightman 14edacdf9a DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN 2020-05-09 18:26:41 -07:00