9 Commits

Author	SHA1	Message	Date
Ross Wightman	93cc08fdc5	Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).	2021-11-20 15:50:51 -08:00
Ross Wightman	af607b75cc	Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU	2021-11-19 17:37:00 -08:00
Alexander Soare	b25ff96768	wip - pre-rebase	2021-11-12 20:45:05 +00:00
Alexander Soare	bc3d4eb403	wip -rebase	2021-11-12 20:45:05 +00:00
Ross Wightman	fc8b8afb6f	Fix a silly bug in Sample version of EvoNorm missing x* part of swish, update EvoNormBatch to accumulated unbiased variance.	2020-08-13 18:25:01 -07:00
Ross Wightman	6c17d57a2c	Fix some attributions, add copyrights to some file docstrings	2020-07-27 13:44:56 -07:00
Ross Wightman	eb7653614f	Monster commit, activation refactor, VoVNet, norm_act improvements, more ... * refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto * implement hard-mish, better grad for hard-swish * add initial VovNet V1/V2 impl, fix #151 * VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN) * Wrap IABN for any models that use it * make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests	2020-06-01 17:16:52 -07:00
Ross Wightman	780860d140	Add norm_act factory method, move JIT of norm layers to factory	2020-05-09 22:09:21 -07:00
Ross Wightman	14edacdf9a	DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN	2020-05-09 18:26:41 -07:00