Ross Wightman ac469b50da Optimizer improvements, additions, cleanup ...

* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad

2021-08-17 17:51:20 -07:00

..

data

Add mmax config key to auto_augment for increasing upper bound of RandAugment magnitude beyond 10. Make AugMix uniform sampling default not override config setting.

2021-08-12 15:39:05 -07:00

loss

asymmetric_loss

2020-10-16 17:12:28 +03:00

models

Add gc_efficientnetv2_rw_t weights (global context instead of SE attn). Add TF XL weights even though the fine-tuned ones don't validate that well. Change default arg for GlobalContext to use scal (mul) mode.

2021-08-07 16:45:29 -07:00

optim

Optimizer improvements, additions, cleanup

2021-08-17 17:51:20 -07:00

scheduler

1. Added a simple multi step LR scheduler

2021-07-09 16:18:27 -04:00

utils

Merge pull request #550 from amaarora/wandb

2021-04-15 09:26:35 -07:00

__init__.py

Add default_cfg options for min_input_size / fixed_input_size, queries in model registry, and use for testing self-attn models

2021-04-12 11:54:22 -07:00

version.py

Add efficientnetv2_rw_t defs w/ weights, and gc variant, as well as gcresnet26ts for experiments. Version 0.4.13

2021-07-09 16:46:52 -07:00