a024ab3170
Replace radam & nadam impl with torch.optim ver, rename legacy adamw, nadam, radam impl in timm. Update optim factory & tests.
2024-11-26 15:10:15 -08:00
0b5264a108
Missing optimizers in __init__.py, add bind_defaults=False for unit tests
2024-11-13 10:50:46 -08:00
dde990785e
More fixes for new factory & tests, add back adahessian
2024-11-12 20:49:01 -08:00
45490ac52f
Post merge fix reference of old param groups helper fn locations
2024-11-12 20:49:01 -08:00
ee5f6e76bb
A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs
2024-11-12 20:49:01 -08:00
79abc25f55
Add ADOPT optimizer
2024-11-12 20:49:01 -08:00
7c16adca83
An impl of adafactor as per big vision (scaling vit) changes
2024-11-12 20:49:01 -08:00
942726db31
import lion in __init__.py
2023-07-27 09:26:57 +08:00
74d6afb4cd
Add Adan to __init__.py
2022-12-15 11:37:29 +08:00
a16a753852
Add lamb/lars to optim init imports, remove stray comment
2021-08-18 22:55:02 -07:00
8a9eca5157
A few optimizer comments, dead import, missing import
2021-08-17 18:01:33 -07:00
ac469b50da
Optimizer improvements, additions, cleanup
...
* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad
2021-08-17 17:51:20 -07:00
addfc7c1ac
adabelief
2021-04-04 23:48:15 -04:00
288682796f
Update benchmark script to add precision arg. Fix some downstream (DeiT) compat issues with latest changes. Bump version to 0.4.7
2021-04-01 16:40:12 -07:00
0e16d4e9fb
Add benchmark.py script, and update optimizer factory to be more friendly to use outside of argparse interface.
2021-02-23 15:38:12 -08:00
80078c47bb
Add Adafactor and Adahessian optimizers, cleanup optimizer arg passing, add gradient clipping support.
2020-10-09 17:24:43 -07:00
e93e571f7a
Add `adamp` and 'sgdp' optimizers.
...
Update requirements.txt
Update optim_factory.py
Add `adamp` optimizer
Update __init__.py
copy files of adamp & sgdp
Create adamp.py
Update __init__.py
Create sgdp.py
Update optim_factory.py
Update optim_factory.py
Update requirements.txt
Update adamp.py
Update sgdp.py
Update sgdp.py
Update adamp.py
2020-07-25 15:33:20 -07:00
64966f61f7
Add Nvidia's NovogGrad impl from Jasper (cleaner/faster than current) and Apex Fused optimizers
2019-08-29 15:21:38 -07:00
fac58f609a
Add RAdam, NovoGrad, Lookahead, and AdamW optimizers, a few ResNet tweaks and scheduler factory tweak.
...
* Add some of the trendy new optimizers. Decent results but not clearly better than the standards.
* Can create a None scheduler for constant LR
* ResNet defaults to zero_init of last BN in residual
* add resnet50d config
2019-08-28 00:14:10 -07:00
aa4354f466
Big re-org, working towards making pip/module as 'timm'
2019-06-19 17:20:51 -07:00
Ross Wightman
alec.tu
juntang
Sangdoo Yun