Ross Wightman 19090ea966 Need to init momentum with correct dtype

2024-11-12 20:49:01 -08:00

..

__init__.py

An impl of adafactor as per big vision (scaling vit) changes

2024-11-12 20:49:01 -08:00

adabelief.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

adafactor_bv.py

Need to init momentum with correct dtype

2024-11-12 20:49:01 -08:00

adafactor.py

…

adahessian.py

…

adamp.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

adamw.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

adan.py

…

lamb.py

MOAR optimizer changes. Woo!

2021-08-18 22:20:35 -07:00

lars.py

…

lion.py

Add multi-tensor (foreach) version of Lion in style of upcoming PyTorch 2.0 optimizers

2023-02-16 15:48:00 -08:00

lookahead.py

Fix Pytorch 2.0 breakage for Lookahead optimizer adapter

2023-06-02 08:39:07 -07:00

madgrad.py

…

nadam.py

Add NadamW based on mlcommons algorithm, added multi-tensor step

2023-06-13 20:45:17 -07:00

nadamw.py

Missed extra nadam algo step for capturable path

2023-06-13 20:51:31 -07:00

nvnovograd.py

…

optim_factory.py

An impl of adafactor as per big vision (scaling vit) changes

2024-11-12 20:49:01 -08:00

radam.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

rmsprop_tf.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

sgdp.py

More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.

2021-08-18 17:21:56 -07:00

sgdw.py

Update sgdw for older pytorch

2023-12-11 12:10:29 -08:00