Ross Wightman afdf11d9ae Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

..

__init__.py

…

_optim_factory.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

_param_groups.py

…

_types.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adabelief.py

…

adafactor_bv.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adafactor.py

Update README, missed small discrep in adafactor min dim update

2024-11-29 10:57:47 -08:00

adahessian.py

…

adamp.py

…

adamw.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adan.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

adopt.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

lamb.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

laprop.py

Make LaProp weight decay match typical PyTorch 'decoupled' behaviour where it's scaled by LR

2024-11-29 16:44:43 -08:00

lars.py

…

lion.py

To be technically correct, need to check the in-place _ ver of op

2024-11-28 15:11:58 -08:00

lookahead.py

…

madgrad.py

…

mars.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

nadam.py

…

nadamw.py

To be technically correct, need to check the in-place _ ver of op

2024-11-28 15:11:58 -08:00

nvnovograd.py

…

optim_factory.py

…

radam.py

…

rmsprop_tf.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

sgdp.py

…

sgdw.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00