Ross Wightman 2e2687aee2 Cleanup original adafactor impl, add row/col dim heuristic that works with both conv and linear layers

2024-11-08 08:35:25 -08:00

..

__init__.py

…

adabelief.py

…

adafactor_bv.py

Change eps defaults in adafactor_bv again after some checking

2024-11-07 21:45:46 -08:00

adafactor.py

Cleanup original adafactor impl, add row/col dim heuristic that works with both conv and linear layers

2024-11-08 08:35:25 -08:00

adahessian.py

…

adamp.py

…

adamw.py

…

adan.py

…

lamb.py

A bit of lars/lamb cleanup, torch.where supports scalars properly now, make lamb grad clipping optional, clean it up a bit

2024-11-07 21:42:24 -08:00

lars.py

A bit of lars/lamb cleanup, torch.where supports scalars properly now, make lamb grad clipping optional, clean it up a bit

2024-11-07 21:42:24 -08:00

lion.py

…

lookahead.py

…

madgrad.py

…

nadam.py

…

nadamw.py

…

nvnovograd.py

…

optim_factory.py

…

radam.py

…

rmsprop_tf.py

…

sgdp.py

…

sgdw.py

…