Ross Wightman 5f85f8eefa Fix comment, add 'stochastic weight decay' idea because why not

2025-01-30 15:44:02 -08:00

..

__init__.py

Add guard around 'somewhat' newer torch RAdam / NAdam imports

2024-11-26 15:10:15 -08:00

_optim_factory.py

Some more kron work. Figured out why some tests fail, implemented a deterministic rng state load but too slow so skipping some tests for now.

2025-01-27 21:02:26 -08:00

_param_groups.py

More fixes for new factory & tests, add back adahessian

2024-11-12 20:49:01 -08:00

_types.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adabelief.py

Fix typos

2025-01-19 13:39:40 -08:00

adafactor_bv.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adafactor.py

Update README, missed small discrep in adafactor min dim update

2024-11-29 10:57:47 -08:00

adahessian.py

Fix typos

2025-01-19 13:39:40 -08:00

adamp.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adamw.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

adan.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

adopt.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

kron.py

Fix comment, add 'stochastic weight decay' idea because why not

2025-01-30 15:44:02 -08:00

lamb.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

laprop.py

Make LaProp weight decay match typical PyTorch 'decoupled' behaviour where it's scaled by LR

2024-11-29 16:44:43 -08:00

lars.py

A bit of lars/lamb cleanup, torch.where supports scalars properly now, make lamb grad clipping optional, clean it up a bit

2024-11-12 20:49:01 -08:00

lion.py

To be technically correct, need to check the in-place _ ver of op

2024-11-28 15:11:58 -08:00

lookahead.py

Fix Pytorch 2.0 breakage for Lookahead optimizer adapter

2023-06-02 08:39:07 -07:00

madgrad.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

mars.py

Add caution to Adan. Add decouple decay option to LAMB.

2024-12-05 13:50:30 -08:00

nadam.py

Replace radam & nadam impl with torch.optim ver, rename legacy adamw, nadam, radam impl in timm. Update optim factory & tests.

2024-11-26 15:10:15 -08:00

nadamw.py

To be technically correct, need to check the in-place _ ver of op

2024-11-28 15:11:58 -08:00

nvnovograd.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

optim_factory.py

More fixes for new factory & tests, add back adahessian

2024-11-12 20:49:01 -08:00

radam.py

Replace radam & nadam impl with torch.optim ver, rename legacy adamw, nadam, radam impl in timm. Update optim factory & tests.

2024-11-26 15:10:15 -08:00

rmsprop_tf.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00

sgdp.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

sgdw.py

Cautious optimizer impl plus some typing cleanup.

2024-11-28 15:11:58 -08:00