Ross Wightman 7b54eab807 Add MARS and LaProp impl, simplified from originals

2024-11-26 15:10:15 -08:00

..

__init__.py

Missing optimizers in __init__.py, add bind_defaults=False for unit tests

2024-11-13 10:50:46 -08:00

_optim_factory.py

Small optim factory tweak. default bind_defaults=True for get_optimizer_class

2024-11-13 10:45:48 -08:00

_param_groups.py

More fixes for new factory & tests, add back adahessian

2024-11-12 20:49:01 -08:00

adabelief.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adafactor_bv.py

Update adafactor comments / attrib

2024-11-12 20:49:01 -08:00

adafactor.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adahessian.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adamp.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adamw.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

adan.py

Add Adan optimizer

2022-09-23 16:05:52 -07:00

adopt.py

Update Adopt to include clipping for stability, separate wd so no param decay if update not taken on first step

2024-11-26 15:10:15 -08:00

lamb.py

A bit of lars/lamb cleanup, torch.where supports scalars properly now, make lamb grad clipping optional, clean it up a bit

2024-11-12 20:49:01 -08:00

laprop.py

Add MARS and LaProp impl, simplified from originals

2024-11-26 15:10:15 -08:00

lars.py

A bit of lars/lamb cleanup, torch.where supports scalars properly now, make lamb grad clipping optional, clean it up a bit

2024-11-12 20:49:01 -08:00

lion.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

lookahead.py

Fix Pytorch 2.0 breakage for Lookahead optimizer adapter

2023-06-02 08:39:07 -07:00

madgrad.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

mars.py

Add MARS and LaProp impl, simplified from originals

2024-11-26 15:10:15 -08:00

nadam.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

nadamw.py

Missed extra nadam algo step for capturable path

2023-06-13 20:51:31 -07:00

nvnovograd.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

optim_factory.py

More fixes for new factory & tests, add back adahessian

2024-11-12 20:49:01 -08:00

radam.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

rmsprop_tf.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

sgdp.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00

sgdw.py

A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs

2024-11-12 20:49:01 -08:00