12 Commits

Author	SHA1	Message	Date
Ross Wightman	5f10450235	Some more kron work. Figured out why some tests fail, implemented a deterministic rng state load but too slow so skipping some tests for now.	2025-01-27 21:02:26 -08:00
Ross Wightman	cd21e80d03	Fiddling with Kron (PSGD)	2025-01-27 21:02:26 -08:00
Ross Wightman	afdf11d9ae	Add caution to Adan. Add decouple decay option to LAMB.	2024-12-05 13:50:30 -08:00
Ross Wightman	303f7691a1	Add cautious mars, improve test reliability by skipping grad diff for first step	2024-12-02 11:29:02 -08:00
Ross Wightman	7cf683628f	Cautious optimizer impl plus some typing cleanup.	2024-11-28 15:11:58 -08:00
Ross Wightman	a024ab3170	Replace radam & nadam impl with torch.optim ver, rename legacy adamw, nadam, radam impl in timm. Update optim factory & tests.	2024-11-26 15:10:15 -08:00
Ross Wightman	d0161f303a	Small optim factory tweak. default bind_defaults=True for get_optimizer_class	2024-11-13 10:45:48 -08:00
Ross Wightman	8b9b6824ae	Minor changes, has_eps=False missing for bnb lion	2024-11-12 20:49:01 -08:00
Ross Wightman	61305cc26a	Fix adopt descriptions	2024-11-12 20:49:01 -08:00
Ross Wightman	dde990785e	More fixes for new factory & tests, add back adahessian	2024-11-12 20:49:01 -08:00
Ross Wightman	53657a31b7	Try to fix documentation build, add better docstrings to public optimizer api	2024-11-12 20:49:01 -08:00
Ross Wightman	ee5f6e76bb	A bit of an optimizer overhaul, added an improved factory, list_optimizers, class helper and add info classes with descriptions, arg configs	2024-11-12 20:49:01 -08:00