Ross Wightman b1b024dfed Scheduler update, add v2 factory method, support scheduling on updates instead of just epochs. Add LR to summary csv. Add lr_base scaling calculations to train script. Fix #1168		2022-10-07 10:43:04 -07:00
..
__init__.py	Add lamb/lars to optim init imports, remove stray comment	2021-08-18 22:55:02 -07:00
adabelief.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
adafactor.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
adahessian.py	Add Adafactor and Adahessian optimizers, cleanup optimizer arg passing, add gradient clipping support.	2020-10-09 17:24:43 -07:00
adamp.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
adamw.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
adan.py	Add Adan optimizer	2022-09-23 16:05:52 -07:00
lamb.py	MOAR optimizer changes. Woo!	2021-08-18 22:20:35 -07:00
lars.py	fix lars	2022-01-19 17:49:43 +08:00
lookahead.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
madgrad.py	MOAR optimizer changes. Woo!	2021-08-18 22:20:35 -07:00
nadam.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
nvnovograd.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
optim_factory.py	Scheduler update, add v2 factory method, support scheduling on updates instead of just epochs. Add LR to summary csv. Add lr_base scaling calculations to train script. Fix #1168	2022-10-07 10:43:04 -07:00
radam.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
rmsprop_tf.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00
sgdp.py	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	2021-08-18 17:21:56 -07:00