Ross Wightman fac58f609a Add RAdam, NovoGrad, Lookahead, and AdamW optimizers, a few ResNet tweaks and scheduler factory tweak.
* Add some of the trendy new optimizers. Decent results but not clearly better than the standards.
* Can create a None scheduler for constant LR
* ResNet defaults to zero_init of last BN in residual
* add resnet50d config
2019-08-28 00:14:10 -07:00

8 lines
217 B
Python

from .nadam import Nadam
from .rmsprop_tf import RMSpropTF
from .adamw import AdamW
from .radam import RAdam
from .novograd import NovoGrad
from .lookahead import Lookahead
from .optim_factory import create_optimizer