3 Commits

Author	SHA1	Message	Date
Ross Wightman	9bcd65181b	Add exponential moving average for model weights + few other additions and cleanup ... * ModelEma class added to track an EMA set of weights for the model being trained * EMA handling added to train, validation and clean_checkpoint scripts * Add multi checkpoint or multi-model validation support to validate.py * Add syncbn option (APEX) to train script for experimentation * Cleanup interface of CheckpointSaver while adding ema functionality	2019-06-07 15:39:36 -07:00
Ross Wightman	20d66beead	Move RMSpropTF another step closer to Tensorflow impl ... * init square_avg with one instead of zero as per TF * match TF order of ops for square_avg accumulation * move LR scaling to momentum buffer accumulator as per TF * add decoupled weight decay flag (not in TF)	2019-05-14 18:35:43 -07:00
Ross Wightman	bc264269c9	Morph mnasnet impl into a generic mobilenet that covers Mnasnet, MobileNetV1/V2, ChamNet, FBNet, and related ... * add an alternate RMSprop opt that applies eps like TF * add bn params for passing through alternates and changing defaults to TF style	2019-04-21 15:54:28 -07:00