# torchreid.optim

## Optimizer

`torchreid.optim.optimizer.``build_optimizer`(model, optim='adam', lr=0.0003, weight_decay=0.0005, momentum=0.9, sgd_dampening=0, sgd_nesterov=False, rmsprop_alpha=0.99, adam_beta1=0.9, adam_beta2=0.99, staged_lr=False, new_layers='', base_lr_mult=0.1)

  A function wrapper for building an optimizer.

  Parameters:

  - model (nn.Module) – model.
  - optim (str, optional) – optimizer. Default is “adam”.
  - lr (float, optional) – learning rate. Default is 0.0003.
  - weight_decay (float, optional) – weight decay (L2 penalty). Default is 5e-04.
  - momentum (float, optional) – momentum factor in sgd. Default is 0.9,
  - sgd_dampening (float, optional) – dampening for momentum. Default is 0.
  - sgd_nesterov (bool, optional) – enables Nesterov momentum. Default is False.
  - rmsprop_alpha (float, optional) – smoothing constant for rmsprop. Default is 0.99.
  - adam_beta1 (float, optional) – beta-1 value in adam. Default is 0.9.
  - adam_beta2 (float, optional) – beta-2 value in adam. Default is 0.99,
  - staged_lr (bool, optional) – uses different learning rates for base and new layers. Base layers are pretrained layers while new layers are randomly initialized, e.g. the identity classification layer. Enabling `staged_lr` can allow the base layers to be trained with a smaller learning rate determined by `base_lr_mult`, while the new layers will take the `lr`. Default is False.
  - new_layers (str or list) – attribute names in `model`. Default is empty.
  - base_lr_mult (float, optional) – learning rate multiplier for base layers. Default is 0.1.

  Examples::

        >>> # A normal optimizer can be built by
        >>> optimizer = torchreid.optim.build_optimizer(model, optim='sgd', lr=0.01)
        >>> # If you want to use a smaller learning rate for pretrained layers
        >>> # and the attribute name for the randomly initialized layer is 'classifier',
        >>> # you can do
        >>> optimizer = torchreid.optim.build_optimizer(
        >>>     model, optim='sgd', lr=0.01, staged_lr=True,
        >>>     new_layers='classifier', base_lr_mult=0.1
        >>> )
        >>> # Now the `classifier` has learning rate 0.01 but the base layers
        >>> # have learning rate 0.01 * 0.1.
        >>> # new_layers can also take multiple attribute names. Say the new layers
        >>> # are 'fc' and 'classifier', you can do
        >>> optimizer = torchreid.optim.build_optimizer(
        >>>     model, optim='sgd', lr=0.01, staged_lr=True,
        >>>     new_layers=['fc', 'classifier'], base_lr_mult=0.1
        >>> )

## LR Scheduler

`torchreid.optim.lr_scheduler.``build_lr_scheduler`(optimizer, lr_scheduler, stepsize, gamma=0.1)

  A function wrapper for building a learning rate scheduler.

  Parameters:

  - optimizer (Optimizer) – an Optimizer.
  - lr_scheduler (str) – learning rate scheduler method. Currently supports “single_step” and “multi_step”.
  - stepsize (int or list) – step size to decay learning rate. When `lr_scheduler` is “single_step”, `stepsize` should be an integer. When `lr_scheduler` is “multi_step”, `stepsize` is a list.
  - gamma (float, optional) – decay rate. Default is 0.1.

  Examples::

        >>> # Decay learning rate by every 20 epochs.
        >>> scheduler = torchreid.optim.build_lr_scheduler(
        >>>     optimizer, lr_scheduler='single_step', stepsize=20
        >>> )
        >>> # Decay learning rate at 30, 50 and 55 epochs.
        >>> scheduler = torchreid.optim.build_lr_scheduler(
        >>>     optimizer, lr_scheduler='multi_step', stepsize=[30, 50, 55]
        >>> )