Visualize Training Logs

MMEngine integrates experiment management tools such as TensorBoard, Weights & Biases (WandB), MLflow, ClearML and Neptune, making it easy to track and visualize metrics like loss and accuracy.

Below, we'll show you how to configure an experiment management tool in just one line, based on the example from 15 minutes to get started with MMEngine.

TensorBoard

Configure the visualizer in the initialization parameters of the Runner, and set vis_backends to TensorboardVisBackend.

runner = Runner(
    model=MMResNet50(),
    work_dir='./work_dir',
    train_dataloader=train_dataloader,
    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
    val_dataloader=val_dataloader,
    val_cfg=dict(),
    val_evaluator=dict(type=Accuracy),
    visualizer=dict(type='Visualizer', vis_backends=[dict(type='TensorboardVisBackend')]),
)
runner.train()

WandB

Before using WandB, you need to install the wandb dependency library and log in to WandB.

pip install wandb
wandb login

Configure the visualizer in the initialization parameters of the Runner, and set vis_backends to WandbVisBackend.

runner = Runner(
    model=MMResNet50(),
    work_dir='./work_dir',
    train_dataloader=train_dataloader,
    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
    val_dataloader=val_dataloader,
    val_cfg=dict(),
    val_evaluator=dict(type=Accuracy),
    visualizer=dict(type='Visualizer', vis_backends=[dict(type='WandbVisBackend')]),
)
runner.train()

You can click on WandbVisBackend API to view the configurable parameters for WandbVisBackend. For example, the init_kwargs parameter will be passed to the wandb.init method.

runner = Runner(
    ...
    visualizer=dict(
        type='Visualizer',
        vis_backends=[
            dict(
                type='WandbVisBackend',
                init_kwargs=dict(project='toy-example')
            ),
        ],
    ),
    ...
)
runner.train()

MLflow (WIP)

ClearML

Before using ClearML, you need to install the clearml dependency library and refer to Connect ClearML SDK to the Server for configuration.

pip install clearml
clearml-init

Configure the visualizer in the initialization parameters of the Runner, and set vis_backends to ClearMLVisBackend.

runner = Runner(
    model=MMResNet50(),
    work_dir='./work_dir',
    train_dataloader=train_dataloader,
    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
    val_dataloader=val_dataloader,
    val_cfg=dict(),
    val_evaluator=dict(type=Accuracy),
    visualizer=dict(type='Visualizer', vis_backends=[dict(type='ClearMLVisBackend')]),
)
runner.train()

Neptune

Before using Neptune, you need to install neptune dependency library and refer to Neptune.AI for configuration.

pip install neptune

Configure the Runner in the initialization parameters of the Runner, and set vis_backends to NeptuneVisBackend.

runner = Runner(
    model=MMResNet50(),
    work_dir='./work_dir',
    train_dataloader=train_dataloader,
    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
    val_dataloader=val_dataloader,
    val_cfg=dict(),
    val_evaluator=dict(type=Accuracy),
    visualizer=dict(type='Visualizer', vis_backends=[dict(type='NeptuneVisBackend')]),
)
runner.train()

Please note: If the project and api_token are not specified, neptune will be set to offline mode and the generated files will be saved to the local .neptune file. It is recommended to specify the project and api_token during initialization as shown below.

runner = Runner(
    ...
    visualizer=dict(
        type='Visualizer',
        vis_backends=[
            dict(
                type='NeptuneVisBackend',
                init_kwargs=dict(project='workspace-name/project-name',
                                 api_token='your api token')
            ),
        ],
    ),
    ...
)
runner.train()

More initialization configuration parameters are available at neptune.init_run API.

5.3 KiB Raw Blame History