Visualization provides an intuitive explanation of the training and testing process of the deep learning model. In OpenMMLab, we expect the visualization module to meet the following requirements:
- Provides rich out-of-the-box features that can meet most computer vision visualization tasks.
- Versatile, expandable, and can be customized easily
- Able to visualize at anywhere in the training and testing process.
- Unified APIs for all OpenMMLab libraries, which is convenient for users to understand and use.
Based on the above requirements, we proposed the `Visualizer` and various `VisBackend` such as `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend` in OpenMMLab 2.0. The visualizer could not only visualize the image data, but also things like configurations, scalars, and model structure.
- For convenience, the APIs provided by the `Visualizer` implement the drawing and storage functions. As an internal property of `Visualizer`, `VisBackend` will be called by `Visualizer` to write data to different backends.
- Considering that you may want to write data to multiple backends after drawing, `Visualizer` can be configured with multiple backends. When the user calls the storage API of the `Visualizer`, it will traverse and call all the specified APIs of `VisBackend` internally.
- [draw_featmap](mmengine.visualization.Visualizer.draw_featmap) draws feature map (**static method**)
The above APIs can be called in a chain except for `draw_featmap` because the image size may change after this method is called. To avoid confusion, `draw_featmap` is a static method.
Interfaces beginning with the `add` prefix represent storage APIs. \[datasample\] (`./data_element.md`)is the unified interface of each downstream repository in the OpenMMLab 2.0, and `add_datasample` can process the data sample directly .
3. Other APIs
- [set_image](mmengine.visualization.Visualizer.set_image) sets the original image data, the default input image format is RGB
- [get_image](mmengine.visualization.Visualizer.get_image) gets the image data in Numpy format after drawing, the default output format is RGB
- [show](mmengine.visualization.Visualizer.show) for visualization
- [get_backend](mmengine.visualization.Visualizer.get_backend) gets a specific storage backend by name
- [close](mmengine.visualization.Visualizer.close) closes all resources, including `VisBackend`
After drawing, the drawn data can be stored in multiple visualization storage backends. To unify the interfaces, MMEngine provides an abstract class, `BaseVisBackend`, and some commonly used backends such as `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend`.
The main interfaces and properties of `BaseVisBackend` are as follows:
`BaseVisBackend` defines five common data writing interfaces. Some writing backends are very powerful, such as WandB, which could write tables and videos. Users can directly obtain the `experiment` object for such needs and then call native APIs of the corresponding backend. `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend` are all inherited from `BaseVisBackend` and implement corresponding storage functions according to their features. Users can also customize `BaseVisBackend` to extend the storage backends and implement custom storage requirements.
For more details, you can refer to [Storage Backend Tutorial](../advanced_tutorials//visualization.md).
