# Visualization ## 1 Overall Design Visualization provides an intuitive explanation of the training and testing process of the deep learning model. In OpenMMLab, we expect the visualization module to meet the following requirements: - Provides rich out-of-the-box features that can meet most computer vision visualization tasks. - Versatile, expandable, and can be customized easily - Able to visualize at anywhere in the training and testing process. - Unified APIs for all OpenMMLab libraries, which is convenient for users to understand and use. Based on the above requirements, we proposed the `Visualizer` and various `VisBackend` such as `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend` in OpenMMLab 2.0. The visualizer could not only visualize the image data, but also things like configurations, scalars, and model structure. - For convenience, the APIs provided by the `Visualizer` implement the drawing and storage functions. As an internal property of `Visualizer`, `VisBackend` will be called by `Visualizer` to write data to different backends. - Considering that you may want to write data to multiple backends after drawing, `Visualizer` can be configured with multiple backends. When the user calls the storage API of the `Visualizer`, it will traverse and call all the specified APIs of `VisBackend` internally. The UML diagram of the two is as follows.
## 2 Visualizer The external interface of `Visualizer` can be divided into three categories. 1. Drawing APIs - [draw_bboxes](mmengine.visualization.Visualizer.draw_bboxes) draws a single or multiple bounding boxes - [draw_points](mmengine.visualization.Visualizer.draw_points) draws a single or multiple points - [draw_texts](mmengine.visualization.Visualizer.draw_texts) draws a single or multiple text boxes - [draw_lines](mmengine.visualization.Visualizer.lines) draws a single or multiple line segments - [draw_circles](mmengine.visualization.Visualizer.draw_circles) draws a single or multiple circles - [draw_polygons](mmengine.visualization.Visualizer.draw_polygons) draws a single or multiple polygons - [draw_binary_masks](mmengine.visualization.Visualizer.draw_binary_mask) draws single or multiple binary masks - [draw_featmap](mmengine.visualization.Visualizer.draw_featmap) draws feature map (**static method**) The above APIs can be called in a chain except for `draw_featmap` because the image size may change after this method is called. To avoid confusion, `draw_featmap` is a static method. 2. Storage APIs - [add_config](mmengine.visualization.writer.BaseWriter.add_config) writes configuration to a specific storage backend - [add_graph](mmengine.visualization.writer.BaseWriter.add_graph) writes model graph to a specific storage backend - [add_image](mmengine.visualization.writer.BaseWriter.add_image) writes image to a specific storage backend - [add_scalar](mmengine.visualization.writer.BaseWriter.add_scalar) writes scalar to a specific storage backend - [add_scalars](mmengine.visualization.writer.BaseWriter.add_scalars) writes multiple scalars to a specific storage backend at once - [add_datasample](mmengine.visualization.writer.BaseWriter.add_datasample) the abstract interface for each repositories to draw data sample Interfaces beginning with the `add` prefix represent storage APIs. \[datasample\] (`./data_element.md`)is the unified interface of each downstream repository in the OpenMMLab 2.0, and `add_datasample` can process the data sample directly . 3. Other APIs - [set_image](mmengine.visualization.Visualizer.set_image) sets the original image data, the default input image format is RGB - [get_image](mmengine.visualization.Visualizer.get_image) gets the image data in Numpy format after drawing, the default output format is RGB - [show](mmengine.visualization.Visualizer.show) for visualization - [get_backend](mmengine.visualization.Visualizer.get_backend) gets a specific storage backend by name - [close](mmengine.visualization.Visualizer.close) closes all resources, including `VisBackend` For more details, you can refer to [Visualizer Tutorial](../tutorials/visualization.md). ## 3 VisBackend After drawing, the drawn data can be stored in multiple visualization storage backends. To unify the interfaces, MMEngine provides an abstract class, `BaseVisBackend`, and some commonly used backends such as `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend`. The main interfaces and properties of `BaseVisBackend` are as follows: - [add_config](mmengine.visualization.vis_backend.BaseVisBackend.add_config) writes configuration to a specific storage backend - [add_graph](mmengine.visualization.vis_backend.BaseVisBackend.add_graph) writes model graph to a specific backend - [add_image](mmengine.visualization.vis_backend.BaseVisBackend.add_image) writes image to a specific backend - [add_scalar](mmengine.visualization.vis_backend.BaseVisBackend.add_scalar) writes scalar to a specific backend - [add_scalars](mmengine.visualization.vis_backend.BaseVisBackend.add_scalars) writes multiple scalars to a specific backend at once - [close](mmengine.visualization.vis_backend.BaseVisBackend.close) closes the resource that has been opened - [experiment](mmengine.visualization.vis_backend.BaseVisBackend.experiment) writes backend objects, such as WandB objects and Tensorboard objects `BaseVisBackend` defines five common data writing interfaces. Some writing backends are very powerful, such as WandB, which could write tables and videos. Users can directly obtain the `experiment` object for such needs and then call native APIs of the corresponding backend. `LocalVisBackend`, `WandbVisBackend`, and `TensorboardVisBackend` are all inherited from `BaseVisBackend` and implement corresponding storage functions according to their features. Users can also customize `BaseVisBackend` to extend the storage backends and implement custom storage requirements. For more details, you can refer to [Storage Backend Tutorial](../advanced_tutorials//visualization.md).