mirror of https://github.com/open-mmlab/mmocr.git
[Docs] Add Chinese Translation of deployment.md. (#506)
* Add Chinese Translation of deployment.md. * final fix Co-authored-by: zhaoyuxuan <zhaoyuxuan@zhaoyuxuandeMacBook-Pro.local> Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>pull/518/head
fatfishZhao
GitHub
parent
06c726c130
commit
66ce0efee0
|
|
@ -1,10 +1,10 @@
|
|||
# Deployment
|
||||
# 部署
|
||||
|
||||
We provide deployment tools under `tools/deployment` directory.
|
||||
我们在 `tools/deployment` 目录下提供了一些部署工具。
|
||||
|
||||
## Convert to ONNX (experimental)
|
||||
## 转换至 ONNX (试验性的)
|
||||
|
||||
We provide a script to convert the model to [ONNX](https://github.com/onnx/onnx) format. The converted model could be visualized by tools like [Netron](https://github.com/lutzroeder/netron). Besides, we also support comparing the output results between PyTorch and ONNX model.
|
||||
我们提供了将模型转换至 [ONNX](https://github.com/onnx/onnx) 格式的脚本。转换后的模型可以使用诸如 [Netron](https://github.com/lutzroeder/netron) 的工具可视化。 此外，我们也支持比较 PyTorch 和 ONNX 模型的输出结果。
|
||||
|
||||
```bash
|
||||
python tools/deployment/pytorch2onnx.py
|
||||
|
|
@ -21,48 +21,48 @@ python tools/deployment/pytorch2onnx.py
|
|||
--dynamic-export
|
||||
```
|
||||
|
||||
Description of arguments:
|
||||
参数说明：
|
||||
|
||||
| ARGS | Type | Description |
|
||||
| -------------- | --------------------- | ----------------------------------------------------------- |
|
||||
| `model_config` | str | The path to a model config file. |
|
||||
| `model_ckpt` | str | The path to a model checkpoint file. |
|
||||
| `model_type` | 'recog', 'det' | The model type of the config file. |
|
||||
| `image_path` | str | The path to input image file. |
|
||||
| `--output-file`| str | The path to output ONNX model. Defaults to `tmp.onnx`. |
|
||||
| `--device-id`| int | Which GPU to use. Defaults to 0. |
|
||||
| `--opset-version` | int | ONNX opset version. Defaults to 11. |
|
||||
| `--verify`| bool | Determines whether to verify the correctness of an exported model. Defaults to `False`. |
|
||||
| `--verbose`| bool | Determines whether to print the architecture of the exported model. Defaults to `False`. |
|
||||
| `--show`| bool | Determines whether to visualize outputs of ONNXRuntime and PyTorch. Defaults to `False`. |
|
||||
| `--dynamic-export`| bool | Determines whether to export ONNX model with dynamic input and output shapes. Defaults to `False`. |
|
||||
| 参数 | 类型 | 描述 |
|
||||
| ------------------ | -------------- | ------------------------------------------------------------ |
|
||||
| `model_config` | str | 模型配置文件的路径。 |
|
||||
| `model_ckpt` | str | 模型权重文件的路径。 |
|
||||
| `model_type` | 'recog', 'det' | 配置文件对应的模型类型。 |
|
||||
| `image_path` | str | 输入图片的路径。 |
|
||||
| `--output-file` | str | 输出的 ONNX 模型路径。 默认为 `tmp.onnx`。 |
|
||||
| `--device-id` | int | 使用哪块 GPU。默认为0。 |
|
||||
| `--opset-version` | int | ONNX 操作集版本。默认为11。 |
|
||||
| `--verify` | bool | 决定是否验证输出模型的正确性。默认为 `False`。 |
|
||||
| `--verbose` | bool | 决定是否打印导出模型的结构，默认为 `False`。 |
|
||||
| `--show` | bool | 决定是否可视化 ONNXRuntime 和 PyTorch 的输出。默认为 `False`。 |
|
||||
| `--dynamic-export` | bool | 决定是否导出有动态输入和输出尺寸的 ONNX 模型。默认为 `False`。 |
|
||||
|
||||
:::{note}
|
||||
This tool is still experimental. For now, some customized operators are not supported, and we only support a subset of detection and recognition algorithms.
|
||||
这个工具仍然是试验性的。一些定制的操作没有被支持，并且我们目前仅支持一部分的文本检测和文本识别算法。
|
||||
:::
|
||||
|
||||
### List of supported models exportable to ONNX
|
||||
### 支持导出到 ONNX 的模型列表
|
||||
|
||||
The table below lists the models that are guaranteed to be exportable to ONNX and runnable in ONNX Runtime.
|
||||
下表列出的模型可以保证导出到 ONNX 并且可以在 ONNX Runtime 下运行。
|
||||
|
||||
| Model | Config | Dynamic Shape | Batch Inference | Note |
|
||||
| 模型 | 配置 | 动态尺寸 | 批推理 | 注 |
|
||||
|:------:|:------------------------------------------------------------------------------------------------------------------------------------------------:|:-------------:|:---------------:|:----:|
|
||||
| DBNet | [dbnet_r18_fpnc_1200e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/dbnet/dbnet_r18_fpnc_1200e_icdar2015.py) | Y | N | |
|
||||
| PSENet | [psenet_r50_fpnf_600e_ctw1500.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/psenet/psenet_r50_fpnf_600e_ctw1500.py) | Y | Y | |
|
||||
| PSENet | [psenet_r50_fpnf_600e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/psenet/psenet_r50_fpnf_600e_icdar2015.py) | Y | Y | |
|
||||
| PANet | [panet_r18_fpem_ffm_600e_ctw1500.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/panet/panet_r18_fpem_ffm_600e_ctw1500.py) | Y | Y | |
|
||||
| PANet | [panet_r18_fpem_ffm_600e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/panet/panet_r18_fpem_ffm_600e_icdar2015.py) | Y | Y | |
|
||||
| CRNN | [crnn_academic_dataset.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textrecog/crnn/crnn_academic_dataset.py) | Y | Y | CRNN only accepts input with height 32 |
|
||||
| CRNN | [crnn_academic_dataset.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textrecog/crnn/crnn_academic_dataset.py) | Y | Y | CRNN 仅接受高度为32的输入 |
|
||||
|
||||
:::{note}
|
||||
- *All models above are tested with PyTorch==1.8.1 and onnxruntime==1.7.0*
|
||||
- If you meet any problem with the listed models above, please create an issue and it would be taken care of soon.
|
||||
- Because this feature is experimental and may change fast, please always try with the latest `mmcv` and `mmocr`.
|
||||
- *以上所有模型测试基于 PyTorch==1.8.1，onnxruntime==1.7.0 进行*
|
||||
- 如果你在上述模型中遇到问题，请创建一个issue，我们会尽快处理。
|
||||
- 因为这个特性是试验性的，可能变动很快，请尽量使用最新版的 `mmcv` 和 `mmocr` 尝试。
|
||||
:::
|
||||
|
||||
## Convert ONNX to TensorRT (experimental)
|
||||
## ONNX 转 TensorRT （试验性的）
|
||||
|
||||
We also provide a script to convert [ONNX](https://github.com/onnx/onnx) model to [TensorRT](https://github.com/NVIDIA/TensorRT) format. Besides, we support comparing the output results between ONNX and TensorRT model.
|
||||
我们也提供了从 [ONNX](https://github.com/onnx/onnx) 模型转换至 [TensorRT](https://github.com/NVIDIA/TensorRT) 格式的脚本。另外，我们支持比较 ONNX 和 TensorRT 模型的输出结果。
|
||||
|
||||
|
||||
```bash
|
||||
|
|
@ -81,55 +81,55 @@ python tools/deployment/onnx2tensorrt.py
|
|||
--verbose
|
||||
```
|
||||
|
||||
Description of arguments:
|
||||
参数说明：
|
||||
|
||||
| ARGS | Type | Description |
|
||||
| -------------- | --------------------- | ----------------------------------------------------------- |
|
||||
| `model_config` | str | The path to a model config file. |
|
||||
| `model_type` | 'recog', 'det' | The model type of the config file. |
|
||||
| `image_path` | str | The path to input image file. |
|
||||
| `onnx_file` | str | The path to input ONNX file. |
|
||||
| `--trt-file` | str | The path of output TensorRT model. Defaults to `tmp.trt`. |
|
||||
| `--max-shape` | int * 4 | Maximum shape of model input. |
|
||||
| `--min-shape` | int * 4 | Minimum shape of model input. |
|
||||
| `--workspace-size`| int | Max workspace size in GiB. Defaults to 1. |
|
||||
| `--fp16`| bool | Determines whether to export TensorRT with fp16 mode. Defaults to `False`. |
|
||||
| `--verify`| bool | Determines whether to verify the correctness of an exported model. Defaults to `False`. |
|
||||
| `--show`| bool | Determines whether to show the output of ONNX and TensorRT. Defaults to `False`. |
|
||||
| `--verbose`| bool | Determines whether to verbose logging messages while creating TensorRT engine. Defaults to `False`. |
|
||||
| 参数 | 类型 | 描述 |
|
||||
| ------------------ | -------------- | ------------------------------------------------------------ |
|
||||
| `model_config` | str | 模型配置文件的路径。 |
|
||||
| `model_type` | 'recog', 'det' | 配置文件对应的模型类型。 |
|
||||
| `image_path` | str | 输入图片的路径。 |
|
||||
| `onnx_file` | str | 输入的 ONNX 文件路径。 |
|
||||
| `--trt-file` | str | 输出的 TensorRT 模型路径。默认为 `tmp.trt`。 |
|
||||
| `--max-shape` | int * 4 | 模型输入的最大尺寸。 |
|
||||
| `--min-shape` | int * 4 | 模型输入的最小尺寸。 |
|
||||
| `--workspace-size` | int | 最大工作空间大小，单位为 GiB。默认为1。 |
|
||||
| `--fp16` | bool | 决定是否输出 fp16 模式的 TensorRT 模型。默认为 `False`。 |
|
||||
| `--verify` | bool | 决定是否验证输出模型的正确性。默认为 `False`。 |
|
||||
| `--show` | bool | 决定是否可视化 ONNX 和 TensorRT 的输出。默认为 `False`。 |
|
||||
| `--verbose` | bool | 决定是否在创建 TensorRT 引擎时打印日志信息。默认为 `False`。 |
|
||||
|
||||
:::{note}
|
||||
This tool is still experimental. For now, some customized operators are not supported, and we only support a subset of detection and recognition algorithms.
|
||||
这个工具仍然是试验性的。一些定制的操作模型没有被支持。我们目前仅支持一部的文本检测和文本识别算法。
|
||||
:::
|
||||
|
||||
### List of supported models exportable to TensorRT
|
||||
### 支持导出到 TensorRT 的模型列表
|
||||
|
||||
The table below lists the models that are guaranteed to be exportable to TensorRT engine and runnable in TensorRT.
|
||||
下表列出的模型可以保证导出到 TensorRT 引擎并且可以在 TensorRT 下运行。
|
||||
|
||||
| Model | Config | Dynamic Shape | Batch Inference | Note |
|
||||
| 模型 | 配置 | 动态尺寸 | 批推理 | 注 |
|
||||
|:------:|:------------------------------------------------------------------------------------------------------------------------------------------------:|:-------------:|:---------------:|:----:|
|
||||
| DBNet | [dbnet_r18_fpnc_1200e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/dbnet/dbnet_r18_fpnc_1200e_icdar2015.py) | Y | N | |
|
||||
| PSENet | [psenet_r50_fpnf_600e_ctw1500.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/psenet/psenet_r50_fpnf_600e_ctw1500.py) | Y | Y | |
|
||||
| PSENet | [psenet_r50_fpnf_600e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/psenet/psenet_r50_fpnf_600e_icdar2015.py) | Y | Y | |
|
||||
| PANet | [panet_r18_fpem_ffm_600e_ctw1500.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/panet/panet_r18_fpem_ffm_600e_ctw1500.py) | Y | Y | |
|
||||
| PANet | [panet_r18_fpem_ffm_600e_icdar2015.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textdet/panet/panet_r18_fpem_ffm_600e_icdar2015.py) | Y | Y | |
|
||||
| CRNN | [crnn_academic_dataset.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textrecog/crnn/crnn_academic_dataset.py) | Y | Y | CRNN only accepts input with height 32 |
|
||||
| CRNN | [crnn_academic_dataset.py](https://github.com/open-mmlab/mmocr/blob/main/configs/textrecog/crnn/crnn_academic_dataset.py) | Y | Y | CRNN 仅接受高度为32的输入 |
|
||||
|
||||
:::{note}
|
||||
- *All models above are tested with PyTorch==1.8.1, onnxruntime==1.7.0 and tensorrt==7.2.1.6*
|
||||
- If you meet any problem with the listed models above, please create an issue and it would be taken care of soon.
|
||||
- Because this feature is experimental and may change fast, please always try with the latest `mmcv` and `mmocr`.
|
||||
- *以上所有模型测试基于 PyTorch==1.8.1，onnxruntime==1.7.0，tensorrt==7.2.1.6 进行*
|
||||
- 如果你在上述模型中遇到问题，请创建一个 issue，我们会尽快处理。
|
||||
- 因为这个特性是试验性的，可能变动很快，请尽量使用最新版的 `mmcv` 和 `mmocr` 尝试。
|
||||
:::
|
||||
|
||||
|
||||
## Evaluate ONNX and TensorRT Models (experimental)
|
||||
## 评估 ONNX 和 TensorRT 模型（试验性的）
|
||||
|
||||
We provide methods to evaluate TensorRT and ONNX models in `tools/deployment/deploy_test.py`.
|
||||
我们在 `tools/deployment/deploy_test.py ` 中提供了评估 TensorRT 和 ONNX 模型的方法。
|
||||
|
||||
### Prerequisite
|
||||
To evaluate ONNX and TensorRT models, ONNX, ONNXRuntime and TensorRT should be installed first. Install `mmcv-full` with ONNXRuntime custom ops and TensorRT plugins follow [ONNXRuntime in mmcv](https://mmcv.readthedocs.io/en/latest/onnxruntime_op.html) and [TensorRT plugin in mmcv](https://github.com/open-mmlab/mmcv/blob/master/docs/tensorrt_plugin.md).
|
||||
### 前提条件
|
||||
在评估 ONNX 和 TensorRT 模型之前，首先需要安装 ONNX，ONNXRuntime 和 TensorRT。根据 [ONNXRuntime in mmcv](https://mmcv.readthedocs.io/en/latest/onnxruntime_op.html) 和 [TensorRT plugin in mmcv](https://github.com/open-mmlab/mmcv/blob/master/docs/tensorrt_plugin.md) 安装 ONNXRuntime 定制操作和 TensorRT 插件。
|
||||
|
||||
### Usage
|
||||
### 使用
|
||||
|
||||
```bash
|
||||
python tools/deploy_test.py \
|
||||
|
|
@ -141,27 +141,26 @@ python tools/deploy_test.py \
|
|||
--device ${DEVICE}
|
||||
```
|
||||
|
||||
### Description of all arguments
|
||||
### 参数说明
|
||||
|
||||
| ARGS | Type | Description |
|
||||
| -------------- | --------------------- | ----------------------------------------------------------- |
|
||||
| `model_config` | str | The path to a model config file. |
|
||||
| `model_file` | str | The path to a TensorRT or an ONNX model file. |
|
||||
| `model_type` | 'recog', 'det' | Detection or recognition model to deploy. |
|
||||
| `backend` | 'TensorRT', 'ONNXRuntime' | The backend for testing. |
|
||||
| `--eval` | 'acc', 'hmean-iou' | The evaluation metrics. 'acc' for recognition models, 'hmean-iou' for detection models. |
|
||||
| `--device` | str | Device for evaluation. Defaults to `cuda:0`. |
|
||||
|
||||
## Results and Models
|
||||
| 参数 | 类型 | 描述 |
|
||||
| -------------- | ------------------------- | ------------------------------------------------------ |
|
||||
| `model_config` | str | 模型配置文件的路径。 |
|
||||
| `model_file` | str | TensorRT 或 ONNX 模型路径。 |
|
||||
| `model_type` | 'recog', 'det' | 部署检测还是识别模型。 |
|
||||
| `backend` | 'TensorRT', 'ONNXRuntime' | 测试后端。 |
|
||||
| `--eval` | 'acc', 'hmean-iou' | 评估指标。“acc”用于识别模型，“hmean-iou”用于检测模型。 |
|
||||
| `--device` | str | 评估使用的设备。默认为 `cuda:0`。 |
|
||||
|
||||
## 结果和模型
|
||||
|
||||
<table class="tg">
|
||||
<thead>
|
||||
<tr>
|
||||
<th class="tg-9wq8">Model</th>
|
||||
<th class="tg-9wq8">Config</th>
|
||||
<th class="tg-9wq8">Dataset</th>
|
||||
<th class="tg-9wq8">Metric</th>
|
||||
<th class="tg-9wq8">模型</th>
|
||||
<th class="tg-9wq8">配置</th>
|
||||
<th class="tg-9wq8">数据集</th>
|
||||
<th class="tg-9wq8">指标</th>
|
||||
<th class="tg-9wq8">PyTorch</th>
|
||||
<th class="tg-9wq8">ONNX Runtime</th>
|
||||
<th class="tg-9wq8">TensorRT FP32</th>
|
||||
|
|
@ -303,8 +302,8 @@ python tools/deploy_test.py \
|
|||
</table>
|
||||
|
||||
:::{note}
|
||||
- TensorRT upsampling operation is a little different from PyTorch. For DBNet and PANet, we suggest replacing upsampling operations with the nearest mode to operations with bilinear mode. [Here](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpem_ffm.py#L33) for PANet, [here](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpn_cat.py#L111) and [here](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpn_cat.py#L121) for DBNet. As is shown in the above table, networks with tag * mean the upsampling mode is changed.
|
||||
- Note that changing upsampling mode reduces less performance compared with using the nearest mode. However, the weights of networks are trained through the nearest mode. To pursue the best performance, using bilinear mode for both training and TensorRT deployment is recommended.
|
||||
- All ONNX and TensorRT models are evaluated with dynamic shapes on the datasets, and images are preprocessed according to the original config file.
|
||||
- This tool is still experimental, and we only support a subset of detection and recognition algorithms for now.
|
||||
- TensorRT 上采样（upsample）操作和 PyTorch 有一点不同。对于 DBNet 和 PANet，我们建议把上采样的最近邻 (nearest) 模式代替成双线性 (bilinear) 模式。 PANet 的替换处在[这里](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpem_ffm.py#L33) ，DBNet 的替换处在[这里](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpn_cat.py#L111)和[这里](https://github.com/open-mmlab/mmocr/blob/50a25e718a028c8b9d96f497e241767dbe9617d1/mmocr/models/textdet/necks/fpn_cat.py#L121)。如在上表中显示的，带有标记*的网络的上采样模式均被改变了。
|
||||
- 注意到，相比最近邻模式，使用更改后的上采样模式会降低性能。然而，默认网络的权重是通过最近邻模式训练的。为了保持在部署中的最佳性能，建议在训练和 TensorRT 部署中使用双线性模式。
|
||||
- 所有 ONNX 和 TensorRT 模型都使用数据集上的动态尺寸进行评估，图像根据原始配置文件进行预处理。
|
||||
- 这个工具仍然是试验性的。一些定制的操作模型没有被支持。并且我们目前仅支持一部分的文本检测和文本识别算法。
|
||||
:::
|
||||
|
|
|
|||
Loading…
Reference in New Issue