mirrors
/
mmdeploy
mirror of https://github.com/open-mmlab/mmdeploy.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Update docstring and docs of mmseg and mmpose () 

* fix reg

* update docstring

* Revert "fix reg"

This reverts commit fe56bffb5b.

* update benchmark of mmseg and mmpose

* fix ci
pull/1342/head
RunningLeon 2022-11-09 11:03:19 +08:00 committed by GitHub
parent 0da93ea3e3
commit 20a7c35ca7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 187 additions and 132 deletions

66
docs/en/03-benchmark/benchmark.md
View File

@ -373,7 +373,7 @@ Users can directly test the speed through [model profiling](../02-how-to-run/pro
<td align="center" colspan="1">fp16</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fcn/fcn_r50-d8_512x1024_40k_cityscapes.py">FCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fcn/fcn_r50-d8_4xb2-40k_cityscapes-512x1024.py">FCN</a></td>
<td align="center">512x1024</td>
<td align="center">128.42</td>
<td align="center">23.97</td>
@ -382,7 +382,7 @@ Users can directly test the speed through [model profiling](../02-how-to-run/pro
<td align="center">27.00</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py">PSPNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/pspnet/pspnet_r50-d8_4xb2-80k_cityscapes-512x1024.py">PSPNet</a></td>
<td align="center">1x3x512x1024</td>
<td align="center">119.77</td>
<td align="center">24.10</td>
@ -391,7 +391,7 @@ Users can directly test the speed through [model profiling](../02-how-to-run/pro
<td align="center">27.26</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3/deeplabv3_r50-d8_512x1024_80k_cityscapes.py">DeepLabV3</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3/deeplabv3/deeplabv3_r50-d8_4xb2-80k_cityscapes-512x1024.py">DeepLabV3</a></td>
<td align="center">512x1024</td>
<td align="center">226.75</td>
<td align="center">31.80</td>
@ -400,7 +400,7 @@ Users can directly test the speed through [model profiling](../02-how-to-run/pro
<td align="center">36.01</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3plus/deeplabv3plus_r50-d8_512x1024_80k_cityscapes.py">DeepLabV3+</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3plus/deeplabv3plus_r50-d8_4xb2-80k_cityscapes-512x1024.py">DeepLabV3+</a></td>
<td align="center">512x1024</td>
<td align="center">151.25</td>
<td align="center">47.03</td>
@ -1274,7 +1274,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">fp32</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fcn/fcn_r50-d8_512x1024_40k_cityscapes.py">FCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fcn/fcn_r50-d8_4xb2-40k_cityscapes-512x1024.py">FCN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">72.25</td>
@ -1287,7 +1287,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">72.35</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py">PSPNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/pspnet/pspnet_r50-d8_4xb2-80k_cityscapes-512x1024.py">PSPNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">78.55</td>
@ -1300,7 +1300,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">78.67</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3/deeplabv3_r50-d8_512x1024_40k_cityscapes.py">deeplabv3</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3/deeplabv3_r50-d8_4xb2-40k_cityscapes-512x1024.py">deeplabv3</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.09</td>
@ -1313,7 +1313,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">79.06</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3plus/deeplabv3plus_r50-d8_512x1024_40k_cityscapes.py">deeplabv3+</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3plus/deeplabv3plus_r50-d8_4xb2-40k_cityscapes-512x1024.py">deeplabv3+</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.61</td>
@ -1326,7 +1326,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">79.51</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastscnn/fast_scnn_lr0.12_8x4_160k_cityscapes.py">Fast-SCNN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fastscnn/fast_scnn_8xb4-160k_cityscapes-512x1024.py">Fast-SCNN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">70.96</td>
@ -1339,7 +1339,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/unet/fcn_unet_s5-d16_4x4_512x1024_160k_cityscapes.py">UNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/unet/unet-s5-d16_fcn_4xb4-160k_cityscapes-512x1024.py">UNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">69.10</td>
@ -1352,7 +1352,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ann/ann_r50-d8_512x1024_40k_cityscapes.py">ANN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-512x1024.py">ANN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
@ -1365,7 +1365,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes.py">APCNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">APCNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
@ -1378,7 +1378,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes.py">BiSeNetV1</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_cityscapes-1024x1024.py">BiSeNetV1</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.44</td>
@ -1391,7 +1391,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes.py">BiSeNetV2</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/bisenetv2/bisenetv2_fcn_4xb4-160k_cityscapes-1024x1024.py">BiSeNetV2</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">73.21</td>
@ -1404,7 +1404,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/cgnet/cgnet_512x1024_60k_cityscapes.py">CGNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/cgnet/cgnet_fcn_4xb8-60k_cityscapes-512x1024.py">CGNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">68.25</td>
@ -1417,7 +1417,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/emanet/emanet_r50-d8_512x1024_80k_cityscapes.py">EMANet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/emanet/emanet_r50-d8_4xb2-80k_cityscapes-512x1024.py">EMANet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.59</td>
@ -1430,7 +1430,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/encnet/encnet_r50-d8_512x1024_40k_cityscapes.py">EncNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/encnet/encnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">EncNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.67</td>
@ -1443,7 +1443,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/erfnet/erfnet_fcn_4x4_512x1024_160k_cityscapes.py">ERFNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/erfnet/erfnet_fcn_4xb4-160k_cityscapes-512x1024.py">ERFNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">71.08</td>
@ -1456,7 +1456,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py">FastFCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4xb2-80k_cityscapes-512x1024.py">FastFCN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.12</td>
@ -1469,7 +1469,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/gcnet/gcnet_r50-d8_512x1024_40k_cityscapes.py">GCNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/gcnet/gcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">GCNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.69</td>
@ -1482,7 +1482,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/icnet/icnet_r18-d8_832x832_80k_cityscapes.py">ICNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/icnet/icnet_r18-d8_4xb2-80k_cityscapes-832x832.py">ICNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.29</td>
@ -1495,7 +1495,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/isanet/isanet_r50-d8_512x1024_40k_cityscapes.py">ISANet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/isanet/isanet_r50-d8_4xb2-40k_cityscapes-512x1024.py">ISANet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">78.49</td>
@ -1508,7 +1508,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ocrnet/ocrnet_hr18s_512x1024_40k_cityscapes.py">OCRNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/ocrnet/ocrnet_hr18s_4xb2-40k_cityscapes-512x1024.py">OCRNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.30</td>
@ -1521,7 +1521,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/point_rend/pointrend_r50_512x1024_80k_cityscapes.py">PointRend</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/point_rend/pointrend_r50_4xb2-80k_cityscapes-512x1024.py">PointRend</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.47</td>
@ -1534,7 +1534,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/sem_fpn/fpn_r50_512x1024_80k_cityscapes.py">Semantic FPN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/sem_fpn/fpn_r50_4xb2-80k_cityscapes-512x1024.py">Semantic FPN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.52</td>
@ -1547,7 +1547,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/stdc/stdc1_in1k-pre_512x1024_80k_cityscapes.py">STDC</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/stdc/stdc1_in1k-pre_4xb12-80k_cityscapes-512x1024.py">STDC</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.10</td>
@ -1560,7 +1560,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/stdc/stdc2_in1k-pre_512x1024_80k_cityscapes.py">STDC</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/stdc/stdc2_in1k-pre_4xb12-80k_cityscapes-512x1024.py">STDC</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.17</td>
@ -1573,7 +1573,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/upernet/upernet_r50_512x1024_40k_cityscapes.py">UPerNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/upernet/upernet_r50_4xb2-40k_cityscapes-512x1024.py">UPerNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.10</td>
@ -1586,7 +1586,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/blob/master/configs/segmenter/segmenter_vit-s_linear_8x1_512x512_160k_ade20k.py">Segmenter</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/blob/1.x/configs/segmenter/segmenter_vit-s_fcn_8xb1-160k_ade20k-512x512.py">Segmenter</a></td>
<td align="center">ADE20K</td>
<td align="center">mIoU</td>
<td align="center">44.32</td>
@ -1627,7 +1627,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">fp32</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py">HRNet</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py">HRNet</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1648,7 +1648,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">0.802</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_30_coco_256x192.py">LiteHRNet</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_litehrnet-30_8xb64-210e_coco-256x192.py">LiteHRNet</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1669,7 +1669,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">0.728</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/4xmspn50_coco_256x192.py">MSPN</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_4xmspn50_8xb32-210e_coco-256x192.py">MSPN</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1690,7 +1690,7 @@ Users can directly test the performance through [how_to_evaluate_a_model.md](../
<td align="center">0.825</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_256x256.py">Hourglass</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hourglass52_8xb32-210e_coco-256x256.py">Hourglass</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>

12
docs/en/03-benchmark/benchmark_edge.md
View File

@ -28,9 +28,9 @@ tips:
## mmpose
| model | dataset | spatial | snpe hybrid AR@IoU=0.50 | snpe hybrid AP@IoU=0.50 | latency(ms) |
| :---------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :-----: | :---------------------: | :---------------------: | :---------: |
| [pose_hrnet_w32](https://github.com/open-mmlab/mmpose/blob/master/configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/hrnet_w32_animalpose_256x256.py) | Animalpose | 256x256 | 0.997 | 0.989 | 630±50 |
| model | dataset | spatial | snpe hybrid AR@IoU=0.50 | snpe hybrid AP@IoU=0.50 | latency(ms) |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :-----: | :---------------------: | :---------------------: | :---------: |
| [pose_hrnet_w32](https://github.com/open-mmlab/mmpose/blob/1.x/configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py) | Animalpose | 256x256 | 0.997 | 0.989 | 630±50 |
tips:
@ -38,9 +38,9 @@ tips:
## mmseg
| model | dataset | spatial | mIoU | latency(ms) |
| :---------------------------------------------------------------------------------------------------------------: | :--------: | :------: | :---: | :---------: |
| [fcn](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fcn/fcn_r18-d8_512x1024_80k_cityscapes.py) | Cityscapes | 512x1024 | 71.11 | 4915±500 |
| model | dataset | spatial | mIoU | latency(ms) |
| :-----------------------------------------------------------------------------------------------------------------: | :--------: | :------: | :---: | :---------: |
| [fcn](https://github.com/open-mmlab/mmsegmentation/blob/1.x/configs/fcn/fcn_r18-d8_4xb2-80k_cityscapes-512x1024.py) | Cityscapes | 512x1024 | 71.11 | 4915±500 |
tips:

6
docs/en/03-benchmark/quantization.md
View File

@ -29,8 +29,8 @@ Note: [mmocr](https://github.com/open-mmlab/mmocr) Uses 'shapely' to compute I
### Pose detection
| model | dataset | fp32 AP | int8 AP |
| :----------------------------------------------------------------------------------------------------------------------------------------------: | :------: | :-----: | :-----: |
| [Hourglass](https://github.com/open-mmlab/mmpose/blob/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_256x256.py) | COCO2017 | 0.717 | 0.713 |
| model | dataset | fp32 AP | int8 AP |
| :---------------------------------------------------------------------------------------------------------------------------------------------------: | :------: | :-----: | :-----: |
| [Hourglass](https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hourglass52_8xb32-210e_coco-256x256.py) | COCO2017 | 0.717 | 0.713 |
Note: MMPose models are tested with `flip_test` explicitly set to `False` in model configs.

23
docs/en/04-supported-codebases/mmseg.md
View File

@ -1,15 +1,16 @@
# MMSegmentation Deployment
- [Installation](#installation)
- [Install mmseg](#install-mmseg)
- [Install mmdeploy](#install-mmdeploy)
- [Convert model](#convert-model)
- [Model specification](#model-specification)
- [Model inference](#model-inference)
- [Backend model inference](#backend-model-inference)
- [SDK model inference](#sdk-model-inference)
- [Supported models](#supported-models)
- [Reminder](#reminder)
- [MMSegmentation Deployment](#mmsegmentation-deployment)
- [Installation](#installation)
- [Install mmseg](#install-mmseg)
- [Install mmdeploy](#install-mmdeploy)
- [Convert model](#convert-model)
- [Model specification](#model-specification)
- [Model inference](#model-inference)
- [Backend model inference](#backend-model-inference)
- [SDK model inference](#sdk-model-inference)
- [Supported models](#supported-models)
- [Reminder](#reminder)
______________________________________________________________________
@ -227,6 +228,6 @@ Besides python API, mmdeploy SDK also provides other FFI (Foreign Function Inter
- Only `whole` inference mode is supported for all mmseg models.
- <i id="static_shape">PSPNet, Fast-SCNN</i> only support static shape, because [nn.AdaptiveAvgPool2d](https://github.com/open-mmlab/mmsegmentation/blob/97f9670c5a4a2a3b4cfb411bcc26db16b23745f7/mmseg/models/decode_heads/psp_head.py#L38) is not supported by most inference backends.
- <i id="static_shape">PSPNet, Fast-SCNN</i> only support static shape, because [nn.AdaptiveAvgPool2d](https://github.com/open-mmlab/mmsegmentation/blob/0c87f7a0c9099844eff8e90fa3db5b0d0ca02fee/mmseg/models/decode_heads/psp_head.py#L38) is not supported by most inference backends.
- For models that only supports static shape, you should use the deployment config file of static shape such as `configs/mmseg/segmentation_tensorrt_static-1024x2048.py`.

93
docs/zh_cn/03-benchmark/benchmark.md
View File

@ -118,7 +118,6 @@ GPU: ncnn, TensorRT, PPLNN
</tr>
</tbody>
</table>
<table class="docutils">
<thead>
<tr>
@ -142,8 +141,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center" colspan="1">fp32</td>
<td align="center" colspan="1">fp16</td>
</tr>
<tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/master/configs/yolo/yolov3_d53_320_273e_coco.py">YOLOv3</a></td>
<td align="center">320x320</td>
<td align="center">14.76</td>
@ -372,7 +370,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center" colspan="1">fp16</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fcn/fcn_r50-d8_512x1024_40k_cityscapes.py">FCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fcn/fcn_r50-d8_4xb2-40k_cityscapes-512x1024.py">FCN</a></td>
<td align="center">512x1024</td>
<td align="center">128.42</td>
<td align="center">23.97</td>
@ -381,7 +379,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">27.00</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py">PSPNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/pspnet/pspnet_r50-d8_4xb2-80k_cityscapes-512x1024.py">PSPNet</a></td>
<td align="center">1x3x512x1024</td>
<td align="center">119.77</td>
<td align="center">24.10</td>
@ -390,7 +388,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">27.26</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3/deeplabv3_r50-d8_512x1024_80k_cityscapes.py">DeepLabV3</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3/deeplabv3/deeplabv3_r50-d8_4xb2-80k_cityscapes-512x1024.py">DeepLabV3</a></td>
<td align="center">512x1024</td>
<td align="center">226.75</td>
<td align="center">31.80</td>
@ -399,7 +397,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">36.01</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3plus/deeplabv3plus_r50-d8_512x1024_80k_cityscapes.py">DeepLabV3+</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3plus/deeplabv3plus_r50-d8_4xb2-80k_cityscapes-512x1024.py">DeepLabV3+</a></td>
<td align="center">512x1024</td>
<td align="center">151.25</td>
<td align="center">47.03</td>
@ -600,6 +598,27 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
<td align="center">97.77</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmclassification/blob/master/configs/swin_transformer/swin-tiny_16xb64_in1k.py">Swin Transformer</a></td>
<td align="center">top-1</td>
<td align="center">81.18</td>
<td align="center">81.18</td>
<td align="center">81.18</td>
<td align="center">81.18</td>
<td align="center">81.18</td>
<td align="center">-</td>
<td align="center">-</td>
</tr>
<tr>
<td align="center">top-5</td>
<td align="center">95.61</td>
<td align="center">95.61</td>
<td align="center">95.61</td>
<td align="center">95.61</td>
<td align="center">95.61</td>
<td align="center">-</td>
<td align="center">-</td>
</tr>
</tbody>
</table>
</div>
@ -727,7 +746,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">37.3</td>
<td align="center">37.1</td>
<td align="center">37.3</td>
<td align="center">-</td>
<td align="center">37.2</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmdetection/tree/master/configs/atss/atss_r50_fpn_1x_coco.py">ATSS</a></td>
@ -1250,7 +1269,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">fp32</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fcn/fcn_r50-d8_512x1024_40k_cityscapes.py">FCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fcn/fcn_r50-d8_4xb2-40k_cityscapes-512x1024.py">FCN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">72.25</td>
@ -1263,7 +1282,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">72.35</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py">PSPNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/pspnet/pspnet_r50-d8_4xb2-80k_cityscapes-512x1024.py">PSPNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">78.55</td>
@ -1276,7 +1295,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">78.67</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3/deeplabv3_r50-d8_512x1024_40k_cityscapes.py">deeplabv3</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3/deeplabv3_r50-d8_4xb2-40k_cityscapes-512x1024.py">deeplabv3</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.09</td>
@ -1289,7 +1308,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">79.06</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/deeplabv3plus/deeplabv3plus_r50-d8_512x1024_40k_cityscapes.py">deeplabv3+</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/deeplabv3plus/deeplabv3plus_r50-d8_4xb2-40k_cityscapes-512x1024.py">deeplabv3+</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.61</td>
@ -1302,7 +1321,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">79.51</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastscnn/fast_scnn_lr0.12_8x4_160k_cityscapes.py">Fast-SCNN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fastscnn/fast_scnn_8xb4-160k_cityscapes-512x1024.py">Fast-SCNN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">70.96</td>
@ -1315,7 +1334,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/unet/fcn_unet_s5-d16_4x4_512x1024_160k_cityscapes.py">UNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/unet/unet-s5-d16_fcn_4xb4-160k_cityscapes-512x1024.py">UNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">69.10</td>
@ -1328,7 +1347,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ann/ann_r50-d8_512x1024_40k_cityscapes.py">ANN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/ann/ann_r50-d8_4xb2-40k_cityscapes-512x1024.py">ANN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
@ -1341,7 +1360,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/apcnet/apcnet_r50-d8_512x1024_40k_cityscapes.py">APCNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/apcnet/apcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">APCNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.40</td>
@ -1354,7 +1373,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv1/bisenetv1_r18-d32_4x4_1024x1024_160k_cityscapes.py">BiSeNetV1</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/bisenetv1/bisenetv1_r18-d32_4xb4-160k_cityscapes-1024x1024.py">BiSeNetV1</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.44</td>
@ -1367,7 +1386,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/bisenetv2/bisenetv2_fcn_4x4_1024x1024_160k_cityscapes.py">BiSeNetV2</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/bisenetv2/bisenetv2_fcn_4xb4-160k_cityscapes-1024x1024.py">BiSeNetV2</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">73.21</td>
@ -1380,7 +1399,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/cgnet/cgnet_512x1024_60k_cityscapes.py">CGNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/cgnet/cgnet_fcn_4xb8-60k_cityscapes-512x1024.py">CGNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">68.25</td>
@ -1393,7 +1412,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/emanet/emanet_r50-d8_512x1024_80k_cityscapes.py">EMANet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/emanet/emanet_r50-d8_4xb2-80k_cityscapes-512x1024.py">EMANet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.59</td>
@ -1406,7 +1425,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/encnet/encnet_r50-d8_512x1024_40k_cityscapes.py">EncNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/encnet/encnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">EncNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.67</td>
@ -1419,7 +1438,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/erfnet/erfnet_fcn_4x4_512x1024_160k_cityscapes.py">ERFNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/erfnet/erfnet_fcn_4xb4-160k_cityscapes-512x1024.py">ERFNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">71.08</td>
@ -1432,7 +1451,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py">FastFCN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4xb2-80k_cityscapes-512x1024.py">FastFCN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">79.12</td>
@ -1445,7 +1464,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/gcnet/gcnet_r50-d8_512x1024_40k_cityscapes.py">GCNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/gcnet/gcnet_r50-d8_4xb2-40k_cityscapes-512x1024.py">GCNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.69</td>
@ -1458,7 +1477,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/icnet/icnet_r18-d8_832x832_80k_cityscapes.py">ICNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/icnet/icnet_r18-d8_4xb2-80k_cityscapes-832x832.py">ICNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.29</td>
@ -1471,7 +1490,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/isanet/isanet_r50-d8_512x1024_40k_cityscapes.py">ISANet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/isanet/isanet_r50-d8_4xb2-40k_cityscapes-512x1024.py">ISANet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">78.49</td>
@ -1484,7 +1503,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/ocrnet/ocrnet_hr18s_512x1024_40k_cityscapes.py">OCRNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/ocrnet/ocrnet_hr18s_4xb2-40k_cityscapes-512x1024.py">OCRNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.30</td>
@ -1497,7 +1516,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/point_rend/pointrend_r50_512x1024_80k_cityscapes.py">PointRend</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/point_rend/pointrend_r50_4xb2-80k_cityscapes-512x1024.py">PointRend</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">76.47</td>
@ -1510,7 +1529,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/sem_fpn/fpn_r50_512x1024_80k_cityscapes.py">Semantic FPN</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/sem_fpn/fpn_r50_4xb2-80k_cityscapes-512x1024.py">Semantic FPN</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">74.52</td>
@ -1523,7 +1542,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/stdc/stdc1_in1k-pre_512x1024_80k_cityscapes.py">STDC</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/stdc/stdc1_in1k-pre_4xb12-80k_cityscapes-512x1024.py">STDC</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">75.10</td>
@ -1536,7 +1555,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/stdc/stdc2_in1k-pre_512x1024_80k_cityscapes.py">STDC</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/stdc/stdc2_in1k-pre_4xb12-80k_cityscapes-512x1024.py">STDC</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.17</td>
@ -1549,7 +1568,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/master/configs/upernet/upernet_r50_512x1024_40k_cityscapes.py">UPerNet</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/tree/1.x/configs/upernet/upernet_r50_4xb2-40k_cityscapes-512x1024.py">UPerNet</a></td>
<td align="center">Cityscapes</td>
<td align="center">mIoU</td>
<td align="center">77.10</td>
@ -1562,7 +1581,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">-</td>
</tr>
<tr>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/blob/master/configs/segmenter/segmenter_vit-s_linear_8x1_512x512_160k_ade20k.py">Segmenter</a></td>
<td align="center"><a href="https://github.com/open-mmlab/mmsegmentation/blob/1.x/configs/segmenter/segmenter_vit-s_fcn_8xb1-160k_ade20k-512x512.py">Segmenter</a></td>
<td align="center">ADE20K</td>
<td align="center">mIoU</td>
<td align="center">44.32</td>
@ -1603,7 +1622,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">fp32</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py">HRNet</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py">HRNet</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1624,7 +1643,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">0.802</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/litehrnet_30_coco_256x192.py">LiteHRNet</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_litehrnet-30_8xb64-210e_coco-256x192.py">LiteHRNet</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1645,7 +1664,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">0.728</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/tree/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/4xmspn50_coco_256x192.py">MSPN</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_4xmspn50_8xb32-210e_coco-256x192.py">MSPN</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>
@ -1666,7 +1685,7 @@ GPU: ncnn, TensorRT, PPLNN
<td align="center">0.825</td>
</tr>
<tr>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_256x256.py">Hourglass</a></td>
<td align="center" rowspan="2"><a href="https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hourglass52_8xb32-210e_coco-256x256.py">Hourglass</a></td>
<td align="center" rowspan="2">Pose Detection</td>
<td align="center" rowspan="2">COCO</td>
<td align="center">AP</td>

12
docs/zh_cn/03-benchmark/benchmark_edge.md
View File

@ -28,9 +28,9 @@ tips:
## mmpose 模型
| model | dataset | spatial | snpe hybrid AR@IoU=0.50 | snpe hybrid AP@IoU=0.50 | latency(ms) |
| :---------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :-----: | :---------------------: | :---------------------: | :---------: |
| [pose_hrnet_w32](https://github.com/open-mmlab/mmpose/blob/master/configs/animal/2d_kpt_sview_rgb_img/topdown_heatmap/animalpose/hrnet_w32_animalpose_256x256.py) | Animalpose | 256x256 | 0.997 | 0.989 | 630±50 |
| model | dataset | spatial | snpe hybrid AR@IoU=0.50 | snpe hybrid AP@IoU=0.50 | latency(ms) |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :--------: | :-----: | :---------------------: | :---------------------: | :---------: |
| [pose_hrnet_w32](https://github.com/open-mmlab/mmpose/blob/1.x/configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py) | Animalpose | 256x256 | 0.997 | 0.989 | 630±50 |
tips:
@ -38,9 +38,9 @@ tips:
## mmseg
| model | dataset | spatial | mIoU | latency(ms) |
| :---------------------------------------------------------------------------------------------------------------: | :--------: | :------: | :---: | :---------: |
| [fcn](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fcn/fcn_r18-d8_512x1024_80k_cityscapes.py) | Cityscapes | 512x1024 | 71.11 | 4915±500 |
| model | dataset | spatial | mIoU | latency(ms) |
| :-----------------------------------------------------------------------------------------------------------------: | :--------: | :------: | :---: | :---------: |
| [fcn](https://github.com/open-mmlab/mmsegmentation/blob/1.x/configs/fcn/fcn_r18-d8_4xb2-80k_cityscapes-512x1024.py) | Cityscapes | 512x1024 | 71.11 | 4915±500 |
tips:

6
docs/zh_cn/03-benchmark/quantization.md
View File

@ -29,8 +29,8 @@
### 姿态检测任务
| model | dataset | fp32 AP | int8 AP |
| :----------------------------------------------------------------------------------------------------------------------------------------------: | :------: | :-----: | :-----: |
| [Hourglass](https://github.com/open-mmlab/mmpose/blob/master/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hourglass52_coco_256x256.py) | COCO2017 | 0.717 | 0.713 |
| model | dataset | fp32 AP | int8 AP |
| :---------------------------------------------------------------------------------------------------------------------------------------------------: | :------: | :-----: | :-----: |
| [Hourglass](https://github.com/open-mmlab/mmpose/blob/1.x/configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hourglass52_8xb32-210e_coco-256x256.py) | COCO2017 | 0.717 | 0.713 |
备注：测试转换后的模型精度时，对于 mmpose 模型，在模型配置文件中 `flip_test` 需设置为 `False`。

22
docs/zh_cn/04-supported-codebases/mmseg.md
View File

@ -1,14 +1,16 @@
# MMSegmentation 模型部署
- [安装](#安装)
- [安装 mmcls](#安装-mmseg)
- [安装 mmdeploy](#安装-mmdeploy)
- [模型转换](#模型转换)
- [模型规范](#模型规范)
- [模型推理](#模型推理)
- [后端模型推理](#后端模型推理)
- [SDK 模型推理](#sdk-模型推理)
- [模型支持列表](#模型支持列表)
- [MMSegmentation 模型部署](#mmsegmentation-模型部署)
- [安装](#安装)
- [安装 mmseg](#安装-mmseg)
- [安装 mmdeploy](#安装-mmdeploy)
- [模型转换](#模型转换)
- [模型规范](#模型规范)
- [模型推理](#模型推理)
- [后端模型推理](#后端模型推理)
- [SDK 模型推理](#sdk-模型推理)
- [模型支持列表](#模型支持列表)
- [注意事项](#注意事项)
______________________________________________________________________
@ -230,6 +232,6 @@ cv2.imwrite('output_segmentation.png', img)
- 所有 mmseg 模型仅支持 "whole" 推理模式。
- <i id=“static_shape”>PSPNet，Fast-SCNN</i> 仅支持静态输入，因为多数推理框架的 [nn.AdaptiveAvgPool2d](https://github.com/open-mmlab/mmsegmentation/blob/97f9670c5a4a2a3b4cfb411bcc26db16b23745f7/mmseg/models/decode_heads/psp_head.py#L38) 不支持动态输入。
- <i id=“static_shape”>PSPNet，Fast-SCNN</i> 仅支持静态输入，因为多数推理框架的 [nn.AdaptiveAvgPool2d](https://github.com/open-mmlab/mmsegmentation/blob/0c87f7a0c9099844eff8e90fa3db5b0d0ca02fee/mmseg/models/decode_heads/psp_head.py#L38) 不支持动态输入。
- 对于仅支持静态形状的模型，应使用静态形状的部署配置文件，例如 `configs/mmseg/segmentation_tensorrt_static-1024x2048.py`

2
mmdeploy/codebase/base/backend_model.py
View File

@ -48,7 +48,7 @@ class BaseBackendModel(BaseModel, metaclass=ABCMeta):
Args:
backend (Backend): The backend enum type.
beckend_files (Sequence[str]): Paths to all required backend files(
backend_files (Sequence[str]): Paths to all required backend files(
e.g. '.onnx' for ONNX Runtime, '.param' and '.bin' for ncnn).
device (str): A string specifying device type.
input_names (Sequence[str] | None): Names of model inputs in

11
mmdeploy/codebase/base/runner.py
View File

@ -8,15 +8,22 @@ from mmengine.runner import Runner
class DeployTestRunner(Runner):
"""The runner for test models.
Args:
log_file (str | None): The path of log file. Default is ``None``.
device (str): The device type.
"""
def __init__(self,
log_file: Optional[str] = None,
device: str = get_device(),
*args,
**kwargs):
self._log_file = log_file
self._device = device
super().__init__(*args, **kwargs)
super(DeployTestRunner, self).__init__(*args, **kwargs)
def wrap_model(self, model_wrapper_cfg: Optional[Dict],
model: BaseModel) -> BaseModel:
@ -46,7 +53,7 @@ class DeployTestRunner(Runner):
log_level: Union[int, str] = 'INFO',
log_file: str = None,
**kwargs) -> MMLogger:
"""Build a global asscessable MMLogger.
"""Build a global accessible MMLogger.
Args:
log_level (int or str): The log level of MMLogger handlers.

9
mmdeploy/codebase/base/task.py
View File

@ -24,6 +24,9 @@ class BaseTask(metaclass=ABCMeta):
model_cfg (str | Config): Model config file.
deploy_cfg (str | Config): Deployment config file.
device (str): A string specifying device type.
experiment_name (str, optional): Name of current experiment.
If not specified, timestamp will be used as
``experiment_name``. Defaults to ``None``.
"""
def __init__(self,
@ -70,6 +73,12 @@ class BaseTask(metaclass=ABCMeta):
pass
def build_data_preprocessor(self):
"""build data preprocessor.
Returns:
BaseDataPreprocessor:
Initialized instance of :class:`BaseDataPreprocessor`.
"""
model = deepcopy(self.model_cfg.model)
preprocess_cfg = model['data_preprocessor']

17
mmdeploy/codebase/mmseg/deploy/segmentation.py
View File

@ -65,9 +65,9 @@ def _get_dataset_metainfo(model_cfg: Config):
"""Get metainfo of dataset.
Args:
model_cfg Config: Input model Config object.
model_cfg (Config): Input model Config object.
Returns:
(list[str], list[np.ndarray]): Class names and palette
(list[str], list[np.ndarray]): Class names and palette.
"""
from mmseg import datasets # noqa
from mmseg.registry import DATASETS
@ -106,10 +106,12 @@ class MMSegmentation(MMCodebase):
@classmethod
def register_deploy_modules(cls):
"""register deploy modules."""
import mmdeploy.codebase.mmseg.models # noqa: F401
@classmethod
def register_all_modules(cls):
"""register all modules."""
from mmseg.utils.set_env import register_all_modules
cls.register_deploy_modules()
@ -167,7 +169,8 @@ class Segmentation(BaseTask):
`np.ndarray`, `torch.Tensor`.
input_shape (list[int]): A list of two integer in (width, height)
format specifying input shape. Defaults to `None`.
data_preprocessor (BaseDataPreprocessor | None): Input data pre-
processor. Default is ``None``.
Returns:
tuple: (data, img), meta information for the input image and input.
"""
@ -200,11 +203,11 @@ class Segmentation(BaseTask):
"""
Args:
name:
save_dir:
name (str): Name of visualizer.
save_dir (str): Directory to save drawn results.
Returns:
SegLocalVisualizer: Instance of mmseg visualizer.
"""
# import to make SegLocalVisualizer could be built
from mmseg.visualization import SegLocalVisualizer # noqa: F401,F403
@ -236,7 +239,7 @@ class Segmentation(BaseTask):
window_name (str): The name of visualization window. Defaults to
an empty string.
show_result (bool): Whether to show result in windows, defaults
to `False`.
to ``False``.
opacity: (float): Opacity of painted segmentation map.
Defaults to `0.5`.
"""

40
mmdeploy/codebase/mmseg/deploy/segmentation_model.py
View File

@ -29,10 +29,10 @@ class End2EndModel(BaseBackendModel):
backend_files (Sequence[str]): Paths to all required backend files(e.g.
'.onnx' for ONNX Runtime, '.param' and '.bin' for ncnn).
device (str): A string represents device type.
class_names (Sequence[str]): A list of string specifying class names.
palette (np.ndarray): The palette of segmentation map.
deploy_cfg (str | mmengine.Config): Deployment config file or loaded
Config object.
data_preprocessor (dict | nn.Module | None): Input data pre-
processor. Default is ``None``.
"""
def __init__(self,
@ -73,8 +73,9 @@ class End2EndModel(BaseBackendModel):
Args:
inputs (torch.Tensor): Input image tensor
in [N x C x H x W] format.
data_samples (List[BaseDataElement]): A list of meta info for
image(s).
data_samples (list[:obj:`SegDataSample`]): The seg data
samples. It usually includes information such as
`metainfo` and `gt_sem_seg`. Default to None.
mode (str): forward mode, only support 'predict'.
**kwargs: Other key-pair arguments.
@ -93,6 +94,18 @@ class End2EndModel(BaseBackendModel):
def pack_result(self, batch_outputs: torch.Tensor,
data_samples: List[BaseDataElement]):
"""Pack segmentation result to data samples.
Args:
batch_outputs (Tensor): Batched segmentation output
tensor.
data_samples (list[:obj:`SegDataSample`]): The seg data
samples. It usually includes information such as
`metainfo` and `gt_sem_seg`. Default to None.
Returns:
list[:obj:`SegDataSample`]: The updated seg data samples.
"""
predictions = []
for seg_pred, data_sample in zip(batch_outputs, data_samples):
# resize seg_pred to original image shape
@ -123,9 +136,9 @@ class RKNNModel(End2EndModel):
Args:
inputs (Tensor): Inputs with shape (N, C, H, W).
data_samples (List[:obj:`DetDataSample`]): The Data
Samples. It usually includes information such as
`gt_instance`, `gt_panoptic_seg` and `gt_sem_seg`.
data_samples (list[:obj:`SegDataSample`]): The seg data
samples. It usually includes information such as
`metainfo` and `gt_sem_seg`. Default to None.
Returns:
list: A list contains predictions.
@ -156,12 +169,13 @@ class SDKEnd2EndModel(End2EndModel):
"""Run forward inference.
Args:
img (Sequence[torch.Tensor]): A list contains input image(s)
in [N x C x H x W] format.
img_metas (Sequence[Sequence[dict]]): A list of meta info for
image(s).
*args: Other arguments.
**kwargs: Other key-pair arguments.
inputs (Sequence[torch.Tensor]): A list contains input
image(s) in [C x H x W] format.
data_samples (list[:obj:`SegDataSample`]): The seg data
samples. It usually includes information such as
`metainfo` and `gt_sem_seg`. Default to None.
mode (str): Return what kind of value. Defaults to
'predict'.
Returns:
list: A list contains predictions.