mirrors/mmdeploy
1
0
Fork 0
mirror of https://github.com/open-mmlab/mmdeploy.git synced 2025-01-14 08:09:43 +08:00
Code Issues Projects Releases Wiki Activity
mmdeploy/tests/regression/mmocr.yml

206 lines
8.7 KiB
YAML
Raw Normal View History

[Feature] Regression test for mmdeploy (#302) * Add regression test script * Add doc * Add test yaml for mmdet * evaluate_outputs Add return result (cherry picked from commit c8c9cd75df7916aa4d80a7b8bfb8a78e04446cad) * object_detection return metric after eval (cherry picked from commit 1b8dcaa39ed14f016bf51d1aee65c2c416cd7c33) * move `deploy_config_dir` to `global_info` in test yaml * fix path error * Improve test yaml structure * Add test env for saving regression report * Fix SDK test report will crash * Get SDK FPS * Add mmcls regression test yaml * Using CMD to test the backend result * Get metric from log file * Imporve coding * Imporve coding * restructure test yaml * resturcture the test yaml and coding, using pipeline style * Fixed wont saving into report when cant find `backend_test` and `sdk_config` * set `metric_info` in test yaml * improve test yaml * Fixed will get black checkpoint file name * Fix lint * Fix yaml * Add common in test yaml * Resturcture mmcla test yaml * Resturcture mmcla test yaml * Improve mmcls test yaml * mmcls test success * Improve tes yaml field * Add `--test-img` only when `test_img_path` is not None * Add `precision_type` in report * Not saving pkl result file any more * Add 'x' install of '-' when script crash * Fix some field in mmcls test yaml * Add mmseg test yaml * Add unknown backend final file name * Improve backend file dict * Add mmseg success * unify the checkpoint path to relate path * unify the checkpoint path to relate path * Add mmpose, need to test * Support backend file list to `--model` when test the backend * Fix lint * Add some common * FPS get from log always get 1:10 line * Add dataset in report when test backend * Get dataset type from model config file * Replace pipeline.json topk * SDK report add backend name * Add txt report, it will save each test * update mmcls config * Add `calib-dataset-cfg` in cmd when it exist in tset yaml * make model path shorter by cutting the work_dir_root * Add `task_name` in test yaml * Add `task_name` in report * Improve test yaml * Add mmocr test yaml * Get mmocr fps metric success * Add `dataset` feild in test yaml * Report will skip when the dataset name not in test yaml * Add dbnet in mmocr test yaml and success get metric * Add mmedit test yaml * Improve some common * Add mmedit success * Fix lint * Fix isort lint * Fix yapf lint * Undo some changes in `evaluate_outputs` * Undo some changes in `evaluate_outputs` * Improve test requirement.txt * Undo some changes in `evaluate_outputs` * Improve doc * Improve mmedit test yaml * Using `--divice` * Fix lint * Using `--performance` replace `--test-type` * Fix lint * Fix page link * Fix backend name * Using `logger` instead of `print` * Fix lint * Add TorchScript in the doc * Add type hint for all the funcs * Fix docformatter lint * Fix path in report have the root of work dir * mmdet add other backend in tast yaml * mmdet add other backend in tast yaml * mmocr add other backend in tast yaml * mmedit add other backend in tast yaml * mmpose add other backend in tast yaml * Delete filed `codebase_model_config_dir` in test yaml * Using `Config` in metafile.yml instal of `Name` from allmodel config files. * Fix yapf lint * update mmpose mmseg config * Fix lint * Imporve mmcls test yaml * Imporve mmedit test yaml * Imporve mmedit test yaml * Imporve mmseg test yaml * update mmdet yml * Not using pth when conver sucess when in only convert mode * Using metafile dataset when can not get `model_cfg.dataset_type` * Fixed `model_name` incorrect in some codebase * Improve mmcls test yaml image * Improve mmedit test yaml image * Improve mmocr test yaml image * Improve mmseg test yaml image * Fix test yaml bug * Support overwirte `metric_tolerance` * Add `metric_tolerance` in mmcls * Fixed yaml bug * mmcls add all models, which had already supported, in test yaml * Fix report will not replace to ${WORK_DIR} * Add metric tolerance in mmcls test yaml * Modefied mmcls global metric tolreance * remove `metric_tolerance` in each pipeline * Improve mmcls test yaml * mmcls add TODO * imporve test yaml `pp`l -> `pplnn` * mmdet add all models, which had already supported, in test yaml * mmedit add all models, which had already supported, in test yaml * mmocr add all models, which had already supported, in test yaml * mmpose add all models, which had already supported, in test yaml * Replace `sdk_xxx_fp32` to `sdk_xxx` * mmseg add all models, which had already supported, in test yaml * Import the backend order of test yaml * Fix mmseg test yaml pplnn static config * Add notes for Windows user in the doc * Fix get metric from different metric name but correct dataset name. * Fix mmedit dataset incorrect * Fix test yaml of mmedit * Fix lint * Fix mmpose can't find metric * Improve mmseg trt testing config * Add dataset in mmdet test yaml * Add logs when continue in the code * Fix mmpose get metric log error * Fix mmdet can not get metric * Add merge report into one * Fix lint * Add version in the report * log2file for each backend * Fix lint * Improve report merge * update mmseg yml to shape 1024x2048 * Fix dependences for merging report * Imporve mmcls yaml, add `ShuffleNetV1` and `ShuffleNetV2` * Fix name of model in test yaml with a space will crash when convert * Add commons for test yaml metric tolerance * Add mmdet seg detail config in test yaml * Improve mmdet test yaml * Fix mmdet mskrcnn metric Co-authored-by: maningsheng <mnsheng@yeah.net>
2022-04-28 08:03:59 +08:00
globals:
codebase_dir: ../mmocr
checkpoint_force_download: False
images:
img_densetext_det: &img_densetext_det ../mmocr/demo/demo_densetext_det.jpg
img_demo_text_det: &img_demo_text_det ../mmocr/demo/demo_text_det.jpg
img_demo_text_ocr: &img_demo_text_ocr ../mmocr/demo/demo_text_ocr.jpg
img_demo_text_recog: &img_demo_text_recog ../mmocr/demo/demo_text_recog.jpg
metric_info: &metric_info
hmean-iou: # named after metafile.Results.Metrics
eval_name: hmean-iou # test.py --metrics args
metric_key: 0_hmean-iou:hmean # eval key name
tolerance: 0.15 # metric ±n%
task_name: Text Detection # metafile.Results.Task
dataset: ICDAR2015 # metafile.Results.Dataset
word_acc:
eval_name: acc
metric_key: 0_word_acc_ignore_case
tolerance: 0.05 # metric ±n%
task_name: Text Recognition
dataset: IIIT5K
convert_image_det: &convert_image_det
input_img: *img_densetext_det
test_img: *img_demo_text_det
convert_image_rec: &convert_image_rec
input_img: *img_demo_text_recog
test_img: *img_demo_text_recog
backend_test: &default_backend_test True
sdk:
sdk_detection_dynamic: &sdk_detection_dynamic configs/mmocr/text-detection/text-detection_sdk_dynamic.py
sdk_recognition_dynamic: &sdk_recognition_dynamic configs/mmocr/text-recognition/text-recognition_sdk_dynamic.py
onnxruntime:
# ======= detection =======
pipeline_ort_detection_static_fp32: &pipeline_ort_detection_static_fp32
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
deploy_config: configs/mmocr/text-detection/text-detection_onnxruntime_static.py
pipeline_ort_detection_dynamic_fp32: &pipeline_ort_detection_dynamic_fp32
convert_image: *convert_image_det
deploy_config: configs/mmocr/text-detection/text-detection_onnxruntime_dynamic.py
# ======= recognition =======
pipeline_ort_recognition_static_fp32: &pipeline_ort_recognition_static_fp32
convert_image: *convert_image_rec
deploy_config: configs/mmocr/text-recognition/text-recognition_onnxruntime_static.py
pipeline_ort_recognition_dynamic_fp32: &pipeline_ort_recognition_dynamic_fp32
convert_image: *convert_image_rec
deploy_config: configs/mmocr/text-recognition/text-recognition_onnxruntime_dynamic.py
tensorrt:
# ======= detection =======
pipeline_trt_detection_static_fp32: &pipeline_trt_detection_static_fp32
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt_static-512x512.py
pipeline_trt_detection_static_fp16: &pipeline_trt_detection_static_fp16
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt-fp16_static-512x512.py
pipeline_trt_detection_static_int8: &pipeline_trt_detection_static_int8
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt-int8_static-512x512.py
pipeline_trt_detection_dynamic_fp32: &pipeline_trt_detection_dynamic_fp32
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
change resolution and fix regression test for mmocr (#534)
2022-06-02 18:29:41 +08:00
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt_dynamic-320x320-2240x2240.py
[Feature] Regression test for mmdeploy (#302) * Add regression test script * Add doc * Add test yaml for mmdet * evaluate_outputs Add return result (cherry picked from commit c8c9cd75df7916aa4d80a7b8bfb8a78e04446cad) * object_detection return metric after eval (cherry picked from commit 1b8dcaa39ed14f016bf51d1aee65c2c416cd7c33) * move `deploy_config_dir` to `global_info` in test yaml * fix path error * Improve test yaml structure * Add test env for saving regression report * Fix SDK test report will crash * Get SDK FPS * Add mmcls regression test yaml * Using CMD to test the backend result * Get metric from log file * Imporve coding * Imporve coding * restructure test yaml * resturcture the test yaml and coding, using pipeline style * Fixed wont saving into report when cant find `backend_test` and `sdk_config` * set `metric_info` in test yaml * improve test yaml * Fixed will get black checkpoint file name * Fix lint * Fix yaml * Add common in test yaml * Resturcture mmcla test yaml * Resturcture mmcla test yaml * Improve mmcls test yaml * mmcls test success * Improve tes yaml field * Add `--test-img` only when `test_img_path` is not None * Add `precision_type` in report * Not saving pkl result file any more * Add 'x' install of '-' when script crash * Fix some field in mmcls test yaml * Add mmseg test yaml * Add unknown backend final file name * Improve backend file dict * Add mmseg success * unify the checkpoint path to relate path * unify the checkpoint path to relate path * Add mmpose, need to test * Support backend file list to `--model` when test the backend * Fix lint * Add some common * FPS get from log always get 1:10 line * Add dataset in report when test backend * Get dataset type from model config file * Replace pipeline.json topk * SDK report add backend name * Add txt report, it will save each test * update mmcls config * Add `calib-dataset-cfg` in cmd when it exist in tset yaml * make model path shorter by cutting the work_dir_root * Add `task_name` in test yaml * Add `task_name` in report * Improve test yaml * Add mmocr test yaml * Get mmocr fps metric success * Add `dataset` feild in test yaml * Report will skip when the dataset name not in test yaml * Add dbnet in mmocr test yaml and success get metric * Add mmedit test yaml * Improve some common * Add mmedit success * Fix lint * Fix isort lint * Fix yapf lint * Undo some changes in `evaluate_outputs` * Undo some changes in `evaluate_outputs` * Improve test requirement.txt * Undo some changes in `evaluate_outputs` * Improve doc * Improve mmedit test yaml * Using `--divice` * Fix lint * Using `--performance` replace `--test-type` * Fix lint * Fix page link * Fix backend name * Using `logger` instead of `print` * Fix lint * Add TorchScript in the doc * Add type hint for all the funcs * Fix docformatter lint * Fix path in report have the root of work dir * mmdet add other backend in tast yaml * mmdet add other backend in tast yaml * mmocr add other backend in tast yaml * mmedit add other backend in tast yaml * mmpose add other backend in tast yaml * Delete filed `codebase_model_config_dir` in test yaml * Using `Config` in metafile.yml instal of `Name` from allmodel config files. * Fix yapf lint * update mmpose mmseg config * Fix lint * Imporve mmcls test yaml * Imporve mmedit test yaml * Imporve mmedit test yaml * Imporve mmseg test yaml * update mmdet yml * Not using pth when conver sucess when in only convert mode * Using metafile dataset when can not get `model_cfg.dataset_type` * Fixed `model_name` incorrect in some codebase * Improve mmcls test yaml image * Improve mmedit test yaml image * Improve mmocr test yaml image * Improve mmseg test yaml image * Fix test yaml bug * Support overwirte `metric_tolerance` * Add `metric_tolerance` in mmcls * Fixed yaml bug * mmcls add all models, which had already supported, in test yaml * Fix report will not replace to ${WORK_DIR} * Add metric tolerance in mmcls test yaml * Modefied mmcls global metric tolreance * remove `metric_tolerance` in each pipeline * Improve mmcls test yaml * mmcls add TODO * imporve test yaml `pp`l -> `pplnn` * mmdet add all models, which had already supported, in test yaml * mmedit add all models, which had already supported, in test yaml * mmocr add all models, which had already supported, in test yaml * mmpose add all models, which had already supported, in test yaml * Replace `sdk_xxx_fp32` to `sdk_xxx` * mmseg add all models, which had already supported, in test yaml * Import the backend order of test yaml * Fix mmseg test yaml pplnn static config * Add notes for Windows user in the doc * Fix get metric from different metric name but correct dataset name. * Fix mmedit dataset incorrect * Fix test yaml of mmedit * Fix lint * Fix mmpose can't find metric * Improve mmseg trt testing config * Add dataset in mmdet test yaml * Add logs when continue in the code * Fix mmpose get metric log error * Fix mmdet can not get metric * Add merge report into one * Fix lint * Add version in the report * log2file for each backend * Fix lint * Improve report merge * update mmseg yml to shape 1024x2048 * Fix dependences for merging report * Imporve mmcls yaml, add `ShuffleNetV1` and `ShuffleNetV2` * Fix name of model in test yaml with a space will crash when convert * Add commons for test yaml metric tolerance * Add mmdet seg detail config in test yaml * Improve mmdet test yaml * Fix mmdet mskrcnn metric Co-authored-by: maningsheng <mnsheng@yeah.net>
2022-04-28 08:03:59 +08:00
pipeline_trt_detection_dynamic_fp16: &pipeline_trt_detection_dynamic_fp16
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
change resolution and fix regression test for mmocr (#534)
2022-06-02 18:29:41 +08:00
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt-fp16_dynamic-320x320-2240x2240.py
[Feature] Regression test for mmdeploy (#302) * Add regression test script * Add doc * Add test yaml for mmdet * evaluate_outputs Add return result (cherry picked from commit c8c9cd75df7916aa4d80a7b8bfb8a78e04446cad) * object_detection return metric after eval (cherry picked from commit 1b8dcaa39ed14f016bf51d1aee65c2c416cd7c33) * move `deploy_config_dir` to `global_info` in test yaml * fix path error * Improve test yaml structure * Add test env for saving regression report * Fix SDK test report will crash * Get SDK FPS * Add mmcls regression test yaml * Using CMD to test the backend result * Get metric from log file * Imporve coding * Imporve coding * restructure test yaml * resturcture the test yaml and coding, using pipeline style * Fixed wont saving into report when cant find `backend_test` and `sdk_config` * set `metric_info` in test yaml * improve test yaml * Fixed will get black checkpoint file name * Fix lint * Fix yaml * Add common in test yaml * Resturcture mmcla test yaml * Resturcture mmcla test yaml * Improve mmcls test yaml * mmcls test success * Improve tes yaml field * Add `--test-img` only when `test_img_path` is not None * Add `precision_type` in report * Not saving pkl result file any more * Add 'x' install of '-' when script crash * Fix some field in mmcls test yaml * Add mmseg test yaml * Add unknown backend final file name * Improve backend file dict * Add mmseg success * unify the checkpoint path to relate path * unify the checkpoint path to relate path * Add mmpose, need to test * Support backend file list to `--model` when test the backend * Fix lint * Add some common * FPS get from log always get 1:10 line * Add dataset in report when test backend * Get dataset type from model config file * Replace pipeline.json topk * SDK report add backend name * Add txt report, it will save each test * update mmcls config * Add `calib-dataset-cfg` in cmd when it exist in tset yaml * make model path shorter by cutting the work_dir_root * Add `task_name` in test yaml * Add `task_name` in report * Improve test yaml * Add mmocr test yaml * Get mmocr fps metric success * Add `dataset` feild in test yaml * Report will skip when the dataset name not in test yaml * Add dbnet in mmocr test yaml and success get metric * Add mmedit test yaml * Improve some common * Add mmedit success * Fix lint * Fix isort lint * Fix yapf lint * Undo some changes in `evaluate_outputs` * Undo some changes in `evaluate_outputs` * Improve test requirement.txt * Undo some changes in `evaluate_outputs` * Improve doc * Improve mmedit test yaml * Using `--divice` * Fix lint * Using `--performance` replace `--test-type` * Fix lint * Fix page link * Fix backend name * Using `logger` instead of `print` * Fix lint * Add TorchScript in the doc * Add type hint for all the funcs * Fix docformatter lint * Fix path in report have the root of work dir * mmdet add other backend in tast yaml * mmdet add other backend in tast yaml * mmocr add other backend in tast yaml * mmedit add other backend in tast yaml * mmpose add other backend in tast yaml * Delete filed `codebase_model_config_dir` in test yaml * Using `Config` in metafile.yml instal of `Name` from allmodel config files. * Fix yapf lint * update mmpose mmseg config * Fix lint * Imporve mmcls test yaml * Imporve mmedit test yaml * Imporve mmedit test yaml * Imporve mmseg test yaml * update mmdet yml * Not using pth when conver sucess when in only convert mode * Using metafile dataset when can not get `model_cfg.dataset_type` * Fixed `model_name` incorrect in some codebase * Improve mmcls test yaml image * Improve mmedit test yaml image * Improve mmocr test yaml image * Improve mmseg test yaml image * Fix test yaml bug * Support overwirte `metric_tolerance` * Add `metric_tolerance` in mmcls * Fixed yaml bug * mmcls add all models, which had already supported, in test yaml * Fix report will not replace to ${WORK_DIR} * Add metric tolerance in mmcls test yaml * Modefied mmcls global metric tolreance * remove `metric_tolerance` in each pipeline * Improve mmcls test yaml * mmcls add TODO * imporve test yaml `pp`l -> `pplnn` * mmdet add all models, which had already supported, in test yaml * mmedit add all models, which had already supported, in test yaml * mmocr add all models, which had already supported, in test yaml * mmpose add all models, which had already supported, in test yaml * Replace `sdk_xxx_fp32` to `sdk_xxx` * mmseg add all models, which had already supported, in test yaml * Import the backend order of test yaml * Fix mmseg test yaml pplnn static config * Add notes for Windows user in the doc * Fix get metric from different metric name but correct dataset name. * Fix mmedit dataset incorrect * Fix test yaml of mmedit * Fix lint * Fix mmpose can't find metric * Improve mmseg trt testing config * Add dataset in mmdet test yaml * Add logs when continue in the code * Fix mmpose get metric log error * Fix mmdet can not get metric * Add merge report into one * Fix lint * Add version in the report * log2file for each backend * Fix lint * Improve report merge * update mmseg yml to shape 1024x2048 * Fix dependences for merging report * Imporve mmcls yaml, add `ShuffleNetV1` and `ShuffleNetV2` * Fix name of model in test yaml with a space will crash when convert * Add commons for test yaml metric tolerance * Add mmdet seg detail config in test yaml * Improve mmdet test yaml * Fix mmdet mskrcnn metric Co-authored-by: maningsheng <mnsheng@yeah.net>
2022-04-28 08:03:59 +08:00
pipeline_trt_detection_dynamic_int8: &pipeline_trt_detection_dynamic_int8
convert_image: *convert_image_det
backend_test: *default_backend_test
sdk_config: *sdk_detection_dynamic
change resolution and fix regression test for mmocr (#534)
2022-06-02 18:29:41 +08:00
deploy_config: configs/mmocr/text-detection/text-detection_tensorrt-int8_dynamic-320x320-2240x2240.py
[Feature] Regression test for mmdeploy (#302) * Add regression test script * Add doc * Add test yaml for mmdet * evaluate_outputs Add return result (cherry picked from commit c8c9cd75df7916aa4d80a7b8bfb8a78e04446cad) * object_detection return metric after eval (cherry picked from commit 1b8dcaa39ed14f016bf51d1aee65c2c416cd7c33) * move `deploy_config_dir` to `global_info` in test yaml * fix path error * Improve test yaml structure * Add test env for saving regression report * Fix SDK test report will crash * Get SDK FPS * Add mmcls regression test yaml * Using CMD to test the backend result * Get metric from log file * Imporve coding * Imporve coding * restructure test yaml * resturcture the test yaml and coding, using pipeline style * Fixed wont saving into report when cant find `backend_test` and `sdk_config` * set `metric_info` in test yaml * improve test yaml * Fixed will get black checkpoint file name * Fix lint * Fix yaml * Add common in test yaml * Resturcture mmcla test yaml * Resturcture mmcla test yaml * Improve mmcls test yaml * mmcls test success * Improve tes yaml field * Add `--test-img` only when `test_img_path` is not None * Add `precision_type` in report * Not saving pkl result file any more * Add 'x' install of '-' when script crash * Fix some field in mmcls test yaml * Add mmseg test yaml * Add unknown backend final file name * Improve backend file dict * Add mmseg success * unify the checkpoint path to relate path * unify the checkpoint path to relate path * Add mmpose, need to test * Support backend file list to `--model` when test the backend * Fix lint * Add some common * FPS get from log always get 1:10 line * Add dataset in report when test backend * Get dataset type from model config file * Replace pipeline.json topk * SDK report add backend name * Add txt report, it will save each test * update mmcls config * Add `calib-dataset-cfg` in cmd when it exist in tset yaml * make model path shorter by cutting the work_dir_root * Add `task_name` in test yaml * Add `task_name` in report * Improve test yaml * Add mmocr test yaml * Get mmocr fps metric success * Add `dataset` feild in test yaml * Report will skip when the dataset name not in test yaml * Add dbnet in mmocr test yaml and success get metric * Add mmedit test yaml * Improve some common * Add mmedit success * Fix lint * Fix isort lint * Fix yapf lint * Undo some changes in `evaluate_outputs` * Undo some changes in `evaluate_outputs` * Improve test requirement.txt * Undo some changes in `evaluate_outputs` * Improve doc * Improve mmedit test yaml * Using `--divice` * Fix lint * Using `--performance` replace `--test-type` * Fix lint * Fix page link * Fix backend name * Using `logger` instead of `print` * Fix lint * Add TorchScript in the doc * Add type hint for all the funcs * Fix docformatter lint * Fix path in report have the root of work dir * mmdet add other backend in tast yaml * mmdet add other backend in tast yaml * mmocr add other backend in tast yaml * mmedit add other backend in tast yaml * mmpose add other backend in tast yaml * Delete filed `codebase_model_config_dir` in test yaml * Using `Config` in metafile.yml instal of `Name` from allmodel config files. * Fix yapf lint * update mmpose mmseg config * Fix lint * Imporve mmcls test yaml * Imporve mmedit test yaml * Imporve mmedit test yaml * Imporve mmseg test yaml * update mmdet yml * Not using pth when conver sucess when in only convert mode * Using metafile dataset when can not get `model_cfg.dataset_type` * Fixed `model_name` incorrect in some codebase * Improve mmcls test yaml image * Improve mmedit test yaml image * Improve mmocr test yaml image * Improve mmseg test yaml image * Fix test yaml bug * Support overwirte `metric_tolerance` * Add `metric_tolerance` in mmcls * Fixed yaml bug * mmcls add all models, which had already supported, in test yaml * Fix report will not replace to ${WORK_DIR} * Add metric tolerance in mmcls test yaml * Modefied mmcls global metric tolreance * remove `metric_tolerance` in each pipeline * Improve mmcls test yaml * mmcls add TODO * imporve test yaml `pp`l -> `pplnn` * mmdet add all models, which had already supported, in test yaml * mmedit add all models, which had already supported, in test yaml * mmocr add all models, which had already supported, in test yaml * mmpose add all models, which had already supported, in test yaml * Replace `sdk_xxx_fp32` to `sdk_xxx` * mmseg add all models, which had already supported, in test yaml * Import the backend order of test yaml * Fix mmseg test yaml pplnn static config * Add notes for Windows user in the doc * Fix get metric from different metric name but correct dataset name. * Fix mmedit dataset incorrect * Fix test yaml of mmedit * Fix lint * Fix mmpose can't find metric * Improve mmseg trt testing config * Add dataset in mmdet test yaml * Add logs when continue in the code * Fix mmpose get metric log error * Fix mmdet can not get metric * Add merge report into one * Fix lint * Add version in the report * log2file for each backend * Fix lint * Improve report merge * update mmseg yml to shape 1024x2048 * Fix dependences for merging report * Imporve mmcls yaml, add `ShuffleNetV1` and `ShuffleNetV2` * Fix name of model in test yaml with a space will crash when convert * Add commons for test yaml metric tolerance * Add mmdet seg detail config in test yaml * Improve mmdet test yaml * Fix mmdet mskrcnn metric Co-authored-by: maningsheng <mnsheng@yeah.net>
2022-04-28 08:03:59 +08:00
# ======= recognition =======
pipeline_trt_recognition_static_fp32: &pipeline_trt_recognition_static_fp32
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt_static-1x32x32.py
pipeline_trt_recognition_static_fp16: &pipeline_trt_recognition_static_fp16
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt-fp16_static-1x32x32.py
pipeline_trt_recognition_static_int8: &pipeline_trt_recognition_static_int8
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt-int8_static-1x32x32.py
pipeline_trt_recognition_dynamic_fp32: &pipeline_trt_recognition_dynamic_fp32
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt_dynamic-1x32x32-1x32x640.py
pipeline_trt_recognition_dynamic_fp16: &pipeline_trt_recognition_dynamic_fp16
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt-fp16_dynamic-1x32x32-1x32x640.py
pipeline_trt_recognition_dynamic_int8: &pipeline_trt_recognition_dynamic_int8
convert_image: *convert_image_rec
backend_test: *default_backend_test
sdk_config: *sdk_recognition_dynamic
deploy_config: configs/mmocr/text-recognition/text-recognition_tensorrt-int8_dynamic-1x32x32-1x32x640.py
openvino:
pipeline_openvino_detection_dynamic_fp32: &pipeline_openvino_detection_dynamic_fp32
convert_image: *convert_image_det
backend_test: *default_backend_test
deploy_config: configs/mmocr/text-detection/text-detection_openvino_dynamic-640x640.py
# pipeline_openvino_recognition_dynamic_fp32: &pipeline_openvino_recognition_dynamic_fp32
# convert_image: *convert_image_rec
# backend_test: *default_backend_test
# deploy_config:
ncnn:
pipeline_ncnn_detection_static_fp32: &pipeline_ncnn_detection_static_fp32
convert_image: *convert_image_det
backend_test: False
deploy_config: configs/mmocr/text-detection/text-detection_ncnn_static.py
pipeline_ncnn_recognition_static_fp32: &pipeline_ncnn_recognition_static_fp32
convert_image: *convert_image_rec
backend_test: False
deploy_config: configs/mmocr/text-recognition/text-recognition_ncnn_static.py
pplnn:
pipeline_pplnn_detection_dynamic_fp32: &pipeline_pplnn_detection_dynamic_fp32
convert_image: *convert_image_det
backend_test: False
deploy_config: configs/mmocr/text-detection/text-detection_pplnn_dynamic-640x640.py
pipeline_pplnn_recognition_dynamic_fp32: &pipeline_pplnn_recognition_dynamic_fp32
convert_image: *convert_image_rec
backend_test: False
deploy_config: configs/mmocr/text-recognition/text-recognition_pplnn_dynamic-1x32x32.py
torchscript:
pipeline_ts_detection_fp32: &pipeline_ts_detection_fp32
convert_image: *convert_image_det
backend_test: False
deploy_config: configs/mmocr/text-detection/text-detection_torchscript.py
pipeline_ts_recognition_fp32: &pipeline_ts_recognition_fp32
convert_image: *convert_image_rec
backend_test: False
deploy_config: configs/mmocr/text-recognition/text-recognition_torchscript.py
models:
- name: DBNet
metafile: configs/textdet/dbnet/metafile.yml
model_configs:
- configs/textdet/dbnet/dbnet_r18_fpnc_1200e_icdar2015.py
pipelines:
- *pipeline_ts_detection_fp32
- *pipeline_ort_detection_dynamic_fp32
# - *pipeline_trt_detection_dynamic_fp32
- *pipeline_trt_detection_dynamic_fp16
# - *pipeline_trt_detection_dynamic_int8
- *pipeline_ncnn_detection_static_fp32
- *pipeline_pplnn_detection_dynamic_fp32
- *pipeline_openvino_detection_dynamic_fp32
- name: CRNN
metafile: configs/textrecog/crnn/metafile.yml
model_configs:
- configs/textrecog/crnn/crnn_academic_dataset.py
pipelines:
- *pipeline_ts_recognition_fp32
- *pipeline_ort_recognition_dynamic_fp32
- *pipeline_trt_recognition_dynamic_fp16
- *pipeline_ncnn_recognition_static_fp32
- *pipeline_pplnn_recognition_dynamic_fp32
- name: SAR
metafile: configs/textrecog/sar/metafile.yml
model_configs:
- configs/textrecog/sar/sar_r31_parallel_decoder_academic.py
pipelines:
- *pipeline_ort_recognition_dynamic_fp32
Reference in New Issue Copy Permalink