* make regression test as a module under project
* using `--codebase` instead of `--deploy-yml`
* Improve doc for `--codebase`
* Add shorter arg `-p` for `--performance`
* make `checopoint-dir` as an arg for the script
* Gen error log when convert fail.
* Improve res code for testing
* Doc add sdk test detail
* Doc add env setup
* Fix lint
* Fix doc lint
* Improve model path in report
* Improve report title
* Improve report checkpoint path
* Fix lint
* move test yaml under `tests/regression`
* Improve the test yaml path
* Fix lint
* Improve doc
* make func `update_report` code better
* move doc to new location
* Fix arg
* Update arg details
* Use cpu when openvino and onnxruntime cpu package
* Fix word
* Fix func of openpyxl 3.0.9
* Add some info
* Fix lint
* Fix filename
* Fix doc link
* Fix dir name with space when is not sdk
* Add args `--models` for test specific model(s)
* not saving report when no model in codebase when using `--models`
* Fix doc
* Fix lint
* Add table for metric in doc
* Improve table for doc
* Using `None` install of `['all']`
* Improce doc
* set device type properly
* Increate popen bufsize
* Add `precision_type` in `work-dir`
* Fix popen stuck
* Fix lint
* Fix lint
* Fix popen stuck by using file handler
* Make metric dataset as a list
* Update mmseg.yml
* Remove 'FPS' in the report
* Update do_regression_test.md
* Improve log
* Fix codespell
* Fix doc
* ncnn only save `xxx.param` as checkpoint name in the report
Co-authored-by: maningsheng <mnsheng@yeah.net>
* Add regression test script
* Add doc
* Add test yaml for mmdet
* evaluate_outputs Add return result
(cherry picked from commit c8c9cd75df)
* object_detection return metric after eval
(cherry picked from commit 1b8dcaa39e)
* move `deploy_config_dir` to `global_info` in test yaml
* fix path error
* Improve test yaml structure
* Add test env for saving regression report
* Fix SDK test report will crash
* Get SDK FPS
* Add mmcls regression test yaml
* Using CMD to test the backend result
* Get metric from log file
* Imporve coding
* Imporve coding
* restructure test yaml
* resturcture the test yaml and coding, using pipeline style
* Fixed wont saving into report when cant find `backend_test` and `sdk_config`
* set `metric_info` in test yaml
* improve test yaml
* Fixed will get black checkpoint file name
* Fix lint
* Fix yaml
* Add common in test yaml
* Resturcture mmcla test yaml
* Resturcture mmcla test yaml
* Improve mmcls test yaml
* mmcls test success
* Improve tes yaml field
* Add `--test-img` only when `test_img_path` is not None
* Add `precision_type` in report
* Not saving pkl result file any more
* Add 'x' install of '-' when script crash
* Fix some field in mmcls test yaml
* Add mmseg test yaml
* Add unknown backend final file name
* Improve backend file dict
* Add mmseg success
* unify the checkpoint path to relate path
* unify the checkpoint path to relate path
* Add mmpose, need to test
* Support backend file list to `--model` when test the backend
* Fix lint
* Add some common
* FPS get from log always get 1:10 line
* Add dataset in report when test backend
* Get dataset type from model config file
* Replace pipeline.json topk
* SDK report add backend name
* Add txt report, it will save each test
* update mmcls config
* Add `calib-dataset-cfg` in cmd when it exist in tset yaml
* make model path shorter by cutting the work_dir_root
* Add `task_name` in test yaml
* Add `task_name` in report
* Improve test yaml
* Add mmocr test yaml
* Get mmocr fps metric success
* Add `dataset` feild in test yaml
* Report will skip when the dataset name not in test yaml
* Add dbnet in mmocr test yaml and success get metric
* Add mmedit test yaml
* Improve some common
* Add mmedit success
* Fix lint
* Fix isort lint
* Fix yapf lint
* Undo some changes in `evaluate_outputs`
* Undo some changes in `evaluate_outputs`
* Improve test requirement.txt
* Undo some changes in `evaluate_outputs`
* Improve doc
* Improve mmedit test yaml
* Using `--divice`
* Fix lint
* Using `--performance` replace `--test-type`
* Fix lint
* Fix page link
* Fix backend name
* Using `logger` instead of `print`
* Fix lint
* Add TorchScript in the doc
* Add type hint for all the funcs
* Fix docformatter lint
* Fix path in report have the root of work dir
* mmdet add other backend in tast yaml
* mmdet add other backend in tast yaml
* mmocr add other backend in tast yaml
* mmedit add other backend in tast yaml
* mmpose add other backend in tast yaml
* Delete filed `codebase_model_config_dir` in test yaml
* Using `Config` in metafile.yml instal of `Name` from allmodel config files.
* Fix yapf lint
* update mmpose mmseg config
* Fix lint
* Imporve mmcls test yaml
* Imporve mmedit test yaml
* Imporve mmedit test yaml
* Imporve mmseg test yaml
* update mmdet yml
* Not using pth when conver sucess when in only convert mode
* Using metafile dataset when can not get `model_cfg.dataset_type`
* Fixed `model_name` incorrect in some codebase
* Improve mmcls test yaml image
* Improve mmedit test yaml image
* Improve mmocr test yaml image
* Improve mmseg test yaml image
* Fix test yaml bug
* Support overwirte `metric_tolerance`
* Add `metric_tolerance` in mmcls
* Fixed yaml bug
* mmcls add all models, which had already supported, in test yaml
* Fix report will not replace to ${WORK_DIR}
* Add metric tolerance in mmcls test yaml
* Modefied mmcls global metric tolreance
* remove `metric_tolerance` in each pipeline
* Improve mmcls test yaml
* mmcls add TODO
* imporve test yaml `pp`l -> `pplnn`
* mmdet add all models, which had already supported, in test yaml
* mmedit add all models, which had already supported, in test yaml
* mmocr add all models, which had already supported, in test yaml
* mmpose add all models, which had already supported, in test yaml
* Replace `sdk_xxx_fp32` to `sdk_xxx`
* mmseg add all models, which had already supported, in test yaml
* Import the backend order of test yaml
* Fix mmseg test yaml pplnn static config
* Add notes for Windows user in the doc
* Fix get metric from different metric name but correct dataset name.
* Fix mmedit dataset incorrect
* Fix test yaml of mmedit
* Fix lint
* Fix mmpose can't find metric
* Improve mmseg trt testing config
* Add dataset in mmdet test yaml
* Add logs when continue in the code
* Fix mmpose get metric log error
* Fix mmdet can not get metric
* Add merge report into one
* Fix lint
* Add version in the report
* log2file for each backend
* Fix lint
* Improve report merge
* update mmseg yml to shape 1024x2048
* Fix dependences for merging report
* Imporve mmcls yaml, add `ShuffleNetV1` and `ShuffleNetV2`
* Fix name of model in test yaml with a space will crash when convert
* Add commons for test yaml metric tolerance
* Add mmdet seg detail config in test yaml
* Improve mmdet test yaml
* Fix mmdet mskrcnn metric
Co-authored-by: maningsheng <mnsheng@yeah.net>
* align mmedit static cfg
* add for test
* update requirments
* add dependencies from mmlab
* change name
* lower thresh for interrogate at first
* update test
* update to skip
* Move import tensorrt
* Move import statement
Co-authored-by: SingleZombie <singlezombie@163.com>
Chen Xin
HinGwenWoong
RunningLeon
q.yao