mirror of https://github.com/open-mmlab/mmdeploy.git synced 2025-01-14 08:09:43 +08:00

* [Refactor] add enum class and use functions to get configuration (#40)

* add task and codebase enum class

* use funcitons to get config

* Refactor wrappers of mmcls and mmseg (#41)

* move wrappers of cls & det to apis

* remove get_classes_from_config

* rename onnx_helper to onnx_utils

* move import to outside of class

* refactor ortwrappers

* Refactor build dataset and dataloader for mmseg (#44)

* refactor build_dataset and build_dataloader for mmcls and mmseg

* remove repeated classes

* set build_dataloader with shuffle=False

* [Refactor] pplwrapper and mmocr refactor (#46)

* add

* add pplwrapper and refactor mmocr

* fix lint

* remove unused arguments

* apply dict input for pplwrapper and ortwrapper

* add condition before import ppl and ort stuff

* update ppl (#51)

* Refactor return value and extract_model (#54)

* remove ret_value

* refactor extract_model

* fix typo

* resolve comments

* [Refactor] Refactor model inference pipeline (#52)

* move attribute_to_dict to extract_model

* simplify the inference and visualization

* remove unused import

* [Feature] Support SRCNN in mmedit with ONNXRuntime and TensorRT (#45)

* finish mmedit-ort

* edit __init__ files

* add noqa

* add tensorrt support

* 1. Rename "base.py"
2. Move srcnn.py to correct directory

* fix bugs

* remove figures

* align to refactor-v1

* update comment in srcnn

* fix lint

* newfunc -> new_func

* Add visualize.py

split visualize() in each codebase

* fix lint

* fix lint

* remove unnecessary code in ORTRestorer

* remove .api

* edit super(), remove dataset

* [Refactor]: Change name of split to partition (#57)

* refactor mmcls configs

* refactor mmdet configs and split params

* rename rest split to partition from master

* remove base.py

* fix init of inference class

* fix mmocr init, add show_result alias

Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com>
Co-authored-by: RunningLeon <maningsheng@sensetime.com>
Co-authored-by: Yifan Zhou <singlezombie@163.com>

2021-09-01 15:43:49 +08:00

2.7 KiB

Raw Blame History

How to Deploy ncnn Models in deploy_prototype

This tutorial is based on Linux systems like Ubuntu-16.04.

Before starting this tutorial, you should make sure that the prerequisites mentioned by deploy_prototype/README.md are prepared.

Preparation

Download VulkanTools for the compilation of ncnn.

wget https://sdk.lunarg.com/sdk/download/1.2.176.1/linux/vulkansdk-linux-x86_64-1.2.176.1.tar.gz?Human=true -O vulkansdk-linux-x86_64-1.2.176.1.tar.gz
tar -xf vulkansdk-linux-x86_64-1.2.176.1.tar.gz
export VULKAN_SDK=$(pwd)/1.2.176.1/x86_64

Prepare ncnn Framework

Download ncnn source code of tag 20210507

git clone -b 20210507 git@github.com:Tencent/ncnn.git

Make install ncnn library

cd ncnn
mkdir build
cmake -DNCNN_VULKAN=ON -DNCNN_SYSTEM_GLSLANG=ON -DNCNN_BUILD_EXAMPLES=ON -DNCNN_PYTHON=ON -DNCNN_BUILD_TOOLS=ON -DNCNN_BUILD_BENCHMARK=ON -DNCNN_BUILD_TESTS=ON ..
make install

Install pyncnn module
```
cd ncnn/python
pip install .
```

Build ncnn backend ops of deploy_prototype
```
cd deploy_prototype
mkdir build
cd build
cmake -DBUILD_NCNN_OPS=ON -DNCNN_DIR=${PATH_TO_NCNN}/ncnn ..
```
The ${PATH_TO_NCNN} refers as the root directory of ncnn source code.

Install mmdeploy module

cd deploy_prototype
python setup.py develop

Or you will fail on

No module named mmdeploy

FAQs

When running ncnn models for inference with custom ops, it fails and shows the error message like:
```
TypeError: register mm custom layers(): incompatible function arguments. The following argument types are supported:
    1.(ar0: ncnn:Net) -> int

Invoked with: <ncnn.ncnn.Net object at 0x7f7fc4038bb0>
```
This is because of the failure to bind ncnn C++ library to pyncnn. You should build pyncnn from C++ ncnn source code, but not by pip install
When run the tools/deploy.py, it fails:
```
Undefined symbol: __cpu_model
```
This is a bug of gcc-5, you should update to gcc >= 6

Performance Test

MMCls

This table shows the performance of mmclassification models deployed on ncnn.

Dataset: ImageNet val dataset.

Model	Top-1(%)	Top-5(%)
MobileNetV2	71.86 (71.86)	90.42 (90.42)
ResNet	69.88 (70.07)	89.34 (89.44)
ResNeXt	78.61 (78.71)	94.17 (94.12)

The data in the parentheses is the inference result from pytorch. (According to: mmcls model_zoo docs)

2.7 KiB Raw Blame History