Update build doc (#203)
* check in initial version of linux build documents * update en/build/linux.md * remove line break * update build doc * check in build documents in Chinese * reorg chapter * update build documents for windows platform * update zh_cn/windows.md * simplify build commands * update build docs * update build.md * update * reorg the chapters * refine words * correct the commands for windows build * correct build commands for windows * Trim Trailing Whitespace * use prebuilt cmake instead of building it from source * move <br> bewtween <pre><code> and </code></pre> cuz it isn't rendered correctly in github * add build chapter about torchscript * rebase master and update android * fix lint * correct the command of building ppl.cvpull/283/head
lvhan028
GitHub
parent
6948e3d04b
commit
116d2316fc
docs
223
docs/en/build.md
223
docs/en/build.md
|
|
@ -1,20 +1,15 @@
|
|||
## Build MMDeploy
|
||||
# Build MMDeploy
|
||||
|
||||
We provide building methods for both physical and virtual machines. For virtual machine building methods, please refer to
|
||||
[how to use docker](tutorials/how_to_use_docker.md). For physical machine, please follow the steps below.
|
||||
## Download MMDeploy
|
||||
|
||||
### Preparation
|
||||
|
||||
- Download MMDeploy
|
||||
```bash
|
||||
git clone -b master git@github.com:open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
```bash
|
||||
git clone -b master git@github.com:open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
export MMDEPLOY_DIR=$(pwd)
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
Note:
|
||||
Note:
|
||||
|
||||
- If fetching submodule fails, you could get submodule manually by following instructions:
|
||||
|
||||
|
|
@ -29,199 +24,17 @@ We provide building methods for both physical and virtual machines. For virtual
|
|||
cd pybind11
|
||||
git checkout 70a58c5
|
||||
```
|
||||
|
||||
- Install cmake
|
||||
|
||||
Install cmake>=3.14.0, you could refer to [cmake website](https://cmake.org/install) for more detailed info.
|
||||
- If it fails when `git clone` via `SSH`, you can try the `HTTPS` protocol like this:
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y libssl-dev
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0.tar.gz
|
||||
tar -zxvf cmake-3.20.0.tar.gz
|
||||
cd cmake-3.20.0
|
||||
./bootstrap
|
||||
make
|
||||
sudo make install
|
||||
git clone -b master https://github.com/open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
- GCC 7+
|
||||
|
||||
MMDeploy requires compilers that support C++17.
|
||||
```bash
|
||||
# Add repository if ubuntu < 18.04
|
||||
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
|
||||
# Install
|
||||
sudo apt-get install gcc-7
|
||||
sudo apt-get install g++-7
|
||||
```
|
||||
|
||||
### Create Environment
|
||||
|
||||
- Create a conda virtual environment and activate it
|
||||
|
||||
```bash
|
||||
conda create -n mmdeploy python=3.7 -y
|
||||
conda activate mmdeploy
|
||||
```
|
||||
|
||||
- Install PyTorch>=1.8.0, following the [official instructions](https://pytorch.org/)
|
||||
|
||||
```bash
|
||||
# CUDA 11.1
|
||||
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 -c pytorch -c conda-forge
|
||||
```
|
||||
|
||||
- Install mmcv-full. Refer to the [guide](https://github.com/open-mmlab/mmcv#installation) for details.
|
||||
|
||||
```bash
|
||||
export cu_version=cu111 # cuda 11.1
|
||||
export torch_version=torch1.8
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/${cu_version}/${torch_version}/index.html
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Build backend support
|
||||
|
||||
Build the inference engine extension libraries you need.
|
||||
|
||||
- [ONNX Runtime](backends/onnxruntime.md)
|
||||
- [TensorRT](backends/tensorrt.md)
|
||||
- [ncnn](backends/ncnn.md)
|
||||
- [pplnn](backends/pplnn.md)
|
||||
- [OpenVINO](backends/openvino.md)
|
||||
|
||||
### Install mmdeploy
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR} # To mmdeploy root directory
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Note**
|
||||
|
||||
- Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
|
||||
To use optional dependencies, install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e . [optional]`).
|
||||
Valid keys for the extras field are: `all`, `tests`, `build`, `optional`.
|
||||
|
||||
### Build SDK
|
||||
|
||||
You can skip this chapter if you are only interested in model converter.
|
||||
|
||||
#### Dependencies
|
||||
|
||||
Currently, SDK is tested on Linux x86-64, more platforms will be added in the future. The following packages are required to build MMDeploy SDK.
|
||||
|
||||
Each package's installation command is given based on Ubuntu 18.04.
|
||||
|
||||
- OpenCV 3+
|
||||
|
||||
```bash
|
||||
sudo apt-get install libopencv-dev
|
||||
```
|
||||
|
||||
- spdlog 0.16+
|
||||
|
||||
``` bash
|
||||
sudo apt-get install libspdlog-dev
|
||||
```
|
||||
|
||||
On Ubuntu 16.04, please use the following command
|
||||
```bash
|
||||
wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
|
||||
sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
|
||||
```
|
||||
|
||||
You can also build spdlog from its source to enjoy its latest features. But be sure to add **`-fPIC`** compilation flags at first.
|
||||
|
||||
- pplcv
|
||||
|
||||
A high-performance image processing library of openPPL supporting x86 and cuda platforms.</br>
|
||||
It is **OPTIONAL** which only be needed if `cuda` platform is required.
|
||||
|
||||
Using v0.6.1, since latest updates have broughtup breaking changes
|
||||
```Bash
|
||||
wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip
|
||||
unzip v0.6.1.zip && mv ppl.cv-0.6.1 ppl.cv
|
||||
cd ppl.cv
|
||||
./build.sh cuda
|
||||
```
|
||||
|
||||
- backend engines
|
||||
|
||||
SDK uses the same backends as model converter does. Please follow [build backend](#build-backend-support) guide to install your interested backend.
|
||||
|
||||
#### Set Build Option
|
||||
|
||||
- Turn on SDK build switch
|
||||
|
||||
`-DMMDEPLOY_BUILD_SDK=ON`
|
||||
|
||||
- Enabling Devices
|
||||
|
||||
By default, only CPU device is included in the target devices. You can enable device support for other devices by
|
||||
passing a semicolon separated list of device names to `MMDEPLOY_TARGET_DEVICES` variable, e.g. `-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"`. </br>
|
||||
Currently, the following devices are supported.
|
||||
|
||||
| device | name | path setter |
|
||||
|--------|-------|-------------|
|
||||
| Host | cpu | N/A |
|
||||
| CUDA | cuda | CUDA_TOOLKIT_ROOT_DIR & pplcv_DIR |
|
||||
|
||||
If you have multiple CUDA versions installed on your system, you will need to pass `CUDA_TOOLKIT_ROOT_DIR` to cmake to specify the version. </br>
|
||||
Meanwhile, `pplcv_DIR` has to be provided in order to build image processing operators on cuda platform.
|
||||
|
||||
|
||||
- Enabling inference engines
|
||||
|
||||
**By default, no target inference engines are set**, since it's highly dependent on the use case.
|
||||
`MMDEPLOY_TARGET_BACKENDS` must be set to a semicolon separated list of inference engine names,
|
||||
e.g. `-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"`
|
||||
A path to the inference engine library is also needed. The following backends are currently supported
|
||||
|
||||
| library | name | path setter |
|
||||
|-------------|----------|--------------------------|
|
||||
| PPL.nn | pplnn | pplnn_DIR |
|
||||
| ncnn | ncnn | ncnn_DIR |
|
||||
| ONNXRuntime | ort | ONNXRUNTIME_DIR |
|
||||
| TensorRT | trt | TENSORRT_DIR & CUDNN_DIR |
|
||||
| OpenVINO | openvino | InferenceEngine_DIR |
|
||||
|
||||
- Enabling codebase's postprocess components
|
||||
|
||||
`MMDEPLOY_CODEBASES` MUST be specified by a semicolon separated list of codebase names.
|
||||
The currently supported codebases are 'mmcls', 'mmdet', 'mmedit', 'mmseg', 'mmocr'.
|
||||
Instead of listing them one by one in `MMDEPLOY_CODEBASES`, user can also pass `all` to enable all of them, i.e.,
|
||||
`-DMMDEPLOY_CODEBASES=all`
|
||||
|
||||
|
||||
- Put it all together
|
||||
|
||||
The following is a recipe for building MMDeploy SDK with cpu device and ONNXRuntime support
|
||||
```Bash
|
||||
mkdir build && cd build
|
||||
cmake .. \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DONNXRUNTIME_DIR=/path/to/onnxruntime \
|
||||
-DMMDEPLOY_TARGET_DEVICES=cpu \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=ort \
|
||||
-DMMDEPLOY_CODEBASES=all
|
||||
cmake --build . -- -j$(nproc) && cmake --install .
|
||||
```
|
||||
|
||||
Here is another example to build MMDeploy SDK with cuda device and TensorRT backend
|
||||
|
||||
```Bash
|
||||
mkdir build && cd build
|
||||
cmake .. \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-Dpplcv_DIR=/path/to/ppl.cv/cuda-build/install/lib/cmake/ppl \
|
||||
-DTENSORRT_DIR=/path/to/tensorrt \
|
||||
-DCUDNN_DIR=/path/to/cudnn \
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=trt \
|
||||
-DMMDEPLOY_CODEBASES=all
|
||||
cmake --build . -- -j$(nproc) && cmake --install .
|
||||
```
|
||||
## Build MMDeploy
|
||||
Please visit the following links to find out how to build MMDeploy according to the target platform.
|
||||
- [Linux-x86_64](build/linux.md)
|
||||
- [Windows](build/windows.md)
|
||||
- [Android-aarch64](build/android.md)
|
||||
- [NVIDIA Jetson](tutorials/how_to_install_mmdeploy_on_jetsons.md)
|
||||
|
|
|
|||
|
|
@ -32,13 +32,9 @@ This doc is only for how to build SDK using android toolchain on linux.
|
|||
**Make sure cmake version >= 3.14.0**. If not, you can follow instructions below to install cmake 3.20.0. For more versions of cmake, please refer to [cmake website](https://cmake.org/install).
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y libssl-dev
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0.tar.gz
|
||||
tar -zxvf cmake-3.20.0.tar.gz
|
||||
cd cmake-3.20.0
|
||||
./bootstrap
|
||||
make
|
||||
sudo make install
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
|
||||
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
|
||||
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/
|
||||
```
|
||||
|
||||
- ANDROID NDK 19+
|
||||
|
|
@ -206,5 +202,5 @@ cmake .. \
|
|||
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
|
||||
-DANDROID_ABI=arm64-v8a \
|
||||
-DANDROID_PLATFORM=android-30
|
||||
cmake --build . -- -j$(nproc)
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1 +1,453 @@
|
|||
TODO
|
||||
# Build for Linux-x86_64
|
||||
|
||||
- [Build for Linux-x86_64](#build-for-linux-x86_64)
|
||||
- [Dockerfile (RECOMMENDED)](#dockerfile-recommended)
|
||||
- [Build From Source](#build-from-source)
|
||||
- [Install Toolchains](#install-toolchains)
|
||||
- [Install Dependencies](#install-dependencies)
|
||||
- [Install Dependencies for Model Converter](#install-dependencies-for-model-converter)
|
||||
- [Install Dependencies for SDK](#install-dependencies-for-sdk)
|
||||
- [Install Inference Engines for MMDeploy](#install-inference-engines-for-mmdeploy)
|
||||
- [Build MMDeploy](#build-mmdeploy)
|
||||
- [Build Options Spec](#build-options-spec)
|
||||
- [Build Model Converter](#build-model-converter)
|
||||
- [Build Custom Ops](#build-custom-ops)
|
||||
- [Install Model Converter](#install-model-converter)
|
||||
- [Build SDK](#build-sdk)
|
||||
- [Build Demo](#build-demo)
|
||||
|
||||
---
|
||||
MMDeploy provides two build ways for linux-x86_64 platform, including dockerfile and build from source.
|
||||
|
||||
## Dockerfile (RECOMMENDED)
|
||||
please refer to [how to use docker](../tutorials/how_to_use_docker.md).
|
||||
|
||||
## Build From Source
|
||||
|
||||
### Install Toolchains
|
||||
|
||||
- cmake
|
||||
|
||||
**Make sure cmake version >= 3.14.0**. The below script shows how to install cmake 3.20.0. You can find more versions [here](https://cmake.org/install).
|
||||
|
||||
```bash
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
|
||||
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
|
||||
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/
|
||||
```
|
||||
|
||||
- GCC 7+
|
||||
|
||||
MMDeploy requires compilers that support C++17.
|
||||
```bash
|
||||
# Add repository if ubuntu < 18.04
|
||||
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
|
||||
sudo apt-get update
|
||||
sudo apt-get install gcc-7
|
||||
sudo apt-get install g++-7
|
||||
```
|
||||
|
||||
### Install Dependencies
|
||||
#### Install Dependencies for Model Converter
|
||||
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME </th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>conda </td>
|
||||
<td>Please install conda according to the official <a href="https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html">guide</a>. <br>
|
||||
Create a conda virtual environment and activate it. <br>
|
||||
<pre><code>
|
||||
conda create -n mmdeploy python=3.7 -y
|
||||
conda activate mmdeploy
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PyTorch <br>(>=1.8.0) </td>
|
||||
<td>
|
||||
Install PyTorch>=1.8.0 by following the <a href="https://pytorch.org/">official instructions</a>. Be sure the CUDA version PyTorch requires matches that in your host.
|
||||
<pre><code>
|
||||
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 -c pytorch -c conda-forge
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>mmcv-full </td>
|
||||
<td>Install mmcv-full as follows. Refer to the <a href="https://github.com/open-mmlab/mmcv#installation">guide</a> for details.
|
||||
<pre><code>
|
||||
export cu_version=cu111 # cuda 11.1
|
||||
export torch_version=torch1.8
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/${cu_version}/${torch_version}/index.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### Install Dependencies for SDK
|
||||
|
||||
You can skip this chapter if you are only interested in the model converter.
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME </th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>spdlog </td>
|
||||
<td>
|
||||
On Ubuntu >=18.04,
|
||||
<pre><code>
|
||||
sudo apt-get install libspdlog-dev
|
||||
</code></pre>
|
||||
On Ubuntu 16.04,
|
||||
<pre><code>
|
||||
wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
|
||||
sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
|
||||
</code></pre>
|
||||
You can also build spdlog from its source to enjoy its latest features. But be sure to open cmake option <code>-DCMAKE_POSITION_INDEPENDENT_CODE=ON</code>.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenCV<br>(>=3.0) </td>
|
||||
<td>
|
||||
On Ubuntu >=18.04,
|
||||
<pre><code>
|
||||
sudo apt-get install libopencv-dev
|
||||
</code></pre>
|
||||
On Ubuntu 16.04, OpenCV has to be built from the source code. Please refer to the <a href="https://docs.opencv.org/3.4/d7/d9f/tutorial_linux_install.html">guide</a>.
|
||||
</td>
|
||||
|
||||
</tr>
|
||||
<tr>
|
||||
<td>pplcv </td>
|
||||
<td>A high-performance image processing library of openPPL.<br>
|
||||
<b>It is optional which only be needed if <code>cuda</code> platform is required.
|
||||
Now, MMDeploy supports v0.6.2 and has to use <code>git clone</code> to download it.</b><br>
|
||||
<pre><code>
|
||||
git clone https://github.com/openppl-public/ppl.cv.git
|
||||
cd ppl.cv
|
||||
export PPLCV_DIR=$(pwd)
|
||||
git checkout tags/v0.6.2 -b v0.6.2
|
||||
./build.sh cuda
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### Install Inference Engines for MMDeploy
|
||||
|
||||
Both MMDeploy's model converter and SDK share the same inference engines.
|
||||
|
||||
You can select you interested inference engines and do the installation by following the given commands.
|
||||
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME</th>
|
||||
<th>PACKAGE</th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>ONNXRuntime</td>
|
||||
<td>onnxruntime<br>(>=1.8.1) </td>
|
||||
<td>
|
||||
1. Install python package
|
||||
<pre><code>pip install onnxruntime==1.8.1</code></pre>
|
||||
2. Download the linux prebuilt binary package from <a href="https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1">here</a>. Extract it and export environment variables as below:
|
||||
<pre><code>
|
||||
wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
|
||||
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
|
||||
cd onnxruntime-linux-x64-1.8.1
|
||||
export ONNXRUNTIME_DIR=$(pwd)
|
||||
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">TensorRT<br> </td>
|
||||
<td>TensorRT <br> </td>
|
||||
<td>
|
||||
1. Login <a href="https://www.nvidia.com/">NVIDIA</a> and download the TensorRT tar file that matches the CPU architecture and CUDA version you are using from <a href="https://developer.nvidia.com/nvidia-tensorrt-download">here</a>. Follow the <a href="https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar">guide</a> to install TensorRT. <br>
|
||||
2. Here is an example of installing TensorRT 8.2 GA Update 2 for Linux x86_64 and CUDA 11.x that you can refer to. First of all, click <a href="https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.2.3.0/tars/tensorrt-8.2.3.0.linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz">here</a> to download CUDA 11.x TensorRT 8.2.3.0 and then install it like below:
|
||||
<pre><code>
|
||||
cd /the/path/of/tensorrt/tar/gz/file
|
||||
tar -zxvf TensorRT-8.2.3.0.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
|
||||
pip install TensorRT-8.2.3.0/python/tensorrt-8.2.3.0-cp37-none-linux_x86_64.whl
|
||||
export TENSORRT_DIR=$(pwd)/TensorRT-8.2.3.0
|
||||
export LD_LIBRARY_PATH=$TENSORRT_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cuDNN </td>
|
||||
<td>
|
||||
1. Download cuDNN that matches the CPU architecture, CUDA version and TensorRT version you are using from <a href="https://developer.nvidia.com/rdp/cudnn-archive"> cuDNN Archive</a>. <br>
|
||||
In the above TensorRT's installation example, it requires cudnn8.2. Thus, you can download <a href="https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.1.32/11.3_06072021/cudnn-11.3-linux-x64-v8.2.1.32.tgz">CUDA 11.x cuDNN 8.2</a><br>
|
||||
2. Extract the compressed file and set the environment variables
|
||||
<pre><code>
|
||||
cd /the/path/of/cudnn/tgz/file
|
||||
tar -zxvf cudnn-11.3-linux-x64-v8.2.1.32.tgz
|
||||
export CUDNN_DIR=$(pwd)/cuda
|
||||
export LD_LIBRARY_PATH=$CUDNN_DIR/lib64:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PPL.NN</td>
|
||||
<td>ppl.nn </td>
|
||||
<td>
|
||||
1. Please follow the <a href="https://github.com/openppl-public/ppl.nn/blob/master/docs/en/building-from-source.md">guide</a> to build <code>ppl.nn</code> and install <code>pyppl</code>.<br>
|
||||
2. Export pplnn's root path to environment variable
|
||||
<pre><code>
|
||||
cd ppl.nn
|
||||
export PPLNN_DIR=$(pwd)
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenVINO</td>
|
||||
<td>openvino </td>
|
||||
<td>1. Install <a href="https://docs.openvino.ai/2021.4/get_started.html">OpenVINO</a> package
|
||||
<pre><code>
|
||||
pip install openvino-dev
|
||||
</code></pre>
|
||||
2. <b>Optional</b>. If you want to use OpenVINO in MMDeploy SDK, please install and configure it by following the <a href="https://docs.openvino.ai/2021.4/openvino_docs_install_guides_installing_openvino_linux.html#install-openvino">guide</a>.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ncnn </td>
|
||||
<td>ncnn </td>
|
||||
<td>1. Download and build ncnn according to its <a href="https://github.com/Tencent/ncnn/wiki/how-to-build">wiki</a>.
|
||||
Make sure to enable <code>-DNCNN_PYTHON=ON</code> in your build command. <br>
|
||||
2. Export ncnn's root path to environment variable
|
||||
<pre><code>
|
||||
cd ncnn
|
||||
export NCNN_DIR=$(pwd)
|
||||
</code></pre>
|
||||
3. Install pyncnn
|
||||
<pre><code>
|
||||
cd ${NCNN_DIR}/python
|
||||
pip install -e .
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>TorchScript</td>
|
||||
<td>libtorch</td>
|
||||
<td>
|
||||
1. Download libtorch from <a href="https://pytorch.org/get-started/locally/">here</a>. Please note that only <b>Pre-cxx11 ABI</b> and <b>version 1.8.1+</b> on Linux platform are supported by now. For previous versions of libtorch, you can find them in the <a href="https://github.com/pytorch/pytorch/issues/40961#issuecomment-1017317786">issue comment</a>. <br>
|
||||
2. Take Libtorch1.8.1+cu111 as an example. You can install it like this:
|
||||
<pre><code>
|
||||
wget https://download.pytorch.org/libtorch/cu111/libtorch-shared-with-deps-1.8.1%2Bcu111.zip
|
||||
unzip libtorch-shared-with-deps-1.8.1+cu111.zip
|
||||
cd libtorch
|
||||
export Torch_DIR=$(pwd)
|
||||
export LD_LIBRARY_PATH=$Torch_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Note: <br>
|
||||
If you want to make the above environment variables permanent, you could add them to <code>~/.bashrc</code>. Take the ONNXRuntime for example,
|
||||
|
||||
```bash
|
||||
echo '# set env for onnxruntime' >> ~/.bashrc
|
||||
echo "export ONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}" >> ~/.bashrc
|
||||
echo 'export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
### Build MMDeploy
|
||||
```bash
|
||||
cd /the/root/path/of/MMDeploy
|
||||
export MMDEPLOY_DIR=$(pwd)
|
||||
```
|
||||
|
||||
#### Build Options Spec
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME</th>
|
||||
<th>VALUE</th>
|
||||
<th>DEFAULT</th>
|
||||
<th>REMARK</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>Switch to build MMDeploy SDK</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK_PYTHON_API</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>switch to build MMDeploy SDK python package</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_TEST</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>Switch to build MMDeploy SDK unittest cases</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_DEVICES</td>
|
||||
<td>{"cpu", "cuda"}</td>
|
||||
<td>cpu</td>
|
||||
<td>Enable target device. You can enable more by
|
||||
passing a semicolon separated list of device names to <code>MMDEPLOY_TARGET_DEVICES</code> variable, e.g. <code>-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"</code> </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_BACKENDS</td>
|
||||
<td>{"trt", "ort", "pplnn", "ncnn", "openvino", "torchscript"}</td>
|
||||
<td>N/A</td>
|
||||
<td>Enabling inference engine. <b>By default, no target inference engine is set, since it highly depends on the use case.</b> When more than one engine are specified, it has to be set with a semicolon separated list of inference backend names, e.g. <pre><code>-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"</code></pre>
|
||||
After specifying the inference engine, it's package path has to be passed to cmake as follows, <br>
|
||||
1. <b>trt</b>: TensorRT. <code>TENSORRT_DIR</code> and <code>CUDNN_DIR</code> are needed.
|
||||
<pre><code>
|
||||
-DTENSORRT_DIR=${TENSORRT_DIR}
|
||||
-DCUDNN_DIR=${CUDNN_DIR}
|
||||
</code></pre>
|
||||
2. <b>ort</b>: ONNXRuntime. <code>ONNXRUNTIME_DIR</code> is needed.
|
||||
<pre><code>-DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}</code></pre>
|
||||
3. <b>pplnn</b>: PPL.NN. <code>pplnn_DIR</code> is needed.
|
||||
<pre><code>-Dpplnn_DIR=${PPLNN_DIR}</code></pre>
|
||||
4. <b>ncnn</b>: ncnn. <code>ncnn_DIR</code> is needed.
|
||||
<pre><code>-Dncnn_DIR=${NCNN_DIR}</code></pre>
|
||||
5. <b>openvino</b>: OpenVINO. <code>InferenceEngine_DIR</code> is needed.
|
||||
<pre><code>-DInferenceEngine_DIR=${INTEL_OPENVINO_DIR}/deployment_tools/inference_engine/share</code></pre>
|
||||
6. <b>torchscript</b>: TorchScript. <code>Torch_DIR</code> is needed.
|
||||
<pre><code>-DTorch_DIR=${Torch_DIR}</code></pre>
|
||||
Currently, <b>The Model Converter supports torchscript, but SDK doesn't</b>.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_CODEBASES</td>
|
||||
<td>{"mmcls", "mmdet", "mmseg", "mmedit", "mmocr", "all"}</td>
|
||||
<td>all</td>
|
||||
<td>Enable codebase's postprocess modules. You can provide a semicolon separated list of codebase names to enable them, e.g., <code>-DMMDEPLOY_CODEBASES="mmcls;mmdet"</code>. Or you can pass <code>all</code> to enable them all, i.e., <code>-DMMDEPLOY_CODEBASES=all</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BUILD_SHARED_LIBS</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>ON</td>
|
||||
<td>Switch to build shared library or static library of MMDeploy SDK</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### Build Model Converter
|
||||
|
||||
##### Build Custom Ops
|
||||
If one of inference engines among ONNXRuntime, TensorRT, ncnn and libtorch is selected, you have to build the corresponding custom ops.
|
||||
|
||||
- **ONNXRuntime** Custom Ops
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **TensorRT** Custom Ops
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${TENSORRT_DIR} -DCUDNN_DIR=${CUDNN_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **ncnn** Custom Ops
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ncnn -Dncnn_DIR=${NCNN_DIR}/build/install/lib/cmake/ncnn ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **TorchScript** Custom Ops
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=torchscript -DTorch_DIR=${Torch_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
##### Install Model Converter
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
pip install -e .
|
||||
```
|
||||
**Note**
|
||||
|
||||
- Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
|
||||
To use optional dependencies, install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e .[optional]`).
|
||||
Valid keys for the extras field are: `all`, `tests`, `build`, `optional`.
|
||||
#### Build SDK
|
||||
|
||||
MMDeploy provides two recipes as shown below for building SDK with ONNXRuntime and TensorRT as inference engines respectively.
|
||||
You can also activate other engines after the model.
|
||||
|
||||
- cpu + ONNXRuntime
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake .. \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
|
||||
-DMMDEPLOY_TARGET_DEVICES=cpu \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=ort \
|
||||
-DMMDEPLOY_CODEBASES=all \
|
||||
-DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}
|
||||
|
||||
make -j$(nproc) && make install
|
||||
```
|
||||
|
||||
- cuda + TensorRT
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake .. \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=trt \
|
||||
-DMMDEPLOY_CODEBASES=all \
|
||||
-Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
|
||||
-DTENSORRT_DIR=${TENSORRT_DIR} \
|
||||
-DCUDNN_DIR=${CUDNN_DIR}
|
||||
|
||||
make -j$(nproc) && make install
|
||||
```
|
||||
|
||||
#### Build Demo
|
||||
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}/build/install/example
|
||||
mkdir -p build && cd build
|
||||
cmake .. -DMMDeploy_DIR=${MMDEPLOY_DIR}/build/install/lib/cmake/MMDeploy
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1 +1,379 @@
|
|||
TODO
|
||||
# Build for Windows
|
||||
|
||||
- [Build for Windows](#build-for-windows)
|
||||
- [Build From Source](#build-from-source)
|
||||
- [Install Toolchains](#install-toolchains)
|
||||
- [Install Dependencies](#install-dependencies)
|
||||
- [Install Dependencies for Model Converter](#install-dependencies-for-model-converter)
|
||||
- [Install Dependencies for SDK](#install-dependencies-for-sdk)
|
||||
- [Install Inference Engines for MMDeploy](#install-inference-engines-for-mmdeploy)
|
||||
- [Build MMDeploy](#build-mmdeploy)
|
||||
- [Build Options Spec](#build-options-spec)
|
||||
- [Build Model Converter](#build-model-converter)
|
||||
- [Build Custom Ops](#build-custom-ops)
|
||||
- [Install Model Converter](#install-model-converter)
|
||||
- [Build SDK](#build-sdk)
|
||||
- [Build Demo](#build-demo)
|
||||
- [Note](#note)
|
||||
|
||||
---
|
||||
Currently, MMDeploy only provides build-from-source method for windows platform. Prebuilt package will be released in the future.
|
||||
|
||||
## Build From Source
|
||||
All the commands listed in the following chapters are verified on **Windows 10**.
|
||||
|
||||
### Install Toolchains
|
||||
1. Download and install [Visual Studio 2019](https://visualstudio.microsoft.com)
|
||||
2. Add the path of `cmake` to the environment variable `PATH`, i.e., "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin"
|
||||
3. Install cuda toolkit if NVIDIA gpu is available. You can refer to the official [guide](https://developer.nvidia.com/cuda-downloads).
|
||||
|
||||
### Install Dependencies
|
||||
#### Install Dependencies for Model Converter
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME </th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>conda </td>
|
||||
<td> Please install conda according to the official <a href="https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html">guide</a>. <br>
|
||||
After installation, open <code>anaconda powershell prompt</code> under the Start Menu <b>as the administrator</b>, because: <br>
|
||||
1. <b>All the commands listed in the following text are verified in anaconda powershell </b><br>
|
||||
2. <b>As an administrator, you can install the thirdparty libraries to the system path so as to simplify MMDeploy build command</b><br>
|
||||
Note: if you are familiar with how cmake works, you can also use <code>anaconda powershell prompt</code> as an ordinary user.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PyTorch <br>(>=1.8.0) </td>
|
||||
<td>
|
||||
Install PyTorch>=1.8.0 by following the <a href="https://pytorch.org/">official instructions</a>. Be sure the CUDA version PyTorch requires matches that in your host.
|
||||
<pre><code>
|
||||
pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>mmcv-full </td>
|
||||
<td>Install mmcv-full as follows. Refer to the <a href="https://github.com/open-mmlab/mmcv#installation">guide</a> for details.
|
||||
<pre><code>
|
||||
$env:cu_version="cu111"
|
||||
$env:torch_version="torch1.8.0"
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/${cu_version}/${torch_version}/index.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### Install Dependencies for SDK
|
||||
|
||||
You can skip this chapter if you are only interested in the model converter.
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME </th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>spdlog </td>
|
||||
<td>
|
||||
spdlog is a very fast, header-only/compiled, C++ logging library. You can install it like this,
|
||||
<pre><code>
|
||||
Invoke-WebRequest -Uri https://github.com/gabime/spdlog/archive/refs/tags/v1.9.2.zip -OutFile spdlog-1.9.2.zip
|
||||
Expand-Archive spdlog-1.9.2.zip .
|
||||
cd spdlog-1.9.2
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build . --target install --config Release -- /m
|
||||
cd ../..
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenCV<br>(>=3.0) </td>
|
||||
<td>
|
||||
1. Find and download OpenCV 3+ for windows from <a href="https://github.com/opencv/opencv/releases">here</a>.<br>
|
||||
2. You can download the prebuilt package and install it to the target directory. Or you can build OpenCV from its source. <br>
|
||||
3. Find where <code>OpenCVConfig.cmake</code> locates in the installation directory. And export its path to the environment variable <code>PATH</code> like this,
|
||||
<pre><code>$env:path = "\the\path\where\OpenCVConfig.cmake\locates;" + "$env:path"</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>pplcv </td>
|
||||
<td>A high-performance image processing library of openPPL.<br>
|
||||
<b>It is optional which only be needed if <code>cuda</code> platform is required.
|
||||
Now, MMDeploy supports v0.6.2 and has to use <code>git clone</code> to download it.</b><br>
|
||||
<pre><code>
|
||||
git clone https://github.com/openppl-public/ppl.cv.git
|
||||
cd ppl.cv
|
||||
git checkout tags/v0.6.2 -b v0.6.2
|
||||
$env:PPLCV_DIR = "$pwd"
|
||||
mkdir pplcv-build
|
||||
cd pplcv-build
|
||||
cmake .. -G "Visual Studio 16 2019" -T v142 -A x64 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install -DHPCC_USE_CUDA=ON -DHPCC_MSVC_MD=ON
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
cd ../..
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### Install Inference Engines for MMDeploy
|
||||
|
||||
Both MMDeploy's model converter and SDK share the same inference engines.
|
||||
You can select your interested inference engines and do the installation by following the given commands.
|
||||
|
||||
**Currently, MMDeploy only verified ONNXRuntime and TensorRT for windows platform**.
|
||||
As for the rest, MMDeploy will support them in the future.
|
||||
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME</th>
|
||||
<th>PACKAGE</th>
|
||||
<th>INSTALLATION </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>ONNXRuntime</td>
|
||||
<td>onnxruntime<br>(>=1.8.1) </td>
|
||||
<td>
|
||||
1. Install python package
|
||||
<pre><code>pip install onnxruntime==1.8.1</code></pre>
|
||||
2. Download the windows prebuilt binary package from <a href="https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1">here</a>. Extract it and export environment variables as below:
|
||||
<pre><code>
|
||||
Invoke-WebRequest -Uri https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-win-x64-1.8.1.zip -OutFile onnxruntime-win-x64-1.8.1.zip
|
||||
Expand-Archive onnxruntime-win-x64-1.8.1.zip .
|
||||
$env:ONNXRUNTIME_DIR = "$pwd\onnxruntime-win-x64-1.8.1"
|
||||
$env:path = "$env:ONNXRUNTIME_DIR\lib;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">TensorRT<br> </td>
|
||||
<td>TensorRT <br> </td>
|
||||
<td>
|
||||
1. Login <a href="https://www.nvidia.com/">NVIDIA</a> and download the TensorRT tar file that matches the CPU architecture and CUDA version you are using from <a href="https://developer.nvidia.com/nvidia-tensorrt-download">here</a>. Follow the <a href="https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar">guide</a> to install TensorRT. <br>
|
||||
2. Here is an example of installing TensorRT 8.2 GA Update 2 for Windows x86_64 and CUDA 11.x that you can refer to. <br> First of all, click <a href="https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.2.3.0/zip/TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip">here</a> to download CUDA 11.x TensorRT 8.2.3.0 and then install it like below:
|
||||
<pre><code>
|
||||
cd \the\path\of\tensorrt\zip\file
|
||||
Expand-Archive TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip .
|
||||
pip install $env:TENSORRT_DIR\python\tensorrt-8.2.3.0-cp37-none-win_amd64.whl
|
||||
$env:TENSORRT_DIR = "$pwd\TensorRT-8.2.3.0"
|
||||
$env:path = "$env:TENSORRT_DIR\lib;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cuDNN </td>
|
||||
<td>
|
||||
1. Download cuDNN that matches the CPU architecture, CUDA version and TensorRT version you are using from <a href="https://developer.nvidia.com/rdp/cudnn-archive"> cuDNN Archive</a>. <br>
|
||||
In the above TensorRT's installation example, it requires cudnn8.2. Thus, you can download <a href="https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.1.32/11.3_06072021/cudnn-11.3-windows-x64-v8.2.1.32.zip">CUDA 11.x cuDNN 8.2</a><br>
|
||||
2. Extract the zip file and set the environment variables
|
||||
<pre><code>
|
||||
cd \the\path\of\cudnn\zip\file
|
||||
Expand-Archive cudnn-11.3-windows-x64-v8.2.1.32.zip .
|
||||
$env:CUDNN_DIR="$pwd\cuda"
|
||||
$env:path = "$env:CUDNN_DIR\bin;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PPL.NN</td>
|
||||
<td>ppl.nn </td>
|
||||
<td>TODO </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenVINO</td>
|
||||
<td>openvino </td>
|
||||
<td>TODO </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ncnn </td>
|
||||
<td>ncnn </td>
|
||||
<td>TODO </td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
### Build MMDeploy
|
||||
|
||||
```powershell
|
||||
cd \the\root\path\of\MMDeploy
|
||||
$env:MMDEPLOY_DIR="$pwd"
|
||||
```
|
||||
|
||||
#### Build Options Spec
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>NAME</th>
|
||||
<th>VALUE</th>
|
||||
<th>DEFAULT</th>
|
||||
<th>REMARK</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>Switch to build MMDeploy SDK</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK_PYTHON_API</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>switch to build MMDeploy SDK python package</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_TEST</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>Switch to build MMDeploy SDK unittest cases</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_DEVICES</td>
|
||||
<td>{"cpu", "cuda"}</td>
|
||||
<td>cpu</td>
|
||||
<td>Enable target device. You can enable more by
|
||||
passing a semicolon separated list of device names to <code>MMDEPLOY_TARGET_DEVICES</code> variable, e.g. <code>-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"</code> </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_BACKENDS</td>
|
||||
<td>{"trt", "ort", "pplnn", "ncnn", "openvino"}</td>
|
||||
<td>N/A</td>
|
||||
<td>Enabling inference engine. <b>By default, no target inference engine is set, since it highly depends on the use case.</b> When more than one engine are specified, it has to be set with a semicolon separated list of inference backend names, e.g. <pre><code>-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"</code></pre>
|
||||
After specifying the inference engine, it's package path has to be passed to cmake as follows, <br>
|
||||
1. <b>trt</b>: TensorRT. <code>TENSORRT_DIR</code> and <code>CUDNN_DIR</code> are needed.
|
||||
<pre><code>
|
||||
-DTENSORRT_DIR=$env:TENSORRT_DIR
|
||||
-DCUDNN_DIR=$env:CUDNN_DIR
|
||||
</code></pre>
|
||||
2. <b>ort</b>: ONNXRuntime. <code>ONNXRUNTIME_DIR</code> is needed.
|
||||
<pre><code>-DONNXRUNTIME_DIR=$env:ONNXRUNTIME_DIR</code></pre>
|
||||
3. <b>pplnn</b>: PPL.NN. <code>pplnn_DIR</code> is needed. MMDeploy hasn't verified it yet.
|
||||
4. <b>ncnn</b>: ncnn. <code>ncnn_DIR</code> is needed. MMDeploy hasn't verified it yet.
|
||||
5. <b>openvino</b>: OpenVINO. <code>InferenceEngine_DIR</code> is needed. MMDeploy hasn't verified it yet.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_CODEBASES</td>
|
||||
<td>{"mmcls", "mmdet", "mmseg", "mmedit", "mmocr", "all"}</td>
|
||||
<td>all</td>
|
||||
<td>Enable codebase's postprocess modules. You can provide a semicolon separated list of codebase names to enable them. Or you can pass <code>all</code> to enable them all, i.e., <code>-DMMDEPLOY_CODEBASES=all</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BUILD_SHARED_LIBS</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>ON</td>
|
||||
<td>Switch to build shared library or static library of MMDeploy SDK</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
#### Build Model Converter
|
||||
|
||||
##### Build Custom Ops
|
||||
If one of inference engines among ONNXRuntime, TensorRT and ncnn is selected, you have to build the corresponding custom ops.
|
||||
|
||||
- **ONNXRuntime** Custom Ops
|
||||
|
||||
```powershell
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="ort" -DONNXRUNTIME_DIR="$env:ONNXRUNTIME_DIR"
|
||||
cmake --build . --config Release -- /m
|
||||
|
||||
- **TensorRT** Custom Ops
|
||||
|
||||
```powershell
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="trt" -DTENSORRT_DIR="$env:TENSORRT_DIR" -DCUDNN_DIR="$env:CUDNN_DIR"
|
||||
cmake --build . --config Release -- /m
|
||||
|
||||
- **ncnn** Custom Ops
|
||||
|
||||
TODO
|
||||
|
||||
##### Install Model Converter
|
||||
|
||||
```powershell
|
||||
cd $env:MMDEPLOY_DIR
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Note**
|
||||
|
||||
- Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
|
||||
To use optional dependencies, install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e .[optional]`).
|
||||
Valid keys for the extras field are: `all`, `tests`, `build`, `optional`.
|
||||
|
||||
#### Build SDK
|
||||
|
||||
MMDeploy provides two recipes as shown below for building SDK with ONNXRuntime and TensorRT as inference engines respectively.
|
||||
You can also activate other engines after the model.
|
||||
|
||||
- cpu + ONNXRuntime
|
||||
|
||||
```PowerShell
|
||||
cd $env:MMDEPLOY_DIR
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDEPLOY_BUILD_SDK=ON `
|
||||
-DMMDEPLOY_TARGET_DEVICES="cpu" `
|
||||
-DMMDEPLOY_TARGET_BACKENDS="ort" `
|
||||
-DMMDEPLOY_CODEBASES="all" `
|
||||
-DONNXRUNTIME_DIR="$env:ONNXRUNTIME_DIR"
|
||||
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
```
|
||||
|
||||
- cuda + TensorRT
|
||||
|
||||
```PowerShell
|
||||
cd $env:MMDEPLOY_DIR
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDEPLOY_BUILD_SDK=ON `
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda" `
|
||||
-DMMDEPLOY_TARGET_BACKENDS="trt" `
|
||||
-DMMDEPLOY_CODEBASES="all" `
|
||||
-Dpplcv_DIR="$env:PPLCV_DIR/pplcv-build/install/lib/cmake/ppl" `
|
||||
-DTENSORRT_DIR="$env:TENSORRT_DIR" `
|
||||
-DCUDNN_DIR="$env:CUDNN_DIR"
|
||||
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
```
|
||||
|
||||
#### Build Demo
|
||||
|
||||
```PowerShell
|
||||
cd $env:MMDEPLOY_DIR\build\install\example
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDeploy_DIR="$env:MMDEPLOY_DIR/build/install/lib/cmake/MMDeploy"
|
||||
|
||||
cmake --build . --config Release -- /m
|
||||
|
||||
$env:path = "$env:MMDEPLOY_DIR/build/install/bin;" + $env:path
|
||||
```
|
||||
|
||||
### Note
|
||||
1. Release / Debug libraries can not be mixed. If MMDeploy is built with Release mode, all its dependent thirdparty libraries have to be built in Release mode too and vice versa.
|
||||
|
|
|
|||
|
|
@ -28,3 +28,8 @@
|
|||
- "error: [TensorRT] INTERNAL ERROR: Assertion failed: cublasStatus == CUBLAS_STATUS_SUCCESS"
|
||||
|
||||
TRT 7.2.1 switches to use cuBLASLt (previously it was cuBLAS). cuBLASLt is the defaulted choice for SM version >= 7.0. You may need CUDA-10.2 Patch 1 (Released Aug 26, 2020) to resolve some cuBLASLt issues. Another option is to use the new TacticSource API and disable cuBLASLt tactics if you dont want to upgrade.
|
||||
|
||||
### Libtorch
|
||||
- Error: `libtorch/share/cmake/Caffe2/Caffe2Config.cmake:96 (message):Your installed Caffe2 version uses cuDNN but I cannot find the cuDNN libraries. Please set the proper cuDNN prefixes and / or install cuDNN.`
|
||||
|
||||
May `export CUDNN_ROOT=/root/path/to/cudnn` to resolve the build error.
|
||||
|
|
|
|||
|
|
@ -1,220 +1,40 @@
|
|||
## 安装 MMdeploy
|
||||
|
||||
我们提供物理机和虚拟机构建方法。虚拟机搭建方法请参考[如何使用docker](tutorials/how_to_use_docker.md)。对于物理机，请按照以下步骤操作
|
||||
|
||||
### 准备工作
|
||||
|
||||
- 下载代码仓库 MMDeploy
|
||||
|
||||
```bash
|
||||
git clone -b master git@github.com:open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
export MMDEPLOY_DIR=$(pwd)
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
提示:
|
||||
|
||||
- 如果由于网络等原因导致拉取仓库子模块失败，可以尝试通过如下指令手动再次安装子模块:
|
||||
|
||||
```bash
|
||||
git clone git@github.com:NVIDIA/cub.git third_party/cub
|
||||
cd third_party/cub
|
||||
git checkout c3cceac115
|
||||
|
||||
# 返回至 third_party 目录, 克隆 pybind11
|
||||
cd ..
|
||||
git clone git@github.com:pybind/pybind11.git pybind11
|
||||
cd pybind11
|
||||
git checkout 70a58c5
|
||||
```
|
||||
|
||||
- 安装编译工具 cmake
|
||||
|
||||
要求 cmake>=3.14.0, 通过如下指令安装 cmake。您也通过 [cmake](https://cmake.org/install) 官网查看更多安装信息。
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y libssl-dev
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0.tar.gz
|
||||
tar -zxvf cmake-3.20.0.tar.gz
|
||||
cd cmake-3.20.0
|
||||
./bootstrap
|
||||
make
|
||||
make install
|
||||
```
|
||||
|
||||
- 安装 GCC 7+
|
||||
|
||||
MMDeploy SDK 使用了 C++17 特性，因此需要安装gcc 7+以上的版本。
|
||||
### 下载代码仓库 MMDeploy
|
||||
|
||||
```bash
|
||||
# 如果 Ubuntu 版本 < 18.04，需要加入仓库
|
||||
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
|
||||
# 安装
|
||||
sudo apt-get install gcc-7
|
||||
sudo apt-get install g++-7
|
||||
```
|
||||
git clone -b master git@github.com:open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
### 创建环境
|
||||
提示:
|
||||
|
||||
- 通过 conda 创建并激活 Python 环境
|
||||
- 如果由于网络等原因导致拉取仓库子模块失败，可以尝试通过如下指令手动再次安装子模块:
|
||||
|
||||
```bash
|
||||
conda create -n mmdeploy python=3.7 -y
|
||||
conda activate mmdeploy
|
||||
git clone git@github.com:NVIDIA/cub.git third_party/cub
|
||||
cd third_party/cub
|
||||
git checkout c3cceac115
|
||||
|
||||
# 返回至 third_party 目录, 克隆 pybind11
|
||||
cd ..
|
||||
git clone git@github.com:pybind/pybind11.git pybind11
|
||||
cd pybind11
|
||||
git checkout 70a58c5
|
||||
```
|
||||
|
||||
- 安装 PyTorch，要求版本是 torch>=1.8.0, 可查看[官网](https://pytorch.org/)获取更详细的安装教程。
|
||||
|
||||
```bash
|
||||
# CUDA 11.1
|
||||
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 -c pytorch -c conda-forge
|
||||
```
|
||||
|
||||
- 安装 mmcv-full, 更多安装方式可查看[教程](https://github.com/open-mmlab/mmcv#installation)
|
||||
|
||||
```bash
|
||||
export cu_version=cu111 # cuda 11.1
|
||||
export torch_version=torch1.8
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/${cu_version}/${torch_version}/index.html
|
||||
```
|
||||
|
||||
### 安装推理引擎
|
||||
|
||||
您可以根据自身需求，构建和安装如下推理引擎：
|
||||
|
||||
- [ONNX Runtime](https://mmdeploy.readthedocs.io/en/latest/backends/onnxruntime.html)
|
||||
- [TensorRT](https://mmdeploy.readthedocs.io/en/latest/backends/tensorrt.html)
|
||||
- [ncnn](https://mmdeploy.readthedocs.io/en/latest/backends/ncnn.html)
|
||||
- [PPLNN](https://mmdeploy.readthedocs.io/en/latest/backends/pplnn.html)
|
||||
- [OpenVINO](https://mmdeploy.readthedocs.io/en/latest/backends/openvino.html)
|
||||
|
||||
### 安装 MMDeploy
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR} # 切换至项目根目录
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
**Note**
|
||||
|
||||
- 有些依赖项是可选的。运行 `pip install -e .` 将进行最小化依赖安装。 如果需安装其他可选依赖项，请执行`pip install -r requirements/optional.txt`，
|
||||
或者 `pip install -e . [optional]`。其中，`[optional]`可以替换为：`all`、`tests`、`build` 或 `optional`。
|
||||
|
||||
### 构建 SDK
|
||||
|
||||
如果您只对模型转换感兴趣，那么可以跳过本章节
|
||||
|
||||
#### 安装依赖项
|
||||
|
||||
目前，SDK在Linux-x86_64经过测试验证，未来将加入对更多平台的支持。 使用SDK，需要安装若干依赖包。本章节以 Ubuntu 18.04 为例，逐一介绍各依赖项的安装方法。
|
||||
|
||||
- OpenCV 3+
|
||||
- 如果以 `SSH` 方式 `git clone` 代码失败，您可以尝试使用 `HTTPS` 协议下载代码：
|
||||
|
||||
```bash
|
||||
sudo apt-get install libopencv-dev
|
||||
git clone -b master https://github.com/open-mmlab/mmdeploy.git MMDeploy
|
||||
cd MMDeploy
|
||||
git submodule update --init --recursive
|
||||
```
|
||||
|
||||
- spdlog 0.16+
|
||||
|
||||
``` bash
|
||||
sudo apt-get install libspdlog-dev
|
||||
```
|
||||
|
||||
如果使用 Ubuntu 16.04, 请用如下命令下载并安装合适的spdlog版本
|
||||
|
||||
```bash
|
||||
wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
|
||||
sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
|
||||
```
|
||||
|
||||
您也可以使用spdlog的源码进行编译，激活它更多的特性。但是，请务必打开 **`-fPIC`** 编译选项。
|
||||
|
||||
- pplcv
|
||||
|
||||
pplcv 是在x86和cuda平台下的高性能图像处理库。
|
||||
此依赖项为可选项，只有在cuda平台下，才需安装。安装命令如下所示:
|
||||
|
||||
```bash
|
||||
wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip
|
||||
unzip v0.6.1.zip && mv ppl.cv-0.6.1 ppl.cv
|
||||
cd ppl.cv
|
||||
./build.sh cuda
|
||||
```
|
||||
|
||||
- 推理引擎
|
||||
SDK 和 model converter 使用相同的推理引擎。 请参考前文中[安装推理引擎](#安装推理引擎)章节，以此来选择合适的引擎进行安装.
|
||||
|
||||
#### 设置编译选项
|
||||
|
||||
- 打开 SDK 编译开关
|
||||
|
||||
`-DMMDEPLOY_BUILD_SDK=ON`
|
||||
|
||||
- 设置目标设备
|
||||
|
||||
cpu 是 SDK 目标设备的默认选项。您也可以通过`MMDEPLOY_TARGET_DEVICES`传入其他设备名称。当有多个设备时，设备名称之间使用分号隔开。
|
||||
比如，`-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"`。
|
||||
当前，SDK支持以下设备：
|
||||
|
||||
| 设备 | 名称 | 查找路径 |
|
||||
| :--- | :--- | :--------------------------------- |
|
||||
| Host | cpu | N/A |
|
||||
| CUDA | cuda | CUDA_TOOLKIT_ROOT_DIR 和 pplcv_DIR |
|
||||
|
||||
如果您的开发环境中有多个cuda版本，则需要通过`-DCUDA_TOOLKIT_ROOT_DIR=/path/of/cuda`来明确使用的版本。
|
||||
于此同时，还需设置`-Dpplcv_DIR=ppl.cv/path/install/lib/cmake/ppl`，用以编译cuda平台下的图像处理算子。
|
||||
|
||||
- 设置推理后端
|
||||
|
||||
**默认情况下，SDK不设置任何后端**, 因为它与应用场景高度相关。您可以通过设置`MMDEPLOY_TARGET_BACKENDS`激活感兴趣的推理后端。
|
||||
当选择多个时， 中间使用分号隔开。比如：`-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"`
|
||||
构建时，几乎每个后端，都需设置一些环境变量，用来查找依赖包。
|
||||
下表展示了目前SDK支持的后端，以及构建时，每个后端需要设置的变量：
|
||||
|
||||
| 推理引擎 | 名称 | 查找路径 |
|
||||
| :---------- | :------- | :----------------------- |
|
||||
| PPL.nn | pplnn | pplnn_DIR |
|
||||
| ncnn | ncnn | ncnn_DIR |
|
||||
| ONNXRuntime | ort | ONNXRUNTIME_DIR |
|
||||
| TensorRT | trt | TENSORRT_DIR & CUDNN_DIR |
|
||||
| OpenVINO | openvino | InferenceEngine_DIR |
|
||||
|
||||
- 设置后处理组件
|
||||
|
||||
需要通过`MMDEPLOY_CODEBASES`设置SDK后处理组件，才能加载OpenMMLab算法仓库的后处理功能。已支持的算法仓库有'mmcls'，'mmdet'，'mmedit'，'mmseg'和'mmocr'。
|
||||
如果选择多个codebase，中间使用分号隔开。比如，`-DMMDEPLOY_CODEBASES=mmcls;mmdet`。也可以通过`-DMMDEPLOY_CODEBASES=all`方式，加载所有codebase。
|
||||
|
||||
- 汇总以上
|
||||
|
||||
下文展示2个构建SDK的样例，分别用于不同的运行环境。
|
||||
|
||||
- 使用cpu设备和ONNXRuntime推理，请参考
|
||||
|
||||
```Bash
|
||||
mkdir build && cd build
|
||||
cmake .. \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DONNXRUNTIME_DIR=/path/to/onnxruntime \
|
||||
-DMMDEPLOY_TARGET_DEVICES=cpu \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=ort \
|
||||
-DMMDEPLOY_CODEBASES=all
|
||||
cmake --build . -- -j$(nproc) && cmake --install .
|
||||
```
|
||||
|
||||
- 使用cuda设备和TensorRT推理，请按照此例构建
|
||||
|
||||
```Bash
|
||||
mkdir build && cd build
|
||||
cmake .. \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-Dpplcv_DIR=/path/to/ppl.cv/cuda-build/install/lib/cmake/ppl \
|
||||
-DTENSORRT_DIR=/path/to/tensorrt \
|
||||
-DCUDNN_DIR=/path/to/cudnn \
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=trt \
|
||||
-DMMDEPLOY_CODEBASES=all
|
||||
cmake --build . -- -j$(nproc) && cmake --install .
|
||||
```
|
||||
### 编译 MMDeploy
|
||||
根据您的目标平台，点击如下对应的链接，按照说明编译 MMDeploy
|
||||
- [Linux-x86_64](build/linux.md)
|
||||
- [Windows](build/windows.md)
|
||||
- [Android-aarch64](build/android.md)
|
||||
- [NVIDIA Jetson](../en/tutorials/how_to_install_mmdeploy_on_jetsons.md)
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
# Android 下构建方式
|
||||
|
||||
- [Android 下构建方式](#Android 下构建方式)
|
||||
- [Android 下构建方式](#android-下构建方式)
|
||||
- [源码安装](#源码安装)
|
||||
- [安装构建和编译工具链](#安装构建和编译工具链)
|
||||
- [安装依赖包](#安装依赖包)
|
||||
- [安装 MMDeploy SDK 依赖](#安装-MMDeploy-SDK-依赖)
|
||||
- [安装 MMDeploy SDK 依赖](#安装-mmdeploy-sdk-依赖)
|
||||
- [编译 MMDeploy](#编译-mmdeploy)
|
||||
- [编译选项说明](#编译选项说明)
|
||||
- [编译 SDK](#编译-sdk)
|
||||
|
|
@ -32,13 +32,9 @@ MMDeploy 的交叉编译分为两步:
|
|||
**保证 cmake的版本 >= 3.14.0**. 如果不是,可以参考以下命令安装 3.20.0 版本. 如要获取其他版本,请参考 [这里](https://cmake.org/install)
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y libssl-dev
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0.tar.gz
|
||||
tar -zxvf cmake-3.20.0.tar.gz
|
||||
cd cmake-3.20.0
|
||||
./bootstrap
|
||||
make
|
||||
sudo make install
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
|
||||
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
|
||||
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/
|
||||
```
|
||||
|
||||
- ANDROID NDK 19+
|
||||
|
|
@ -201,5 +197,5 @@ cmake .. \
|
|||
-DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake \
|
||||
-DANDROID_ABI=arm64-v8a \
|
||||
-DANDROID_PLATFORM=android-30
|
||||
cmake --build . -- -j$(nproc)
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1 +1,442 @@
|
|||
TODO
|
||||
# Linux-x86_64 下构建方式
|
||||
|
||||
- [Linux-x86_64 下构建方式](#linux-x86_64-下构建方式)
|
||||
- [Dockerfile 方式 (推荐)](#dockerfile-方式-推荐)
|
||||
- [源码安装](#源码安装)
|
||||
- [安装构建和编译工具链](#安装构建和编译工具链)
|
||||
- [安装依赖包](#安装依赖包)
|
||||
- [安装 MMDeploy Converter 依赖](#安装-mmdeploy-converter-依赖)
|
||||
- [安装 MMDeploy SDK 依赖](#安装-mmdeploy-sdk-依赖)
|
||||
- [安装推理引擎](#安装推理引擎)
|
||||
- [编译 MMDeploy](#编译-mmdeploy)
|
||||
- [编译选项说明](#编译选项说明)
|
||||
- [编译安装 Model Converter](#编译安装-model-converter)
|
||||
- [编译自定义算子](#编译自定义算子)
|
||||
- [安装 Model Converter](#安装-model-converter)
|
||||
- [编译SDK](#编译sdk)
|
||||
- [编译 Demo](#编译-demo)
|
||||
---
|
||||
MMDeploy 为 Linux-x86_64 平台提供 2 种编译安装方式，分别是 Dockerfile 方式和源码方式
|
||||
|
||||
## Dockerfile 方式 (推荐)
|
||||
请参考[how to use docker](../tutorials/how_to_use_docker.md)
|
||||
|
||||
## 源码安装
|
||||
|
||||
### 安装构建和编译工具链
|
||||
|
||||
- cmake
|
||||
|
||||
**保证 cmake的版本 >= 3.14.0**。如果不是，可以参考以下命令安装 3.20.0 版本。如需获取其他版本，请参考[这里](https://cmake.org/install)。
|
||||
|
||||
```bash
|
||||
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
|
||||
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
|
||||
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/
|
||||
```
|
||||
|
||||
- GCC 7+
|
||||
|
||||
MMDeploy SDK 使用了 C++17 特性，因此需要安装gcc 7+以上的版本。
|
||||
```bash
|
||||
# 如果 Ubuntu 版本 < 18.04，需要加入仓库
|
||||
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
|
||||
sudo apt-get update
|
||||
sudo apt-get install gcc-7
|
||||
sudo apt-get install g++-7
|
||||
```
|
||||
|
||||
### 安装依赖包
|
||||
#### 安装 MMDeploy Converter 依赖
|
||||
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称 </th>
|
||||
<th>安装说明 </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>conda </td>
|
||||
<td>请参考<a href="https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html">官方说明</a>安装 conda。<br> 通过 conda 创建并激活 Python 环境。<br>
|
||||
<pre><code>
|
||||
conda create -n mmdeploy python=3.7 -y
|
||||
conda activate mmdeploy
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PyTorch <br>(>=1.8.0) </td>
|
||||
<td>安装 PyTorch，要求版本是 torch>=1.8.0。可查看<a href="https://pytorch.org/">官网</a>获取更多详细的安装教程。请确保 PyTorch 要求的 CUDA 版本和您主机的 CUDA 版本是一致<br>
|
||||
<pre><code>
|
||||
conda install pytorch==1.8.0 torchvision==0.9.0 cudatoolkit=11.1 -c pytorch -c conda-forge
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>mmcv-full </td>
|
||||
<td>参考如下命令安装 mmcv-full。更多安装方式，可查看 <a href="https://github.com/open-mmlab/mmcv">mmcv 官网</a><br>
|
||||
<pre><code>
|
||||
export cu_version=cu111 # cuda 11.1
|
||||
export torch_version=torch1.8
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/${cu_version}/${torch_version}/index.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### 安装 MMDeploy SDK 依赖
|
||||
|
||||
如果您只对模型转换感兴趣，那么可以跳过本章节。
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称 </th>
|
||||
<th>安装说明 </th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>spdlog </td>
|
||||
<td>
|
||||
在 Ubuntu 18.04 及以上版本
|
||||
<pre><code>
|
||||
sudo apt-get install libspdlog-dev
|
||||
</code></pre>
|
||||
在 Ubuntu 16.04,
|
||||
<pre><code>
|
||||
wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
|
||||
sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
|
||||
</code></pre>
|
||||
您也可以使用spdlog的源码进行编译，激活它更多的特性。但是，请务必打开 <code>-DCMAKE_POSITION_INDEPENDENT_CODE=ON</code>.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenCV<br>(>=3.0) </td>
|
||||
<td>
|
||||
在 Ubuntu 18.04 及以上版本
|
||||
<pre><code>
|
||||
sudo apt-get install libopencv-dev
|
||||
</code></pre>
|
||||
在 Ubuntu 16.04 中，需要源码安装 OpenCV。您可以参考<a href="https://docs.opencv.org/3.4/d7/d9f/tutorial_linux_install.html">此处</a>.
|
||||
</td>
|
||||
|
||||
</tr>
|
||||
<tr>
|
||||
<td>pplcv </td>
|
||||
<td>pplcv 是 openPPL 开发的高性能图像处理库。 <b>此依赖项为可选项，只有在 cuda 平台下，才需安装。而且，目前必须使用 v0.6.2，且需要使用 git clone 的方式下载源码并编译安装</b><br>
|
||||
<pre><code>
|
||||
git clone https://github.com/openppl-public/ppl.cv.git
|
||||
cd ppl.cv
|
||||
export PPLCV_DIR=$(pwd)
|
||||
git checkout tags/v0.6.2 -b v0.6.2
|
||||
./build.sh cuda
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### 安装推理引擎
|
||||
|
||||
MMDeploy 的 Model Converter 和 SDK 共享推理引擎。您可以参考下文，选择自己感兴趣的推理引擎安装。
|
||||
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>安装包</th>
|
||||
<th>安装说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>ONNXRuntime</td>
|
||||
<td>onnxruntime<br>(>=1.8.1) </td>
|
||||
<td>
|
||||
1. 安装 onnxruntime 的 python 包
|
||||
<pre><code>pip install onnxruntime==1.8.1</code></pre>
|
||||
2. 从<a href="https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1">这里</a>下载 onnxruntime 的预编译包。参考如下命令，解压压缩包并设置环境变量
|
||||
<pre><code>
|
||||
wget https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-linux-x64-1.8.1.tgz
|
||||
tar -zxvf onnxruntime-linux-x64-1.8.1.tgz
|
||||
cd onnxruntime-linux-x64-1.8.1
|
||||
export ONNXRUNTIME_DIR=$(pwd)
|
||||
export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">TensorRT<br> </td>
|
||||
<td>TensorRT <br> </td>
|
||||
<td>
|
||||
1. 登录 <a href="https://www.nvidia.com/">NVIDIA 官网</a>，从<a href="https://developer.nvidia.com/nvidia-tensorrt-download">这里</a>选取并下载 TensorRT tar 包。要保证它和您机器的 CPU 架构以及 CUDA 版本是匹配的。<br>
|
||||
您可以参考这份<a href="https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar">指南</a>安装 TensorRT。<br>
|
||||
2. 这里也有一份 TensorRT 8.2 GA Update 2 在 Linux x86_64 和 CUDA 11.x 下的安装示例，供您参考。首先，点击<a href="https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.2.3.0/tars/tensorrt-8.2.3.0.linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz">此处</a>下载 CUDA 11.x TensorRT 8.2.3.0。然后，根据如下命令，安装并配置 TensorRT。
|
||||
<pre><code>
|
||||
cd /the/path/of/tensorrt/tar/gz/file
|
||||
tar -zxvf TensorRT-8.2.3.0.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
|
||||
pip install TensorRT-8.2.3.0/python/tensorrt-8.2.3.0-cp37-none-linux_x86_64.whl
|
||||
export TENSORRT_DIR=$(pwd)/TensorRT-8.2.3.0
|
||||
export LD_LIBRARY_PATH=$TENSORRT_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cuDNN </td>
|
||||
<td>
|
||||
1. 从 <a href="https://developer.nvidia.com/rdp/cudnn-archive">cuDNN Archive</a> 选择和您环境中 CPU 架构、CUDA 版本以及 TensorRT 版本配套的 cuDNN。以前文 TensorRT 安装说明为例，它需要 cudnn8.2。因此，可以下载 <a href="https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.1.32/11.3_06072021/cudnn-11.3-linux-x64-v8.2.1.32.tgz">CUDA 11.x cuDNN 8.2</a><br>
|
||||
2. 解压压缩包，并设置环境变量
|
||||
<pre><code>
|
||||
cd /the/path/of/cudnn/tgz/file
|
||||
tar -zxvf cudnn-11.3-linux-x64-v8.2.1.32.tgz
|
||||
export CUDNN_DIR=$(pwd)/cuda
|
||||
export LD_LIBRARY_PATH=$CUDNN_DIR/lib64:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PPL.NN</td>
|
||||
<td>ppl.nn </td>
|
||||
<td>
|
||||
1. 请参考 ppl.nn 的 <a href="https://github.com/openppl-public/ppl.nn/blob/master/docs/en/building-from-source.md">安装文档</a> 编译 ppl.nn，并安装 pyppl<br>
|
||||
2. 将 pplnn 的根目录写入环境变量
|
||||
<pre><code>
|
||||
cd ppl.nn
|
||||
export PPLNN_DIR=$(pwd)
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenVINO</td>
|
||||
<td>openvino </td>
|
||||
<td>1. 安装 <a href="https://docs.openvino.ai/2021.4/get_started.html">OpenVINO</a>
|
||||
<pre><code>
|
||||
pip install openvino-dev
|
||||
</code></pre>
|
||||
2. <b>可选</b>. 如果您想在 MMDeploy SDK 中使用 OpenVINO，请根据<a href="https://docs.openvino.ai/2021.4/openvino_docs_install_guides_installing_openvino_linux.html#install-openvino">指南</a>安装并配置它
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ncnn </td>
|
||||
<td>ncnn </td>
|
||||
<td>1. 请参考 ncnn的 <a href="https://github.com/Tencent/ncnn/wiki/how-to-build">wiki</a> 编译 ncnn。
|
||||
编译时，请打开<code>-DNCNN_PYTHON=ON</code><br>
|
||||
2. 将 ncnn 的根目录写入环境变量
|
||||
<pre><code>
|
||||
cd ncnn
|
||||
export NCNN_DIR=$(pwd)
|
||||
</code></pre>
|
||||
3. 安装 pyncnn
|
||||
<pre><code>
|
||||
cd ${NCNN_DIR}/python
|
||||
pip install -e .
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>TorchScript</td>
|
||||
<td>libtorch</td>
|
||||
<td>
|
||||
1. Download libtorch from <a href="https://pytorch.org/get-started/locally/">here</a>. Please note that only <b>Pre-cxx11 ABI</b> and <b>version 1.8.1+</b> on Linux platform are supported by now. For previous versions of libtorch, you can find them in the <a href="https://github.com/pytorch/pytorch/issues/40961#issuecomment-1017317786">issue comment</a>. <br>
|
||||
2. Take Libtorch1.8.1+cu111 as an example. You can install it like this:
|
||||
<pre><code>
|
||||
wget https://download.pytorch.org/libtorch/cu111/libtorch-shared-with-deps-1.8.1%2Bcu111.zip
|
||||
unzip libtorch-shared-with-deps-1.8.1+cu111.zip
|
||||
cd libtorch
|
||||
export Torch_DIR=$(pwd)
|
||||
export LD_LIBRARY_PATH=$Torch_DIR/lib:$LD_LIBRARY_PATH
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
注意: <br>
|
||||
如果您想使上述环境变量永久有效，可以把它们加入<code>~/.bashrc</code>。以 ONNXRuntime 的环境变量为例，
|
||||
```bash
|
||||
echo '# set env for onnxruntime' >> ~/.bashrc
|
||||
echo "export ONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}" >> ~/.bashrc
|
||||
echo 'export LD_LIBRARY_PATH=$ONNXRUNTIME_DIR/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
```
|
||||
### 编译 MMDeploy
|
||||
|
||||
```bash
|
||||
cd /the/root/path/of/MMDeploy
|
||||
export MMDEPLOY_DIR=$(pwd)
|
||||
```
|
||||
|
||||
#### 编译选项说明
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>编译选项</th>
|
||||
<th>取值范围</th>
|
||||
<th>缺省值</th>
|
||||
<th>说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>MMDeploy SDK 编译开关</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_SDK_PYTHON_API</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>MMDeploy SDK python package的编译开关</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_BUILD_TEST</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>MMDeploy SDK 的单元测试程序编译开关</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_DEVICES</td>
|
||||
<td>{"cpu", "cuda"}</td>
|
||||
<td>cpu</td>
|
||||
<td>设置目标设备。当有多个设备时，设备名称之间使用分号隔开。 比如，-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_BACKENDS</td>
|
||||
<td>{"trt", "ort", "pplnn", "ncnn", "openvino", "torchscript"}</td>
|
||||
<td>N/A</td>
|
||||
<td> <b>默认情况下，SDK不设置任何后端</b>, 因为它与应用场景高度相关。 当选择多个后端时， 中间使用分号隔开。比如，<pre><code>-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"</code></pre>
|
||||
构建时，几乎每个后端，都需传入一些路径变量，用来查找依赖包。<br>
|
||||
1. <b>trt</b>: 表示 TensorRT。需要设置 <code>TENSORRT_DIR</code> 和 <code>CUDNN_DIR</code>。
|
||||
<pre><code>
|
||||
-DTENSORRT_DIR=${TENSORRT_DIR}
|
||||
-DCUDNN_DIR=${CUDNN_DIR}
|
||||
</code></pre>
|
||||
2. <b>ort</b>: 表示 ONNXRuntime。需要设置 <code>ONNXRUNTIME_DIR</code>
|
||||
<pre><code>-DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}</code></pre>
|
||||
3. <b>pplnn</b>: 表示 PPL.NN。需要设置 <code>pplnn_DIR</code>
|
||||
<pre><code>-Dpplnn_DIR=${PPLNN_DIR}</code></pre>
|
||||
4. <b>ncnn</b>: 表示 ncnn。需要设置 <code>ncnn_DIR</code>
|
||||
<pre><code>-Dncnn_DIR=${NCNN_DIR}</code></pre>
|
||||
5. <b>openvino</b>: 表示 OpenVINO。需要设置 <code>InferenceEngine_DIR</code>
|
||||
<pre><code>-DInferenceEngine_DIR=${INTEL_OPENVINO_DIR}/deployment_tools/inference_engine/share</code></pre>
|
||||
6. <b>torchscript</b>: TorchScript. 需要设置<code>Torch_DIR</code>
|
||||
<pre><code>-DTorch_DIR=${Torch_DIR}</code></pre>
|
||||
目前，<b>模型转换支持 torchscript，但 SDK 尚不支持 </b>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_CODEBASES</td>
|
||||
<td>{"mmcls", "mmdet", "mmseg", "mmedit", "mmocr", "all"}</td>
|
||||
<td>all</td>
|
||||
<td>用来设置 SDK 后处理组件，加载 OpenMMLab 算法仓库的后处理功能。如果选择多个 codebase，中间使用分号隔开。比如，<code>-DMMDEPLOY_CODEBASES="mmcls;mmdet"</code>。也可以通过 <code>-DMMDEPLOY_CODEBASES=all</code> 方式，加载所有 codebase。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BUILD_SHARED_LIBS</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>ON</td>
|
||||
<td>MMDeploy SDK 的动态库的编译开关。设置 OFF 时，编译静态库</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
#### 编译安装 Model Converter
|
||||
|
||||
##### 编译自定义算子
|
||||
如果您选择了ONNXRuntime，TensorRT，ncnn 和 torchscript 任一种推理后端，您需要编译对应的自定义算子库。
|
||||
|
||||
- **ONNXRuntime** 自定义算子
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ort -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **TensorRT** 自定义算子
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${TENSORRT_DIR} -DCUDNN_DIR=${CUDNN_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **ncnn** 自定义算子
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=ncnn -Dncnn_DIR=${NCNN_DIR}/build/install/lib/cmake/ncnn ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
- **torchscript** 自定义算子
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake -DMMDEPLOY_TARGET_BACKENDS=torchscript -DTorch_DIR=${Torch_DIR} ..
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
##### 安装 Model Converter
|
||||
|
||||
```bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
pip install -e .
|
||||
```
|
||||
**注意**
|
||||
|
||||
- 有些依赖项是可选的。运行 `pip install -e .` 将进行最小化依赖安装。 如果需安装其他可选依赖项，请执行`pip install -r requirements/optional.txt`，
|
||||
或者 `pip install -e .[optional]`。其中，`[optional]`可以替换为：`all`、`tests`、`build` 或 `optional`。
|
||||
#### 编译SDK
|
||||
|
||||
下文展示2个构建SDK的样例，分别用 ONNXRuntime 和 TensorRT 作为推理引擎。您可以参考它们，激活其他的推理引擎。
|
||||
|
||||
- cpu + ONNXRuntime
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake .. \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
|
||||
-DMMDEPLOY_TARGET_DEVICES=cpu \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=ort \
|
||||
-DMMDEPLOY_CODEBASES=all \
|
||||
-DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR}
|
||||
|
||||
make -j$(nproc) && make install
|
||||
```
|
||||
|
||||
- cuda + TensorRT
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}
|
||||
mkdir -p build && cd build
|
||||
cmake .. \
|
||||
-DCMAKE_CXX_COMPILER=g++-7 \
|
||||
-DMMDEPLOY_BUILD_SDK=ON \
|
||||
-DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
|
||||
-DMMDEPLOY_TARGET_BACKENDS=trt \
|
||||
-DMMDEPLOY_CODEBASES=all \
|
||||
-Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
|
||||
-DTENSORRT_DIR=${TENSORRT_DIR} \
|
||||
-DCUDNN_DIR=${CUDNN_DIR}
|
||||
|
||||
make -j$(nproc) && make -install
|
||||
```
|
||||
|
||||
#### 编译 Demo
|
||||
|
||||
```Bash
|
||||
cd ${MMDEPLOY_DIR}/build/install/example
|
||||
mkdir -p build && cd build
|
||||
cmake .. -DMMDeploy_DIR=${MMDEPLOY_DIR}/build/install/lib/cmake/MMDeploy
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
# Windows 下构建方式
|
||||
|
||||
- [Windows 下构建方式](#windows-下构建方式)
|
||||
- [源码安装](#源码安装)
|
||||
- [安装构建和编译工具链](#安装构建和编译工具链)
|
||||
|
|
@ -6,30 +8,27 @@
|
|||
- [安装 MMDeploy SDK 依赖](#安装-mmdeploy-sdk-依赖)
|
||||
- [安装推理引擎](#安装推理引擎)
|
||||
- [编译 MMDeploy](#编译-mmdeploy)
|
||||
- [编译选项说明](#编译选项说明)
|
||||
- [编译安装 Model Converter](#编译安装-model-converter)
|
||||
- [编译自定义算子](#编译自定义算子)
|
||||
- [安装 Model Converter](#安装-model-converter)
|
||||
- [编译 SDK](#编译-sdk)
|
||||
- [编译选项说明](#编译选项说明)
|
||||
- [编译样例](#编译样例)
|
||||
- [编译 SDK Demo](#编译-sdk-demo)
|
||||
- [编译 Demo](#编译-demo)
|
||||
- [注意事项](#注意事项)
|
||||
|
||||
---
|
||||
# Windows 下构建方式
|
||||
|
||||
目前，MMDeploy 在 Windows 平台下仅提供源码编译安装方式。未来会提供预编译包方式。
|
||||
|
||||
## 源码安装
|
||||
下述安装方式，均是在 **Windows 10** 下进行
|
||||
|
||||
### 安装构建和编译工具链
|
||||
1. 下载并安装 [Visual Studio 2019](https://visualstudio.microsoft.com) 。安装时请勾选 "使用C++的桌面开发, "Windows 10 SDK <br>
|
||||
2. 把 cmake 路径加入到环境变量 PATH 中, "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin" <br>
|
||||
3. 如果系统中配置了 NVIDIA 显卡，根据官网教程，下载并安装 cuda toolkit。<br>
|
||||
### 安装依赖包
|
||||
3. 如果系统中配置了 NVIDIA 显卡，根据[官网教程](https://developer.nvidia.com\/cuda-downloads)，下载并安装 cuda toolkit。<br>
|
||||
|
||||
### 安装依赖包
|
||||
#### 安装 MMDeploy Converter 依赖
|
||||
<table>
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称 </th>
|
||||
|
|
@ -39,25 +38,28 @@
|
|||
<tbody>
|
||||
<tr>
|
||||
<td>conda </td>
|
||||
<td>强烈建议安装conda，或者miniconda。比如， <br>https://repo.anaconda.com/miniconda/Miniconda3-py37_4.11.0-Windows-x86_64.exe <br>安装完毕后，打开系统开始菜单，输入prompt，选择并打开 anaconda powershell prompt。 <br><b>下文中的安装命令均是在 anaconda powershell 中测试验证的。</b> </td>
|
||||
<td>请参考 <a href="https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html">这里</a> 安装 conda。安装完毕后，打开系统开始菜单，<b>以管理员的身份打开 anaconda powershell prompt</b>。 因为，<br>
|
||||
<b>1. 下文中的安装命令均是在 anaconda powershell 中测试验证的。</b><br>
|
||||
<b>2. 使用管理员权限，可以把第三方库安装到系统目录。能够简化 MMDeploy 编译命令。</b><br>
|
||||
<b>说明：如果你对 cmake 工作原理很熟悉，也可以使用普通用户权限打开 anaconda powershell prompt</b>。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>pytorch <br>(>=1.8.0) </td>
|
||||
<td>
|
||||
参考<a href="https://pytorch.org/get-started/locally/">pytorch官网</a>，根据系统环境, 选择合适的预编译包进行安装。比如, <br>
|
||||
<pre><code>
|
||||
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
|
||||
</code></pre>
|
||||
<td>PyTorch <br>(>=1.8.0) </td>
|
||||
<td> 安装 PyTorch，要求版本是 torch>=1.8.0。可查看<a href="https://pytorch.org/">官网</a>获取更多详细的安装教程。请确保 PyTorch 要求的 CUDA 版本和您主机的 CUDA 版本是一致<br>
|
||||
<pre><code>
|
||||
pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>mmcv-full </td>
|
||||
<td>参考<a href="https://github.com/open-mmlab/mmcv">mmcv官网</a>，根据系统环境，选择预编译包进行安装。比如，<br>
|
||||
<pre><code>
|
||||
$env:cu_version="cu111"
|
||||
$env:torch_version="torch1.8.0"
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/$env:cu_version/$env:torch_version/index.html
|
||||
</code></pre>
|
||||
<td>参考如下命令安装 mmcv-full。更多安装方式，可查看 <a href="https://github.com/open-mmlab/mmcv">mmcv 官网</a><br>
|
||||
<pre><code>
|
||||
$env:cu_version="cu111"
|
||||
$env:torch_version="torch1.8"
|
||||
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/$env:cu_version/$env:torch_version/index.html
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
|
|
@ -65,7 +67,8 @@
|
|||
|
||||
|
||||
#### 安装 MMDeploy SDK 依赖
|
||||
<table>
|
||||
如果您只对模型转换感兴趣，那么可以跳过本章节。
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称 </th>
|
||||
|
|
@ -76,43 +79,52 @@
|
|||
<tr>
|
||||
<td>spdlog </td>
|
||||
<td>spdlog是一个精巧的日志管理库。请参考如下命令安装： <br>
|
||||
1. 下载 https://github.com/gabime/spdlog/archive/refs/tags/v1.9.2.zip <br>
|
||||
2. 解压后，进入到文件夹 spdlog-v1.9.2 <br>
|
||||
3. 执行编译安装命令 <br>
|
||||
<pre><code>
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_PREFIX=install -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build . --target install -j --config Release
|
||||
</code></pre>
|
||||
</td>
|
||||
<pre><code>
|
||||
Invoke-WebRequest -Uri https://github.com/gabime/spdlog/archive/refs/tags/v1.9.2.zip -OutFile spdlog-1.9.2.zip
|
||||
Expand-Archive spdlog-1.9.2.zip .
|
||||
cd spdlog-1.9.2
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build . --target install --config Release -- /m
|
||||
cd ../..
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenCV </td>
|
||||
<td>
|
||||
1. 下载并安装 OpenCV 在 windows 下的预编译包: https://github.com/opencv/opencv/releases/download/4.5.5/opencv-4.5.5-vc14_vc15.exe <br>
|
||||
2. 把 OpenCV 库的路径加入到环境变量 PATH 中</td>
|
||||
1. 从<a href="https://github.com/opencv/opencv/releases">这里</a>下载 OpenCV 3+。
|
||||
2. 您可以下载并安装 OpenCV 预编译包到指定的目录下。也可以选择源码编译安装的方式
|
||||
3. 在安装目录中，找到 <code>OpenCVConfig.cmake</code>，并把它的路径添加到环境变量 <code>PATH</code> 中。像这样：</td>
|
||||
<pre><code>$env:path = "\the\path\where\OpenCVConfig.cmake\locates;" + "$env:path"</code></pre>
|
||||
|
||||
</tr>
|
||||
<tr>
|
||||
<td>pplcv </td>
|
||||
<td>pplcv 是在x86和cuda平台下的高性能图像处理库。 <b>此依赖项为可选项，只有在cuda平台下，才需安装。而且，目前必须使用v0.6.2，且需要使用git clone的方式下载源码并编译安装</b><br>
|
||||
<pre><code>
|
||||
git clone --recursive git@github.com:openppl-public/ppl.cv.git
|
||||
cd ppl.cv
|
||||
git checkout tags/v0.6.2 -b v0.6.2
|
||||
./build.bat -G "Visual Studio 16 2019" -T v142 -A x64 -DHPCC_USE_CUDA=ON -DHPCC_MSVC_MD=ON
|
||||
</code></pre>
|
||||
<td>pplcv 是 openPPL 开发的高性能图像处理库。 <b>此依赖项为可选项，只有在 cuda 平台下，才需安装。而且，目前必须使用 v0.6.2，且需要使用 git clone 的方式下载源码并编译安装</b><br>
|
||||
<pre><code>
|
||||
git clone https://github.com/openppl-public/ppl.cv.git
|
||||
cd ppl.cv
|
||||
git checkout tags/v0.6.2 -b v0.6.2
|
||||
$env:PPLCV_DIR = "$pwd"
|
||||
mkdir pplcv-build
|
||||
cd pplcv-build
|
||||
cmake .. -G "Visual Studio 16 2019" -T v142 -A x64 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=install -DHPCC_USE_CUDA=ON -DHPCC_MSVC_MD=ON
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
cd ../..
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
|
||||
#### 安装推理引擎
|
||||
目前，在 Windows 平台下，MMDeploy 支持 ONNXRuntime 和 TensorRT 两种推理引擎。其他推理引擎尚未进行验证，或者验证未通过。后续将陆续予以支持
|
||||
<table>
|
||||
MMDeploy 的 Model Converter 和 SDK 共享推理引擎。您可以参考下文，选择自己感兴趣的推理引擎安装。
|
||||
|
||||
**目前，在 Windows 平台下，MMDeploy 支持 ONNXRuntime 和 TensorRT 两种推理引擎**。其他推理引擎尚未进行验证，或者验证未通过。后续将陆续予以支持
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>推理引擎 </th>
|
||||
|
|
@ -121,43 +133,47 @@
|
|||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<tr>
|
||||
<td>ONNXRuntime</td>
|
||||
<td>onnxruntime </td>
|
||||
<td>onnxruntime<br>(>=1.8.1) </td>
|
||||
<td>
|
||||
1. 下载二进制包：https://github.com/microsoft/onnxruntime/releases/download/v1.8.0/onnxruntime-win-x64-1.8.0.zip <br>
|
||||
2. 解压到目标路径。我们使用 <code>onnxruntime_dir</code> 代表此路径 <br>
|
||||
3. 在 PATH 中增加 onnxruntime libs 路径,
|
||||
<pre><code>
|
||||
$env:path = "{onnxruntime_dir}/lib;" + $env:path
|
||||
</code></pre>
|
||||
1. 安装 onnxruntime 的 python 包
|
||||
<pre><code>pip install onnxruntime==1.8.1</code></pre>
|
||||
2. 从<a href="https://github.com/microsoft/onnxruntime/releases/tag/v1.8.1">这里</a>下载 onnxruntime 的预编译二进制包，解压并配置环境变量
|
||||
<pre><code>
|
||||
Invoke-WebRequest -Uri https://github.com/microsoft/onnxruntime/releases/download/v1.8.1/onnxruntime-win-x64-1.8.1.zip -OutFile onnxruntime-win-x64-1.8.1.zip
|
||||
Expand-Archive onnxruntime-win-x64-1.8.1.zip .
|
||||
$env:ONNXRUNTIME_DIR = "$pwd\onnxruntime-win-x64-1.8.1"
|
||||
$env:path = "$env:ONNXRUNTIME_DIR\lib;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="2">TensorRT<br> </td>
|
||||
<td>TensorRT <br> </td>
|
||||
<td>
|
||||
1. 从NVIDIA官网下载二进制包, 比如，<br>
|
||||
https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.2.3.0/zip/TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip <br>
|
||||
1. 解压二进制包到目标路径。我们使用 <code>tensorrt_dir</code> 代表此路径 <br>
|
||||
2. 安装 tensorrt 的 python package<br>
|
||||
3. 在 PATH 中增加 tensorrt libs 路径
|
||||
<pre><code>
|
||||
pip install {tensorrt_dir}/python/tensorrt-8.2.3.0-cp37-none-win_amd64.whl
|
||||
$env:path = "{tensorrt_dir}/lib;" + $env:path
|
||||
</code></pre>
|
||||
1. 登录 <a href="https://www.nvidia.com/">NVIDIA 官网</a>，从<a href="https://developer.nvidia.com/nvidia-tensorrt-download">这里</a>选取并下载 TensorRT tar 包。要保证它和您机器的 CPU 架构以及 CUDA 版本是匹配的。您可以参考这份 <a href="https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar">指南</a> 安装 TensorRT。<br>
|
||||
2. 这里也有一份 TensorRT 8.2 GA Update 2 在 Windows x86_64 和 CUDA 11.x 下的安装示例，供您参考。首先，点击<a href="https://developer.nvidia.com/compute/machine-learning/tensorrt/secure/8.2.3.0/zip/TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip">此处</a>下载 CUDA 11.x TensorRT 8.2.3.0。然后，根据如下命令，安装并配置 TensorRT。
|
||||
<pre><code>
|
||||
cd \the\path\of\tensorrt\zip\file
|
||||
Expand-Archive TensorRT-8.2.3.0.Windows10.x86_64.cuda-11.4.cudnn8.2.zip .
|
||||
pip install $env:TENSORRT_DIR\python\tensorrt-8.2.3.0-cp37-none-win_amd64.whl
|
||||
$env:TENSORRT_DIR = "$pwd\TensorRT-8.2.3.0"
|
||||
$env:path = "$env:TENSORRT_DIR\lib;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cudnn </td>
|
||||
<td>
|
||||
1. 从NVIDIA官网下载二进制包, 比如, <br>
|
||||
https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.1.32/11.3_06072021/cudnn-11.3-windows-x64-v8.2.1.32.zip <br>
|
||||
1. 解压二进制包到目标路径。我们使用 <code>cudnn_dir</code> 代表此路径 <br>
|
||||
2. 在 PATH 中增加 cudnn libs 路径
|
||||
<pre><code>
|
||||
$env:path = "{cudnn_dir}/bin;" + $env:path
|
||||
</code><pre>
|
||||
1. 从 <a href="https://developer.nvidia.com/rdp/cudnn-archive">cuDNN Archive</a> 中选择和您环境中 CPU 架构、CUDA 版本以及 TensorRT 版本配套的 cuDNN。以前文 TensorRT 安装说明为例，它需要 cudnn8.2。因此，可以下载 <a href="https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.2.1.32/11.3_06072021/cudnn-11.3-windows-x64-v8.2.1.32.zip">CUDA 11.x cuDNN 8.2</a><br>
|
||||
2. 解压压缩包，并设置环境变量
|
||||
<pre><code>
|
||||
cd \the\path\of\cudnn\zip\file
|
||||
Expand-Archive cudnn-11.3-windows-x64-v8.2.1.32.zip .
|
||||
$env:CUDNN_DIR="$pwd\cuda"
|
||||
$env:path = "$env:CUDNN_DIR\bin;" + $env:path
|
||||
</code></pre>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
|
|
@ -179,38 +195,13 @@
|
|||
</table>
|
||||
|
||||
### 编译 MMDeploy
|
||||
|
||||
#### 编译安装 Model Converter
|
||||
##### 编译自定义算子
|
||||
- **ONNXRuntime** 自定义算子
|
||||
```powershell
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="ort" -DONNXRUNTIME_DIR={onnxruntime_dir}
|
||||
cmake --build . --config Release -- /maxcpucount:4
|
||||
cd \the\root\path\of\MMDeploy
|
||||
$env:MMDEPLOY_DIR="$pwd"
|
||||
```
|
||||
|
||||
- **TensorRT** 自定义算子
|
||||
|
||||
```powershell
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="trt" -DTENSORRT_DIR={tensorrt_dir} -DCUDNN_DIR={cudnn_dir}
|
||||
cmake --build . --config Release -- /maxcpucount:4
|
||||
```
|
||||
|
||||
- **ncnn** 自定义算子
|
||||
|
||||
TODO
|
||||
|
||||
##### 安装 Model Converter
|
||||
```powershell
|
||||
cd root/path/of/MMDeploy
|
||||
pip install -e .
|
||||
```
|
||||
#### 编译 SDK
|
||||
##### 编译选项说明
|
||||
<table>
|
||||
#### 编译选项说明
|
||||
<table class="docutils">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>编译选项</th>
|
||||
|
|
@ -236,7 +227,7 @@ pip install -e .
|
|||
<td>MMDEPLOY_BUILD_TEST</td>
|
||||
<td>{ON, OFF}</td>
|
||||
<td>OFF</td>
|
||||
<td>MMDeploy SDK的测试程序编译开关</td>
|
||||
<td>MMDeploy SDK 的单元测试程序编译开关</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_TARGET_DEVICES</td>
|
||||
|
|
@ -249,19 +240,24 @@ pip install -e .
|
|||
<td>{"trt", "ort", "pplnn", "ncnn", "openvino"}</td>
|
||||
<td>N/A</td>
|
||||
<td> <b>默认情况下，SDK不设置任何后端</b>, 因为它与应用场景高度相关。 当选择多个后端时， 中间使用分号隔开。比如，<pre><code>-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"</code></pre>
|
||||
构建时，几乎每个后端，都需设置一些环境变量，用来查找依赖包。<br>
|
||||
1. <b>trt</b>: 表示 TensorRT, 需要设置 TENSORRT_DIR 和 CUDNN_DIR。类似， <pre><code>-DTENSORRT_DIR={tensorrt_dir}<br>-DCUDNN_DIR={cudnn_dir}</code></pre>
|
||||
2. <b>ort</b>: 表示 ONNXRuntime，需要设置 ONNXRUNTIME_DIR。类似， <pre><code>-DONNXRUNTIME_DIR={onnxruntime_dir}</code></pre>
|
||||
3. <b>pplnn</b>: 表示 PPL.NN，需要设置 pplnn_DIR。<b>当前版本尚未验证</b> <br>
|
||||
4. <b>ncnn</b>：需要设置 ncnn_DIR。<b>当前版本尚未验证</b> <br>
|
||||
5. <b>openvino</b>: 表示 OpenVINO，需要设置 InferenceEngine_DIR。<b>当前版本尚未验证通过</b>
|
||||
构建时，几乎每个后端，都需设置一些路径变量，用来查找依赖包。<br>
|
||||
1. <b>trt</b>: 表示 TensorRT。需要设置 <code>TENSORRT_DIR</code> 和 <code>CUDNN_DIR</code>。
|
||||
<pre><code>
|
||||
-DTENSORRT_DIR=$env:TENSORRT_DIR
|
||||
-DCUDNN_DIR=$env:CUDNN_DIR
|
||||
</code></pre>
|
||||
2. <b>ort</b>: 表示 ONNXRuntime。需要设置 <code>ONNXRUNTIME_DIR</code>。
|
||||
<pre><code>-DONNXRUNTIME_DIR=$env:ONNXRUNTIME_DIR</code></pre>
|
||||
3. <b>pplnn</b>: 表示 PPL.NN。需要设置 <code>pplnn_DIR</code>。<b>当前版本尚未验证</b> <br>
|
||||
4. <b>ncnn</b>：表示 ncnn。需要设置 <code>ncnn_DIR</code>。<b>当前版本尚未验证</b> <br>
|
||||
5. <b>openvino</b>: 表示 OpenVINO。需要设置 <code>InferenceEngine_DIR</code>。<b>当前版本尚未验证通过</b>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MMDEPLOY_CODEBASES</td>
|
||||
<td>{"mmcls", "mmdet", "mmseg", "mmedit", "mmocr", "all"}</td>
|
||||
<td>N/A</td>
|
||||
<td>用来设置SDK后处理组件，加载OpenMMLab算法仓库的后处理功能。已支持的算法仓库有'mmcls'，'mmdet'，'mmedit'，'mmseg'和'mmocr'。如果选择多个codebase，中间使用分号隔开。比如，<code>-DMMDEPLOY_CODEBASES="mmcls;mmdet"</code>。也可以通过 <code>-DMMDEPLOY_CODEBASES=all</code> 方式，加载所有codebase。</td>
|
||||
<td>all</td>
|
||||
<td>用来设置SDK后处理组件，加载 OpenMMLab 算法仓库的后处理功能。如果选择多个 codebase，中间使用分号隔开。比如，<code>-DMMDEPLOY_CODEBASES="mmcls;mmdet"</code>。也可以通过 <code>-DMMDEPLOY_CODEBASES=all</code> 方式，加载所有 codebase。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BUILD_SHARED_LIBS</td>
|
||||
|
|
@ -273,64 +269,91 @@ pip install -e .
|
|||
</table>
|
||||
|
||||
|
||||
##### 编译样例
|
||||
#### 编译安装 Model Converter
|
||||
##### 编译自定义算子
|
||||
如果您选择了ONNXRuntime，TensorRT 和 ncnn 任一种推理后端，您需要编译对应的自定义算子库。
|
||||
- **ONNXRuntime** 自定义算子
|
||||
```powershell
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="ort" -DONNXRUNTIME_DIR="$env:ONNXRUNTIME_DIR"
|
||||
cmake --build . --config Release -- /m
|
||||
```
|
||||
|
||||
下文展示2个构建SDK的样例，分别用于不同的运行环境。
|
||||
- **TensorRT** 自定义算子
|
||||
|
||||
```powershell
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 -DMMDEPLOY_TARGET_BACKENDS="trt" -DTENSORRT_DIR="$env:TENSORRT_DIR" -DCUDNN_DIR="$env:CUDNN_DIR"
|
||||
cmake --build . --config Release -- /m
|
||||
```
|
||||
|
||||
- **ncnn** 自定义算子
|
||||
|
||||
TODO
|
||||
|
||||
##### 安装 Model Converter
|
||||
```powershell
|
||||
cd $env:MMDEPLOY_DIR
|
||||
pip install -e .
|
||||
```
|
||||
**注意**
|
||||
- 有些依赖项是可选的。运行 `pip install -e .` 将进行最小化依赖安装。 如果需安装其他可选依赖项，请执行`pip install -r requirements/optional.txt`，
|
||||
或者 `pip install -e .[optional]`。其中，`[optional]`可以替换为：`all`、`tests`、`build` 或 `optional`。
|
||||
#### 编译 SDK
|
||||
|
||||
下文展示2个构建SDK的样例，分别用 ONNXRuntime 和 TensorRT 作为推理引擎。您可以参考它们，并结合前文 SDK 的编译选项说明，激活其他的推理引擎。
|
||||
|
||||
- cpu + ONNXRuntime
|
||||
|
||||
```PowerShell
|
||||
mkdir build
|
||||
cd $env:MMDEPLOY_DIR
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDEPLOY_BUILD_SDK=ON `
|
||||
-DMMDEPLOY_TARGET_DEVICES="cpu" `
|
||||
-DMMDEPLOY_TARGET_BACKENDS="ort" `
|
||||
-DMMDEPLOY_CODEBASES="all" `
|
||||
-DONNXRUNTIME_DIR={onnxruntime_dir} `
|
||||
-Dspdlog_DIR={spdlog_dir}/build/install/lib/cmake/spdlog `
|
||||
-DOpenCV_DIR={opencv_dir}/build
|
||||
cmake --build . --config Release -- /maxcpucount:4
|
||||
-DONNXRUNTIME_DIR="$env:ONNXRUNTIME_DIR"
|
||||
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
```
|
||||
|
||||
- cuda + TensorRT
|
||||
|
||||
```PowerShell
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDEPLOY_BUILD_SDK=ON `
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda" `
|
||||
-DMMDEPLOY_TARGET_BACKENDS="trt" `
|
||||
-DMMDEPLOY_CODEBASES="all" `
|
||||
-Dpplcv_DIR={pplcv_dir}/pplcv-build/install/lib/cmake/ppl `
|
||||
-DTENSORRT_DIR={tensorrt_dir} `
|
||||
-DCUDNN_DIR={cudnn_dir} `
|
||||
-Dspdlog_DIR={spdlog_dir}/build/install/lib/cmake/spdlog `
|
||||
-DOpenCV_DIR={opencv_dir}/build
|
||||
cmake --build . --config Release -- /maxcpucount:4
|
||||
cmake --install . --config Release
|
||||
cd $env:MMDEPLOY_DIR
|
||||
mkdir build
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDEPLOY_BUILD_SDK=ON `
|
||||
-DMMDEPLOY_TARGET_DEVICES="cuda" `
|
||||
-DMMDEPLOY_TARGET_BACKENDS="trt" `
|
||||
-DMMDEPLOY_CODEBASES="all" `
|
||||
-Dpplcv_DIR="$env:PPLCV_DIR/pplcv-build/install/lib/cmake/ppl" `
|
||||
-DTENSORRT_DIR="$env:TENSORRT_DIR" `
|
||||
-DCUDNN_DIR="$env:CUDNN_DIR"
|
||||
|
||||
cmake --build . --config Release -- /m
|
||||
cmake --install . --config Release
|
||||
```
|
||||
- 其他
|
||||
|
||||
请参考上述两个示例，以及前述 SDK 的编译选项，在其他运行环境下编译 SDK
|
||||
|
||||
##### 编译 SDK Demo
|
||||
#### 编译 Demo
|
||||
|
||||
```PowerShell
|
||||
cd install/example
|
||||
mkdir build
|
||||
cd $env:MMDEPLOY_DIR\build\install\example
|
||||
mkdir build -ErrorAction SilentlyContinue
|
||||
cd build
|
||||
cmake .. -G "Visual Studio 16 2019" -A x64 -T v142 `
|
||||
-DMMDeploy_DIR={mmdeploy_dir}/build/install/lib/cmake/MMDeploy `
|
||||
-Dspdlog_DIR={spdlog_dir}/build/install/lib/cmake/spdlog `
|
||||
-DOpenCV_DIR={opencv_dir}/build
|
||||
cmake --build . --config Release -- /maxcpucount:4
|
||||
-DMMDeploy_DIR="$env:MMDEPLOY_DIR/build/install/lib/cmake/MMDeploy"
|
||||
|
||||
$env:path = "${mmdeploy_dir}/build/install/bin;" + $env:path
|
||||
cmake --build . --config Release -- /m
|
||||
|
||||
$env:path = "$env:MMDEPLOY_DIR/build/install/bin;" + $env:path
|
||||
```
|
||||
|
||||
### 注意事项
|
||||
1. Release / Debug 库不能混用。MMDeploy要是编译Debug版本，所有第三方依赖都要是Debug版本。
|
||||
1. Release / Debug 库不能混用。MMDeploy 要是编译 Release 版本，所有第三方依赖都要是 Release 版本。反之亦然。
|
||||
|
|
|
|||
Loading…
Reference in New Issue