Merge branch 'develop' into update_car_doc
weishengyu
commit
6432a7c4e0
deploy
configs
PULC
car_exists
language_classification
person_attribute
safety_helmet
text_image_orientation
textline_orientation
traffic_sign
vehicle_attribute
images/PULC
language_classification
person_attribute
safety_helmet
text_image_orientation
vehicle_attribute
docs
en/inference_deployment
images
algorithm_introduction
zh_CN
advanced_tutorials
image_recognition_pipeline
inference_deployment
models
ppcls
configs
ImageNet/Distillation
PULC
traffic_sign
data/postprocess
|
|
@ -1,7 +1,8 @@
|
|||
include LICENSE.txt
|
||||
include README.md
|
||||
include docs/en/whl_en.md
|
||||
recursive-include deploy/python predict_cls.py preprocess.py postprocess.py det_preprocess.py
|
||||
recursive-include deploy/python *.py
|
||||
recursive-include deploy/configs *.yaml
|
||||
recursive-include deploy/utils get_image_list.py config.py logger.py predictor.py
|
||||
|
||||
recursive-include ppcls/ *.py *.txt
|
||||
160
README_ch.md
160
README_ch.md
|
|
@ -4,39 +4,40 @@
|
|||
|
||||
## 简介
|
||||
|
||||
飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别任务的工具集，助力使用者训练出更好的视觉模型和应用落地。
|
||||
飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别和图像分类任务的工具集，助力使用者训练出更好的视觉模型和应用落地。
|
||||
|
||||
**近期更新**
|
||||
- 🔥️ 2022.5.26 [飞桨产业实践范例直播课](http://aglc.cn/v-c4FAR)，解读**超轻量重点区域人员出入管理方案**，欢迎报名来交流。
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/80816848/170166458-767a01ca-1429-437f-a628-dd184732ef53.png" width = "150" />
|
||||
</div>
|
||||
<div align="center">
|
||||
<img src="./docs/images/class_simple.gif" width = "600" />
|
||||
|
||||
PULC实用图像分类模型效果展示
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/recognition.gif" width = "400" />
|
||||
|
||||
PP-ShiTu图像识别系统效果展示
|
||||
</div>
|
||||
|
||||
|
||||
## 近期更新
|
||||
- 🔥️ 2022.6.15 发布PULC超轻量图像分类方案，CPU推理3ms，精度比肩SwinTransformer，覆盖人、车、OCR场景九大常见任务。
|
||||
- 2022.5.26 [飞桨产业实践范例直播课](http://aglc.cn/v-c4FAR)，解读**超轻量重点区域人员出入管理方案**。
|
||||
- 2022.5.23 新增[人员出入管理范例库](https://aistudio.baidu.com/aistudio/projectdetail/4094475)，具体内容可以在 AI Stuio 上体验。
|
||||
- 2022.5.20 上线[PP-HGNet](./docs/zh_CN/models/PP-HGNet.md), [PP-LCNet v2](./docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- 2022.5.20 上线[PP-HGNet](./docs/zh_CN/models/PP-HGNet.md), [PP-LCNet v2](./docs/zh_CN/models/PP-LCNetV2.md)。
|
||||
- 2022.4.21 新增 CVPR2022 oral论文 [MixFormer](https://arxiv.org/pdf/2204.02557.pdf) 相关[代码](https://github.com/PaddlePaddle/PaddleClas/pull/1820/files)。
|
||||
- 2022.1.27 全面升级文档；新增[PaddleServing C++ pipeline部署方式](./deploy/paddleserving)和[18M图像识别安卓部署Demo](./deploy/lite_shitu)。
|
||||
- 2021.11.1 发布[PP-ShiTu技术报告](https://arxiv.org/pdf/2111.00775.pdf)，新增饮料识别demo
|
||||
- 2021.11.1 发布[PP-ShiTu技术报告](https://arxiv.org/pdf/2111.00775.pdf)，新增饮料识别demo。
|
||||
- 2021.10.23 发布轻量级图像识别系统PP-ShiTu，CPU上0.2s即可完成在10w+库的图像识别。
|
||||
[点击这里](./docs/zh_CN/quick_start/quick_start_recognition.md)立即体验
|
||||
[点击这里](./docs/zh_CN/quick_start/quick_start_recognition.md)立即体验。
|
||||
- 2021.09.17 发布PP-LCNet系列超轻量骨干网络模型, 在Intel CPU上，单张图像预测速度约5ms，ImageNet-1K数据集上Top1识别准确率达到80.82%，超越ResNet152的模型效果。PP-LCNet的介绍可以参考[论文](https://arxiv.org/pdf/2109.15099.pdf), 或者[PP-LCNet模型介绍](docs/zh_CN/models/PP-LCNet.md)，相关指标和预训练权重可以从 [这里](docs/zh_CN/algorithm_introduction/ImageNet_models.md)下载。
|
||||
- [more](./docs/zh_CN/others/update_history.md)
|
||||
|
||||
## 特性
|
||||
|
||||
- PP-ShiTu轻量图像识别系统：集成了目标检测、特征学习、图像检索等模块，广泛适用于各类图像识别任务。cpu上0.2s即可完成在10w+库的图像识别。
|
||||
|
||||
- PP-LCNet轻量级CPU骨干网络：专门为CPU设备打造轻量级骨干网络，速度、精度均远超竞品。
|
||||
|
||||
- 丰富的预训练模型库：提供了36个系列共175个ImageNet预训练模型，其中7个精选系列模型支持结构快速修改。
|
||||
|
||||
- 全面易用的特征学习组件：集成arcmargin, triplet loss等12度量学习方法，通过配置文件即可随意组合切换。
|
||||
|
||||
- SSLD知识蒸馏：14个分类预训练模型，精度普遍提升3%以上；其中ResNet50_vd模型在ImageNet-1k数据集上的Top-1精度达到了84.0%，
|
||||
Res2Net200_vd预训练模型Top-1精度高达85.1%。
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/recognition.gif" width = "400" />
|
||||
</div>
|
||||
支持多种图像分类、识别相关算法，在此基础上打造[PULC超轻量图像分类方案](docs/zh_CN/PULC/PULC_person_exists.md)和[PP-ShiTu图像识别系统](./docs/zh_CN/quick_start/quick_start_recognition.md)。
|
||||
|
||||
|
||||
|
||||
## 欢迎加入技术交流群
|
||||
|
|
@ -50,73 +51,81 @@ Res2Net200_vd预训练模型Top-1精度高达85.1%。
|
|||
|
||||
## 快速体验
|
||||
|
||||
PP-ShiTu图像识别快速体验：[点击这里](./docs/zh_CN/quick_start/quick_start_recognition.md)
|
||||
PULC超轻量图像分类方案快速体验：[点击这里](docs/zh_CN/PULC/PULC_person_exists.md)。
|
||||
|
||||
PP-ShiTu图像识别快速体验：[点击这里](./docs/zh_CN/quick_start/quick_start_recognition.md)。
|
||||
|
||||
## 文档教程
|
||||
- [环境准备]()
|
||||
- [PULC超轻量图像分类实用方案]()
|
||||
- [超轻量图像分类快速体验 (@崔程)]()
|
||||
- [超轻量图像分类模型库]（包含benchmark @崔程）
|
||||
- xx
|
||||
- [方案介绍和模型训练]()
|
||||
- [推理部署](@水龙)
|
||||
- 基于python预测引擎推理
|
||||
- 基于C++预测引擎推理
|
||||
- 服务化部署
|
||||
- 端侧部署
|
||||
- Paddle2ONNX模型转化与预测
|
||||
- [模型压缩](@崔程)
|
||||
- [环境准备](docs/zh_CN/installation/install_paddleclas.md)
|
||||
- PULC超轻量图像分类实用方案 文档更新中
|
||||
- 超轻量图像分类模型库 文档更新中
|
||||
- [PULC有人/无人分类模型](docs/zh_CN/PULC/PULC_person_exists.md)
|
||||
- PULC人体属性识别模型 文档更新中
|
||||
- [PULC佩戴安全帽分类模型](docs/zh_CN/PULC/PULC_safety_helmet.md)
|
||||
- [PULC交通标志分类模型](docs/zh_CN/PULC/PULC_traffic_sign.md)
|
||||
- [PULC车辆属性识别模型](docs/zh_CN/PULC/PULC_vehicle_attribute.md)
|
||||
- PULC有车/无车分类模型 文档更新中
|
||||
- [PULC含文字图像方向分类模型](docs/zh_CN/PULC/PULC_text_image_orientation.md)
|
||||
- [PULC文本行方向分类模型](docs/zh_CN/PULC/PULC_textline_orientation.md)
|
||||
- [PULC语种分类模型](docs/zh_CN/PULC/PULC_language_classification.md)
|
||||
- 模型训练 文档更新中
|
||||
- 推理部署
|
||||
- [基于python预测引擎推理](docs/zh_CN/inference_deployment/python_deploy.md#1)
|
||||
- [基于C++预测引擎推理](docs/zh_CN/inference_deployment/cpp_deploy.md)
|
||||
- [服务化部署](docs/zh_CN/inference_deployment/paddle_serving_deploy.md)
|
||||
- [端侧部署](docs/zh_CN/inference_deployment/paddle_lite_deploy.md)
|
||||
- [Paddle2ONNX模型转化与预测](deploy/paddle2onnx/readme.md)
|
||||
- [模型压缩](deploy/slim/README.md)
|
||||
- [PP-ShiTu图像识别系统介绍](#图像识别系统介绍)
|
||||
- 图像识别快速体验
|
||||
- 模块介绍
|
||||
- [图像识别快速体验](docs/zh_CN/quick_start/quick_start_recognition.md)
|
||||
- 模块介绍
|
||||
- [主体检测](./docs/zh_CN/image_recognition_pipeline/mainbody_detection.md)
|
||||
- [特征提取模型](./docs/zh_CN/image_recognition_pipeline/feature_extraction.md)
|
||||
- [向量检索](./docs/zh_CN/image_recognition_pipeline/vector_search.md)
|
||||
- 哈希编码
|
||||
- 模型训练（包含数据集格式说明等）
|
||||
- 推理部署
|
||||
- 基于python预测引擎推理
|
||||
- 基于C++预测引擎推理
|
||||
- 服务化部署
|
||||
- 端侧部署
|
||||
- Paddle2ONNX模型转化与预测
|
||||
- 模型压缩
|
||||
- 模型量化
|
||||
- 模型裁剪
|
||||
- [骨干网络和预训练模型库](./docs/zh_CN/algorithm_introduction/ImageNet_models.md)
|
||||
- PP系列骨干网络模型（包括算法介绍，使用，训推一体链接等）（@崔程）
|
||||
- PP-HGNet
|
||||
- PP-LCNet v2
|
||||
- PP-LCNet
|
||||
- SSLD半监督知识蒸馏方案 （@若愚）
|
||||
- SSLD算法简介
|
||||
- 预训练模型库
|
||||
- 使用方法（？）
|
||||
- [哈希编码](docs/zh_CN/image_recognition_pipeline/)
|
||||
- [模型训练](docs/zh_CN/models_training/recognition.md)
|
||||
- 推理部署
|
||||
- [基于python预测引擎推理](docs/zh_CN/inference_deployment/python_deploy.md#2)
|
||||
- [基于C++预测引擎推理](deploy/cpp_shitu/readme.md)
|
||||
- [服务化部署](docs/zh_CN/inference_deployment/paddle_serving_deploy.md)
|
||||
- [端侧部署](deploy/lite_shitu/README.md)
|
||||
- 模型压缩 文档更新中
|
||||
- PP系列骨干网络模型
|
||||
- [PP-HGNet](docs/zh_CN/models/PP-HGNet.md)
|
||||
- [PP-LCNet v2](docs/zh_CN/models/PP-LCNetV2.md)
|
||||
- [PP-LCNet](docs/zh_CN/models/PP-LCNet.md)
|
||||
- [SSLD半监督知识蒸馏方案](docs/zh_CN/advanced_tutorials/ssld.md)
|
||||
- 前沿算法
|
||||
- 骨干网络和预训练模型库 （@崔程）
|
||||
- 服务端CNN模型库
|
||||
- 移动端CNN模型库
|
||||
- Vision Transformer模型库
|
||||
- 度量学习（arcmargin等算法）（@水龙）
|
||||
- ReID （@水龙）
|
||||
- 向量检索 （@水龙）
|
||||
- 哈希特征 （@水龙）
|
||||
- 模型蒸馏 （@若愚）
|
||||
- 数据增强 （@崔程）
|
||||
- 产业实用范例库 （@胜禹）
|
||||
- 30分钟快速体验图像分类（原尝鲜版）（@崔程）
|
||||
- [骨干网络和预训练模型库](docs/zh_CN/algorithm_introduction/ImageNet_models.md)
|
||||
- [度量学习](docs/zh_CN/algorithm_introduction/metric_learning.md)
|
||||
- ReID 文档更新中
|
||||
- 向量检索 文档更新中
|
||||
- 哈希特征 文档更新中
|
||||
- [模型蒸馏](docs/zh_CN/algorithm_introduction/model_prune_quantization.md)
|
||||
- [数据增强](docs/zh_CN/advanced_tutorials/DataAugmentation.md)
|
||||
- [产业实用范例库](docs/zh_CN/samples)
|
||||
- [30分钟快速体验图像分类](docs/zh_CN/quick_start/quick_start_classification_new_user.md)
|
||||
- FAQ
|
||||
- [图像识别精选问题](docs/zh_CN/faq_series/faq_2021_s2.md)
|
||||
- [图像分类精选问题](docs/zh_CN/faq_series/faq_selected_30.md)
|
||||
- [图像分类FAQ第一季](docs/zh_CN/faq_series/faq_2020_s1.md)
|
||||
- [图像分类FAQ第二季](docs/zh_CN/faq_series/faq_2021_s1.md)
|
||||
- [PaddleClas结构解析](./docs/zh_CN/advanced_tutorials/code_overview.md)
|
||||
- PaddleClas结构解析 文档更新中
|
||||
- [社区贡献指南](./docs/zh_CN/advanced_tutorials/how_to_contribute.md)
|
||||
- [许可证书](#许可证书)
|
||||
- [贡献代码](#贡献代码)
|
||||
|
||||
|
||||
<a name="PULC超轻量图像分类方案"></a>
|
||||
## PULC超轻量图像分类方案
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/19523330/173011854-b10fcd7a-b799-4dfd-a1cf-9504952a3c44.png" width = "800" />
|
||||
</div>
|
||||
PULC融合了骨干网络、数据增广、蒸馏等多种前沿算法，可以自动训练得到轻量且高精度的图像分类模型。
|
||||
PaddleClas提供了覆盖人、车、OCR场景九大常见任务的分类模型，CPU推理3ms，精度比肩SwinTransformer。
|
||||
|
||||
<a name="图像识别系统介绍"></a>
|
||||
## PP-ShiTu图像识别系统介绍
|
||||
## PP-ShiTu图像识别系统
|
||||
|
||||
<div align="center">
|
||||
<img src="./docs/images/structure.jpg" width = "800" />
|
||||
|
|
@ -124,6 +133,11 @@ PP-ShiTu图像识别快速体验：[点击这里](./docs/zh_CN/quick_start/quick
|
|||
|
||||
PP-ShiTu是一个实用的轻量级通用图像识别系统，主要由主体检测、特征学习和向量检索三个模块组成。该系统从骨干网络选择和调整、损失函数的选择、数据增强、学习率变换策略、正则化参数选择、预训练模型使用以及模型裁剪量化8个方面，采用多种策略，对各个模块的模型进行优化，最终得到在CPU上仅0.2s即可完成10w+库的图像识别的系统。更多细节请参考[PP-ShiTu技术方案](https://arxiv.org/pdf/2111.00775.pdf)。
|
||||
|
||||
<a name="分类效果展示"></a>
|
||||
## PULC实用图像分类模型效果展示
|
||||
<div align="center">
|
||||
<img src="docs/images/classification.gif">
|
||||
</div>
|
||||
|
||||
<a name="识别效果展示"></a>
|
||||
## PP-ShiTu图像识别系统效果展示
|
||||
|
|
|
|||
|
|
@ -0,0 +1,36 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/car_exists/objects365_00001507.jpeg"
|
||||
inference_model_dir: "./models/car_exists_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: False
|
||||
cpu_num_threads: 10
|
||||
enable_benchmark: True
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 0.00392157
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: ThreshOutput
|
||||
ThreshOutput:
|
||||
threshold: 0.5
|
||||
label_0: nocar
|
||||
label_1: contains_car
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/language_classification/word_35404.png"
|
||||
inference_model_dir: "./models/language_classification_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: False
|
||||
cpu_num_threads: 10
|
||||
enable_benchmark: True
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 0.00392157
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: Topk
|
||||
Topk:
|
||||
topk: 2
|
||||
class_id_map_file: "../dataset/language_classification/label_list.txt"
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/person_attribute/090004.jpg"
|
||||
inference_model_dir: "./models/person_attribute_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: True
|
||||
cpu_num_threads: 10
|
||||
benchmark: False
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: PersonAttribute
|
||||
PersonAttribute:
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/safety_helmet/safety_helmet_test_1.png"
|
||||
inference_model_dir: "./models/safety_helmet_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: False
|
||||
cpu_num_threads: 10
|
||||
enable_benchmark: True
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 0.00392157
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: ThreshOutput
|
||||
ThreshOutput:
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/text_image_orientation/img_rot0_demo.jpg"
|
||||
inference_model_dir: "./models/text_image_orientation_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: False
|
||||
cpu_num_threads: 10
|
||||
enable_benchmark: True
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 0.00392157
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: Topk
|
||||
Topk:
|
||||
topk: 2
|
||||
class_id_map_file: "../dataset/text_image_orientation/label_list.txt"
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/textline_orientation/textline_orientation_test_0_0.png"
|
||||
inference_model_dir: "./models/textline_orientation_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: True
|
||||
cpu_num_threads: 10
|
||||
enable_benchmark: True
|
||||
use_fp16: False
|
||||
ir_optim: True
|
||||
use_tensorrt: False
|
||||
gpu_mem: 8000
|
||||
enable_profile: False
|
||||
|
||||
PreProcess:
|
||||
transform_ops:
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 0.00392157
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
channel_num: 3
|
||||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: Topk
|
||||
Topk:
|
||||
topk: 1
|
||||
class_id_map_file: "../ppcls/utils/PULC/textline_orientation_label_list.txt"
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
@ -30,6 +30,6 @@ PostProcess:
|
|||
main_indicator: Topk
|
||||
Topk:
|
||||
topk: 5
|
||||
class_id_map_file: "../dataset/traffic_sign/label_name_id.txt"
|
||||
class_id_map_file: "../ppcls/utils/PULC_label_list/traffic_sign_label_list.txt"
|
||||
SavePreLabel:
|
||||
save_dir: ./pre_label/
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
Global:
|
||||
infer_imgs: "./images/PULC/vehicle_attr/0002_c002_00030670_0.jpg"
|
||||
inference_model_dir: "./models/vehicle_attr_infer"
|
||||
infer_imgs: "./images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg"
|
||||
inference_model_dir: "./models/vehicle_attribute_infer"
|
||||
batch_size: 1
|
||||
use_gpu: True
|
||||
enable_mkldnn: True
|
||||
|
|
@ -25,9 +25,9 @@ PreProcess:
|
|||
- ToCHWImage:
|
||||
|
||||
PostProcess:
|
||||
main_indicator: Attribute
|
||||
Attribute:
|
||||
main_indicator: PersonAttribute
|
||||
PersonAttribute:
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 157 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 178 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 6.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 78 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.4 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.8 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 124 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 92 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 42 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 90 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 34 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 5.2 KiB After Width: | Height: | Size: 5.2 KiB |
|
Before Width: | Height: | Size: 8.0 KiB After Width: | Height: | Size: 8.0 KiB |
|
|
@ -189,7 +189,7 @@ class Binarize(object):
|
|||
return byte
|
||||
|
||||
|
||||
class Attribute(object):
|
||||
class PersonAttribute(object):
|
||||
def __init__(self,
|
||||
threshold=0.5,
|
||||
glasses_threshold=0.3,
|
||||
|
|
@ -277,8 +277,7 @@ class Attribute(object):
|
|||
threshold_list[18] = self.hold_threshold
|
||||
pred_res = (np.array(res) > np.array(threshold_list)
|
||||
).astype(np.int8).tolist()
|
||||
|
||||
batch_res.append([label_res, pred_res])
|
||||
batch_res.append({"attributes": label_res, "output": pred_res})
|
||||
return batch_res
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -138,7 +138,7 @@ def main(config):
|
|||
continue
|
||||
batch_results = cls_predictor.predict(batch_imgs)
|
||||
for number, result_dict in enumerate(batch_results):
|
||||
if "Attribute" in config[
|
||||
if "PersonAttribute" in config[
|
||||
"PostProcess"] or "VehicleAttribute" in config[
|
||||
"PostProcess"]:
|
||||
filename = batch_names[number]
|
||||
|
|
|
|||
|
|
@ -212,14 +212,14 @@ You can save the prediction result(s) as pre-label, only need to use `pre_label_
|
|||
```python
|
||||
from paddleclas import PaddleClas
|
||||
clas = PaddleClas(model_name='ResNet50', save_dir='./output_pre_label/')
|
||||
infer_imgs = 'docs/images/inference_deployment/whl_' # it can be infer_imgs folder path which contains all of images you want to predict.
|
||||
infer_imgs = 'docs/images/' # it can be infer_imgs folder path which contains all of images you want to predict.
|
||||
result=clas.predict(infer_imgs)
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
* CLI
|
||||
```bash
|
||||
paddleclas --model_name='ResNet50' --infer_imgs='docs/images/inference_deployment/whl_' --save_dir='./output_pre_label/'
|
||||
paddleclas --model_name='ResNet50' --infer_imgs='docs/images/' --save_dir='./output_pre_label/'
|
||||
```
|
||||
|
||||
<a name="4.8"></a>
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 327 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 574 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.9 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 872 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 509 KiB |
|
Before Width: | Height: | Size: 1.2 MiB After Width: | Height: | Size: 1.2 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 72 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.4 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.8 MiB |
|
|
@ -0,0 +1,424 @@
|
|||
# PULC语种分类模型
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 模型和应用场景介绍](#1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [2.1 安装 paddleclas](#2.1)
|
||||
- [2.2 预测](#2.2)
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [3.1 环境配置](#3.1)
|
||||
- [3.2 数据准备](#3.2)
|
||||
- [3.2.1 数据集来源](#3.2.1)
|
||||
- [3.2.2 数据集获取](#3.2.2)
|
||||
- [3.3 模型训练](#3.3)
|
||||
- [3.4 模型评估](#3.4)
|
||||
- [3.5 模型预测](#3.5)
|
||||
- [4. 模型压缩](#4)
|
||||
- [4.1 SKL-UGI 知识蒸馏](#4.1)
|
||||
- [4.1.1 教师模型训练](#4.1.1)
|
||||
- [4.1.2 蒸馏训练](#4.1.2)
|
||||
- [5. 超参搜索](#5)
|
||||
- [6. 模型推理部署](#6)
|
||||
- [6.1 推理模型准备](#6.1)
|
||||
- [6.1.1 基于训练得到的权重导出 inference 模型](#6.1.1)
|
||||
- [6.1.2 直接下载 inference 模型](#6.1.2)
|
||||
- [6.2 基于 Python 预测引擎推理](#6.2)
|
||||
- [6.2.1 预测单张图片](#6.2.1)
|
||||
- [6.2.2 基于文件夹的批量预测](#6.2.2)
|
||||
- [6.3 基于 C++ 预测引擎推理](#6.3)
|
||||
- [6.4 服务化部署](#6.4)
|
||||
- [6.5 端侧部署](#6.5)
|
||||
- [6.6 Paddle2ONNX 模型转换与预测](#6.6)
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 模型和应用场景介绍
|
||||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的语种分类模型。使用该方法训练得到的模型可以快速判断图片中的文字语种，该模型可以广泛应用于金融、政务等各种涉及多语种OCR处理的场景中。
|
||||
|
||||
下表列出了语种分类模型的相关指标，前两行展现了使用 SwinTranformer_tiny 和 MobileNetV3_small_x0_35 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。其中替换 backbone 为 PPLCNet_x1_0时，将数据预处理时的输入尺寸变为[192,48]，且网络的下采样stride调整为[2, [2, 1], [2, 1], [2, 1], [2, 1]]。
|
||||
|
||||
| 模型 | 精度 | 延时 | 存储 | 策略 |
|
||||
| ----------------------- | --------- | -------- | ------- | ---------------------------------------------- |
|
||||
| SwinTranformer_tiny | 98.12 | 89.09 | 107 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 95.92 | 2.98 | 17 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 98.35 | 2.58 | 6.5 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 98.7 | 2.58 | 6.5 | 使用SSLD预训练模型 |
|
||||
| PPLCNet_x1_0 | 99.12 | 2.58 | 6.5 | 使用SSLD预训练模型+EDA策略 |
|
||||
| **PPLCNet_x1_0** | **99.26** | **2.58** | **6.5** | 使用SSLD预训练模型+EDA策略+SKL-UGI知识蒸馏策略 |
|
||||
|
||||
从表中可以看出，backbone 为 SwinTranformer_tiny 时精度比较高，但是推理速度较慢。将 backboone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度提升明显，但精度有了大幅下降。将 backbone 替换为 PPLCNet_x1_0 且调整预处理输入尺寸和网络的下采样stride时，速度略为提升，同时精度较 MobileNetV3_large_x1_0 高2.43个百分点。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升 0.35 个百分点，进一步地，当融合EDA策略后，精度可以再提升 0.42 个百分点，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.14 个百分点。此时，PPLCNet_x1_0 超过了 SwinTranformer_tiny 模型的精度，并且速度有了明显提升。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**关于PPLCNet的介绍可以参考[PPLCNet介绍](../models/PP-LCNet.md)，相关论文可以查阅[PPLCNet paper](https://arxiv.org/abs/2109.15099)。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddleclas
|
||||
|
||||
```
|
||||
pip3 install paddleclas
|
||||
```
|
||||
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
- 使用命令行快速预测
|
||||
|
||||
```
|
||||
paddleclas --model_name=language_classification --infer_imgs=deploy/images/PULC/language_classification/img_rot0_demo.jpg
|
||||
```
|
||||
|
||||
结果如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
class_ids: [4, 9], scores: [0.96809, 0.01001], label_names: ['japan', 'latin'], filename: deploy/images/PULC/language_classification/word_35404.png
|
||||
Predict complete!
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
- 在 Python 代码中预测
|
||||
|
||||
```
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="language_classification")
|
||||
result = model.predict(input_data="deploy/images/PULC/language_classification/word_35404.png")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="language_classification", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
[{'class_ids': [4, 9], 'scores': [0.96809, 0.01001], 'label_names': ['japan', 'latin'], 'filename': 'deploy/images/PULC/language_classification/word_35404.png'}]
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境配置
|
||||
|
||||
- 安装：请先参考 [Paddle 安装教程](../installation/install_paddle.md) 以及 [PaddleClas 安装教程](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
|
||||
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
<a name="3.2.1"></a>
|
||||
|
||||
#### 3.2.1 数据集来源
|
||||
|
||||
[第1节](#1)中提供的模型使用内部数据训练得到，该数据集暂时不方便公开。这里基于 [Multi-lingual scene text detection and recognition](https://rrc.cvc.uab.es/?ch=15&com=downloads) 开源数据集构造了一个多语种demo数据集，用于体验本案例的预测过程。
|
||||
|
||||
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
|
||||
#### 3.2.2 数据集获取
|
||||
|
||||
[第1节](#1)中提供的模型共支持10个类别，分别为：
|
||||
|
||||
`0` 表示阿拉伯语（arabic）；`1` 表示中文繁体（chinese_cht）；`2` 表示斯拉夫语（cyrillic）；`3` 表示梵文（devanagari）；`4` 表示日语（japan）；`5` 表示卡纳达文（ka）；`6` 表示韩语（korean）；`7` 表示泰米尔文（ta）；`8` 表示泰卢固文（te）；`9` 表示拉丁语（latin）。
|
||||
|
||||
在 Multi-lingual scene text detection and recognition 数据集中，仅包含了阿拉伯语、日语、韩语和拉丁语数据，这里分别将4个语种的数据各抽取120张作为本案例的训练数据，50张作为测试数据，以及30张作为补充数据和训练数据混合用于本案例的`SKL-UGI知识蒸馏策略`实验。
|
||||
|
||||
因此，对于本案例中的demo数据集，类别为：
|
||||
|
||||
`0` 表示阿拉伯语（arabic）；`1` 表示日语（japan）；`2` 表示韩语（korean）；`3` 表示拉丁语（latin）。
|
||||
|
||||
如果想要制作自己的多语种数据集，可以按照需求收集并整理自己任务中需要语种的数据，此处提供了经过上述方法处理好的demo数据，可以直接下载得到。
|
||||
|
||||
**备注：**语种分类任务中的图片数据需要将整图中的文字区域抠取出来，仅仅使用文本行部分作为图片数据。
|
||||
|
||||
进入 PaddleClas 目录。
|
||||
|
||||
```
|
||||
cd path_to_PaddleClas
|
||||
```
|
||||
|
||||
进入 `dataset/` 目录，下载并解压多语种场景的demo数据。
|
||||
|
||||
```shell
|
||||
cd dataset
|
||||
wget https://paddleclas.bj.bcebos.com/data/PULC/language_classification.tar
|
||||
tar -xf language_classification.tar
|
||||
cd ../
|
||||
```
|
||||
|
||||
执行上述命令后，`dataset/`下存在`language_classification`目录，该目录中具有以下数据：
|
||||
|
||||
```
|
||||
├── img
|
||||
│ ├── word_1.png
|
||||
│ ├── word_2.png
|
||||
...
|
||||
├── train_list.txt
|
||||
├── train_list_for_distill.txt
|
||||
├── test_list.txt
|
||||
└── label_list.txt
|
||||
```
|
||||
|
||||
其中`img/`存放了4种语言总计800张数据。`train_list.txt`和`test_list.txt`分别为训练集和验证集的标签文件，`label_list.txt`是4类语言分类模型对应的类别列表，`SKL-UGI知识蒸馏策略`对应的训练标签文件为`train_list_for_distill.txt`。用这些图片可以快速体验本案例中模型的训练预测过程。
|
||||
|
||||
***备注：***
|
||||
|
||||
- 这里的`label_list.txt`是4类语种分类模型对应的类别列表，如果自己构造的数据集语种类别发生变化，需要自行调整。
|
||||
- 如果想要自己构造训练集和验证集，可以参考[PaddleClas分类数据集格式说明](../data_preparation/classification_dataset.md#1-数据集格式说明) 。
|
||||
- 当使用本文档中的demo数据集时，需要添加`-o Arch.class_num=4`来将模型的类别书指定为4。
|
||||
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 模型训练
|
||||
|
||||
在`ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml`中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 模型评估
|
||||
|
||||
训练好模型之后，可以通过以下命令实现对模型指标的评估。
|
||||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测
|
||||
|
||||
模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测：
|
||||
|
||||
```bash
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'class_ids': [4, 9], 'scores': [0.96809, 0.01001], 'file_name': 'deploy/images/PULC/language_classification/word_35404.png', 'label_names': ['japan', 'latin']}]
|
||||
```
|
||||
|
||||
***备注：***
|
||||
|
||||
- 其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
- 默认是对 `deploy/images/PULC/language_classification/word_35404.png` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
- 预测输出为top2的预测结果，`japan` 表示该图中文字语种识别为日语，`latin` 表示该图中文字语种识别为拉丁语。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型压缩
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 SKL-UGI 知识蒸馏
|
||||
|
||||
SKL-UGI 知识蒸馏是 PaddleClas 提出的一种简单有效的知识蒸馏方法，关于该方法的介绍，可以参考[SKL-UGI 知识蒸馏](@ruoyu)。
|
||||
|
||||
<a name="4.1.1"></a>
|
||||
|
||||
#### 4.1.1 教师模型训练
|
||||
|
||||
复用`ppcls/configs/PULC/language_classification/PPLCNet/PPLCNet_x1_0.yaml`中的超参数，训练教师模型，训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet/PPLCNet_x1_0.yaml \
|
||||
-o Arch.name=ResNet101_vd
|
||||
```
|
||||
|
||||
当前教师模型最好的权重保存在`output/ResNet101_vd/best_model.pdparams`。
|
||||
|
||||
**备注：** 训练ResNet101_vd模型需要的显存较多，如果机器显存不够，可以将学习率和 batch size 同时缩小一定的倍数进行训练。
|
||||
|
||||
<a name="4.1.2"></a>
|
||||
|
||||
#### 4.1.2 蒸馏训练
|
||||
|
||||
配置文件`ppcls/configs/PULC/language_classification/PPLCNet_x1_0_distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型，使用[3.2.2节](#3.2.2)中介绍的蒸馏数据作为新增的无标签数据。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0_distillation.yaml \
|
||||
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model
|
||||
```
|
||||
|
||||
当前模型最好的权重保存在`output/DistillationModel/best_model_student.pdparams`。
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 超参搜索
|
||||
|
||||
在 [3.2 节](#3.2)和 [4.1 节](#4.1)所使用的超参数是根据 PaddleClas 提供的 `SHAS 超参数搜索策略` 搜索得到的，如果希望在自己的数据集上得到更好的结果，可以参考[SHAS 超参数搜索策略](#TODO)来获得更好的训练超参数。
|
||||
|
||||
**备注：** 此部分内容是可选内容，搜索过程需要较长的时间，您可以根据自己的硬件情况来选择执行。如果没有更换数据集，可以忽略此节内容。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 模型推理部署
|
||||
|
||||
<a name="6.1"></a>
|
||||
|
||||
### 6.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference可使用MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于Paddle Inference推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
当使用 Paddle Inference 推理时，加载的模型类型为 inference 模型。本案例提供了两种获得 inference 模型的方法，如果希望得到和文档相同的结果，请选择[直接下载 inference 模型](#6.1.2)的方式。
|
||||
|
||||
<a name="6.1.1"></a>
|
||||
|
||||
#### 6.1.1 基于训练得到的权重导出 inference 模型
|
||||
|
||||
此处，我们提供了将权重和模型转换的脚本，执行该脚本可以得到对应的 inference 模型：
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/language_classification/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model_student \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_language_classification_infer
|
||||
```
|
||||
|
||||
执行完该脚本后会在`deploy/models/`下生成`PPLCNet_x1_0_language_classification_infer`文件夹，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_language_classification_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
**备注：** 此处的最佳权重是经过知识蒸馏后的权重路径，如果没有执行知识蒸馏的步骤，最佳模型保存在`output/PPLCNet_x1_0/best_model.pdparams`中。
|
||||
|
||||
<a name="6.1.2"></a>
|
||||
|
||||
#### 6.1.2 直接下载 inference 模型
|
||||
|
||||
[6.1.1 小节](#6.1.1)提供了导出 inference 模型的方法，此处也提供了该场景可以下载的 inference 模型，可以直接下载体验。
|
||||
|
||||
```
|
||||
cd deploy/models
|
||||
# 下载inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/language_classification_infer.tar && tar -xf language_classification_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── language_classification_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
<a name="6.2"></a>
|
||||
|
||||
### 6.2 基于 Python 预测引擎推理
|
||||
|
||||
<a name="6.2.1"></a>
|
||||
|
||||
#### 6.2.1 预测单张图像
|
||||
|
||||
返回 `deploy` 目录：
|
||||
|
||||
```
|
||||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/language_classification/word_35404.png` 进行整图文字方向分类。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
||||
```
|
||||
word_35404.png: class id(s): [4, 6], score(s): [0.89, 0.01], label_name(s): ['japan', 'korean']
|
||||
```
|
||||
|
||||
其中，输出为top2的预测结果，`japan` 表示该图中文字语种为日语，`korean` 表示该图中文字语种为韩语。
|
||||
|
||||
<a name="6.2.2"></a>
|
||||
|
||||
#### 6.2.2 基于文件夹的批量预测
|
||||
|
||||
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/language_classification/inference_language_classification.yaml -o Global.infer_imgs="./images/PULC/language_classification/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的分类结果，如下所示。
|
||||
|
||||
```
|
||||
word_17.png: class id(s): [9, 4], score(s): [0.80, 0.09], label_name(s): ['latin', 'japan']
|
||||
word_20.png: class id(s): [0, 4], score(s): [0.91, 0.02], label_name(s): ['arabic', 'japan']
|
||||
word_35404.png: class id(s): [4, 6], score(s): [0.89, 0.01], label_name(s): ['japan', 'korean']
|
||||
```
|
||||
|
||||
其中，输出为top2的预测结果，`japan` 表示该图中文字语种为日语，`latin` 表示该图中文字语种为拉丁语，`arabic` 表示该图中文字语种为阿拉伯语，`korean` 表示该图中文字语种为韩语。
|
||||
|
||||
<a name="6.3"></a>
|
||||
|
||||
### 6.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="6.4"></a>
|
||||
|
||||
### 6.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.5"></a>
|
||||
|
||||
### 6.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.6"></a>
|
||||
|
||||
### 6.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
|
@ -0,0 +1,431 @@
|
|||
# PULC 人体属性识别模型
|
||||
|
||||
------
|
||||
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 模型和应用场景介绍](#1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [3.1 环境配置](#3.1)
|
||||
- [3.2 数据准备](#3.2)
|
||||
- [3.2.1 数据集来源](#3.2.1)
|
||||
- [3.2.2 数据集获取](#3.2.2)
|
||||
- [3.3 模型训练](#3.3)
|
||||
- [3.4 模型评估](#3.4)
|
||||
- [3.5 模型预测](#3.5)
|
||||
- [4. 模型压缩](#4)
|
||||
- [4.1 SKL-UGI 知识蒸馏](#4.1)
|
||||
- [4.1.1 教师模型训练](#4.1.1)
|
||||
- [4.1.2 蒸馏训练](#4.1.2)
|
||||
- [5. 超参搜索](#5)
|
||||
- [6. 模型推理部署](#6)
|
||||
- [6.1 推理模型准备](#6.1)
|
||||
- [6.1.1 基于训练得到的权重导出 inference 模型](#6.1.1)
|
||||
- [6.1.2 直接下载 inference 模型](#6.1.2)
|
||||
- [6.2 基于 Python 预测引擎推理](#6.2)
|
||||
- [6.2.1 预测单张图像](#6.2.1)
|
||||
- [6.2.2 基于文件夹的批量预测](#6.2.2)
|
||||
- [6.3 基于 C++ 预测引擎推理](#6.3)
|
||||
- [6.4 服务化部署](#6.4)
|
||||
- [6.5 端侧部署](#6.5)
|
||||
- [6.6 Paddle2ONNX 模型转换与预测](#6.6)
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 模型和应用场景介绍
|
||||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的人体属性识别模型。该模型可以广泛应用于行人分析、行人跟踪等场景。
|
||||
|
||||
下表列出了不同人体属性识别模型的相关指标，前两行展现了使用 SwinTransformer_tiny、Res2Net200_vd_26w_4s 和 MobileNetV3_small_x0_35 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
|
||||
|
||||
| 模型 | ma（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
|-------|-----------|----------|---------------|---------------|
|
||||
| Res2Net200_vd_26w_4s | 81.25 | 77.51 | 293 | 使用ImageNet预训练模型 |
|
||||
| SwinTransformer_tiny | 80.17 | 89.51 | 107 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 70.79 | 2.90 | 1.7 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 76.31 | 2.01 | 6.6 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 77.31 | 2.01 | 6.6 | 使用SSLD预训练模型 |
|
||||
| PPLCNet_x1_0 | 77.71 | 2.01 | 6.6 | 使用SSLD预训练模型+EDA策略|
|
||||
| <b>PPLCNet_x1_0<b> | <b>78.59<b> | <b>2.01<b> | <b>6.6<b> | 使用SSLD预训练模型+EDA策略+SKL-UGI知识蒸馏策略|
|
||||
|
||||
从表中可以看出，backbone 为 Res2Net200_vd_26w_4s 和 SwinTransformer_tiny 时精度较高，但是推理速度较慢。将 backbone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度可以大幅提升，但是精度也大幅下降。将 backbone 替换为 PPLCNet_x1_0 时，精度较 MobileNetV3_small_x0_35 高 5.5%，于此同时，速度更快。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升 1%，进一步地，当融合EDA策略后，精度可以再提升 0.4%，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.88%。此时，PPLCNet_x1_0 的精度与 SwinTransformer_tiny 仅相差1.58%，但是速度快 44 倍。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 关于PPLCNet的介绍可以参考[PPLCNet介绍](../models/PP-LCNet.md)，相关论文可以查阅[PPLCNet paper](https://arxiv.org/abs/2109.15099)。
|
||||
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddlepaddle, paddleclas
|
||||
|
||||
```
|
||||
pip3 install paddlepaddle paddleclas
|
||||
```
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
* 使用命令行快速预测
|
||||
|
||||
```bash
|
||||
paddleclas --model_name=person_attribute --infer_imgs=deploy/images/PULC/person_attribute/090004.jpg
|
||||
```
|
||||
|
||||
结果如下：
|
||||
```
|
||||
>>> result
|
||||
待补充
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
|
||||
* 在 Python 代码中预测
|
||||
```python
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="person_attribute")
|
||||
result = model.predict(input_data="deploy/images/PULC/person_attribute/090004.jpg")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="person_attribute", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
待补充
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境配置
|
||||
|
||||
* 安装：请先参考文档 [环境准备](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
|
||||
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
<a name="3.2.1"></a>
|
||||
|
||||
#### 3.2.1 数据集来源
|
||||
|
||||
本案例中所使用的数据为[pa100k 数据集](https://www.v7labs.com/open-datasets/pa-100k)。
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
|
||||
#### 3.2.2 数据集获取
|
||||
|
||||
部分数据可视化如下所示。
|
||||
|
||||
<div align="center">
|
||||
<img src="../../images/PULC/docs/person_attribute_data_demo.png" width = "500" />
|
||||
</div>
|
||||
|
||||
|
||||
我们将原始数据转换成了 PaddleClas 多标签可读的数据格式，可以直接下载。
|
||||
|
||||
进入 PaddleClas 目录。
|
||||
|
||||
```
|
||||
cd path_to_PaddleClas
|
||||
```
|
||||
|
||||
进入 `dataset/` 目录，下载并解压有人/无人场景的数据。
|
||||
|
||||
```shell
|
||||
cd dataset
|
||||
wget https://paddleclas.bj.bcebos.com/data/PULC/pa100k.tar
|
||||
tar -xf pa100k.tar
|
||||
cd ../
|
||||
```
|
||||
|
||||
执行上述命令后，`dataset/` 下存在 `pa100k` 目录，该目录中具有以下数据：
|
||||
|
||||
|
||||
执行上述命令后，`pa100k`目录中具有以下数据：
|
||||
|
||||
```
|
||||
pa100k
|
||||
├── train
|
||||
│ ├── 000001.jpg
|
||||
│ ├── 000002.jpg
|
||||
...
|
||||
├── val
|
||||
│ ├── 080001.jpg
|
||||
│ ├── 080002.jpg
|
||||
...
|
||||
├── test
|
||||
│ ├── 090001.jpg
|
||||
│ ├── 090002.jpg
|
||||
...
|
||||
...
|
||||
├── train_list.txt
|
||||
├── train_val_list.txt
|
||||
├── val_list.txt
|
||||
├── test_list.txt
|
||||
```
|
||||
|
||||
其中`train/`、`val/`、`test/`分别为训练集、验证集和测试集。`train_list.txt`、`val_list.txt`、`test_list.txt`分别为训练集、验证集、测试集的标签文件。在本例子中，`test_list.txt`暂时没有使用。
|
||||
|
||||
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 模型训练
|
||||
|
||||
|
||||
在 `ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml` 中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
验证集的最佳指标在 `90.07%` 左右（数据集较小，一般有0.3%左右的波动）。
|
||||
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 模型评估
|
||||
|
||||
训练好模型之后，可以通过以下命令实现对模型指标的评估。
|
||||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测
|
||||
|
||||
模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测：
|
||||
|
||||
```bash
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/PPLCNet_x1_0/best_model
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'attributes': ['Male', 'Age18-60', 'Back', 'Glasses: False', 'Hat: False', 'HoldObjectsInFront: False', 'Backpack', 'Upper: LongSleeve UpperPlaid', 'Lower: Trousers', 'No boots'], 'output': [0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1]}]
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
* 这里`-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
* 默认是对 `deploy/images/PULC/person_attribute/090004.jpg` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型压缩
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 SKL-UGI 知识蒸馏
|
||||
|
||||
SKL-UGI 知识蒸馏是 PaddleClas 提出的一种简单有效的知识蒸馏方法，关于该方法的介绍，可以参考[SKL-UGI 知识蒸馏](@ruoyu)。
|
||||
|
||||
<a name="4.1.1"></a>
|
||||
|
||||
#### 4.1.1 教师模型训练
|
||||
|
||||
复用 `ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml` 中的超参数，训练教师模型，训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Arch.name=ResNet101_vd
|
||||
```
|
||||
|
||||
验证集的最佳指标为 `80.10%` 左右，当前教师模型最好的权重保存在 `output/ResNet101_vd/best_model.pdparams`。
|
||||
|
||||
<a name="4.1.2"></a>
|
||||
|
||||
#### 4.1.2 蒸馏训练
|
||||
|
||||
配置文件`ppcls/configs/PULC/person_attribute/PPLCNet_x1_0_Distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0_Distillation.yaml \
|
||||
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model
|
||||
```
|
||||
|
||||
验证集的最佳指标为 `78.5%` 左右，当前模型最好的权重保存在 `output/DistillationModel/best_model_student.pdparams`。
|
||||
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 超参搜索
|
||||
|
||||
在 [3.2 节](#3.2)和 [4.1 节](#4.1)所使用的超参数是根据 PaddleClas 提供的 `SHAS 超参数搜索策略` 搜索得到的，如果希望在自己的数据集上得到更好的结果，可以参考[SHAS 超参数搜索策略](#TODO)来获得更好的训练超参数。
|
||||
|
||||
**备注：** 此部分内容是可选内容，搜索过程需要较长的时间，您可以根据自己的硬件情况来选择执行。如果没有更换数据集，可以忽略此节内容。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 模型推理部署
|
||||
|
||||
<a name="6.1"></a>
|
||||
|
||||
### 6.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference可使用MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于Paddle Inference推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
当使用 Paddle Inference 推理时，加载的模型类型为 inference 模型。本案例提供了两种获得 inference 模型的方法，如果希望得到和文档相同的结果，请选择[直接下载 inference 模型](#6.1.2)的方式。
|
||||
|
||||
<a name="6.1.1"></a>
|
||||
|
||||
### 6.1.1 基于训练得到的权重导出 inference 模型
|
||||
|
||||
此处，我们提供了将权重和模型转换的脚本，执行该脚本可以得到对应的 inference 模型：
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/person_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model_student \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_person_attribute_infer
|
||||
```
|
||||
执行完该脚本后会在 `deploy/models/` 下生成 `PPLCNet_x1_0_person_attribute_infer` 文件夹，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_person_attribute_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
**备注：** 此处的最佳权重是经过知识蒸馏后的权重路径，如果没有执行知识蒸馏的步骤，最佳模型保存在`output/PPLCNet_x1_0/best_model.pdparams`中。
|
||||
|
||||
<a name="6.1.2"></a>
|
||||
|
||||
### 6.1.2 直接下载 inference 模型
|
||||
|
||||
[6.1.1 小节](#6.1.1)提供了导出 inference 模型的方法，此处也提供了该场景可以下载的 inference 模型，可以直接下载体验。
|
||||
|
||||
```
|
||||
cd deploy/models
|
||||
# 下载 inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/person_attribute_infer.tar && tar -xf person_attribute_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── person_attribute_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
<a name="6.2"></a>
|
||||
|
||||
### 6.2 基于 Python 预测引擎推理
|
||||
|
||||
|
||||
<a name="6.2.1"></a>
|
||||
|
||||
#### 6.2.1 预测单张图像
|
||||
|
||||
返回 `deploy` 目录：
|
||||
|
||||
```
|
||||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/person_attribute/090004.jpg` 进行车辆属性识别。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/person_attribute/inference_person_attribute.yaml -o Global.use_gpu=True
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/person_attribute/inference_person_attribute.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
||||
```
|
||||
090004.jpg: {'attributes': ['Male', 'Age18-60', 'Back', 'Glasses: False', 'Hat: False', 'HoldObjectsInFront: False', 'Backpack', 'Upper: LongSleeve UpperPlaid', 'Lower: Trousers', 'No boots'], 'output': [0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1]}
|
||||
```
|
||||
|
||||
<a name="6.2.2"></a>
|
||||
|
||||
#### 6.2.2 基于文件夹的批量预测
|
||||
|
||||
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/person_attribute/inference_person_attribute.yaml -o Global.infer_imgs="./images/PULC/person_attribute/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的属性识别结果，如下所示。
|
||||
|
||||
```
|
||||
090004.jpg: {'attributes': ['Male', 'Age18-60', 'Back', 'Glasses: False', 'Hat: False', 'HoldObjectsInFront: False', 'Backpack', 'Upper: LongSleeve UpperPlaid', 'Lower: Trousers', 'No boots'], 'output': [0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1]}
|
||||
090007.jpg: {'attributes': ['Female', 'Age18-60', 'Side', 'Glasses: False', 'Hat: False', 'HoldObjectsInFront: False', 'No bag', 'Upper: ShortSleeve', 'Lower: Skirt&Dress', 'No boots'], 'output': [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0]}
|
||||
```
|
||||
|
||||
<a name="6.3"></a>
|
||||
|
||||
### 6.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="6.4"></a>
|
||||
|
||||
### 6.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.5"></a>
|
||||
|
||||
### 6.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.6"></a>
|
||||
|
||||
### 6.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
|
@ -0,0 +1,412 @@
|
|||
# PULC 佩戴安全帽分类模型
|
||||
|
||||
------
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 模型和应用场景介绍](#1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [3.1 环境配置](#3.1)
|
||||
- [3.2 数据准备](#3.2)
|
||||
- [3.2.1 数据集来源](#3.2.1)
|
||||
- [3.2.2 数据集获取](#3.2.2)
|
||||
- [3.3 模型训练](#3.3)
|
||||
- [3.4 模型评估](#3.4)
|
||||
- [3.5 模型预测](#3.5)
|
||||
- [4. 模型压缩](#4)
|
||||
- [4.1 UDML 知识蒸馏](#4.1)
|
||||
- [4.1.1 教师模型训练](#4.1.1)
|
||||
- [4.1.2 蒸馏训练](#4.1.2)
|
||||
- [5. 超参搜索](#5)
|
||||
- [6. 模型推理部署](#6)
|
||||
- [6.1 推理模型准备](#6.1)
|
||||
- [6.1.1 基于训练得到的权重导出 inference 模型](#6.1.1)
|
||||
- [6.1.2 直接下载 inference 模型](#6.1.2)
|
||||
- [6.2 基于 Python 预测引擎推理](#6.2)
|
||||
- [6.2.1 预测单张图像](#6.2.1)
|
||||
- [6.2.2 基于文件夹的批量预测](#6.2.2)
|
||||
- [6.3 基于 C++ 预测引擎推理](#6.3)
|
||||
- [6.4 服务化部署](#6.4)
|
||||
- [6.5 端侧部署](#6.5)
|
||||
- [6.6 Paddle2ONNX 模型转换与预测](#6.6)
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 模型和应用场景介绍
|
||||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的“是否佩戴安全帽”的二分类模型。该模型可以广泛应用于如建筑施工场景、工厂车间场景、交通场景等。
|
||||
|
||||
下表列出了判断图片中是否佩戴安全帽的二分类模型的相关指标，展现了使用 Res2Net200_vd_26w_4s，SwinTranformer_tiny 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + UDML 知识蒸馏策略训练得到的模型的相关指标。
|
||||
|
||||
| 模型 | Tpr（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
|-------|-----------|----------|---------------|---------------|
|
||||
| SwinTranformer_tiny | 93.57 | 91.32 | 107 | 使用ImageNet预训练模型 |
|
||||
| Res2Net200_vd_26w_4s | 98.92 | 80.99 | 284 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 96.50 | 2.85 | 1.6 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 93.29 | 2.03 | 6.5 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 98.07 | 2.03 | 6.5 | 使用SSLD预训练模型 |
|
||||
| PPLCNet_x1_0 | 99.30 | 2.03 | 6.5 | 使用SSLD预训练模型+EDA策略|
|
||||
| <b>PPLCNet_x1_0<b> | <b>99.38<b> | <b>2.03<b> | <b>6.5<b> | 使用SSLD预训练模型+EDA策略+UDML知识蒸馏策略|
|
||||
|
||||
从表中可以看出，在使用服务器端大模型作为 backbone 时，SwinTranformer_tiny 精度较低，Res2Net200_vd_26w_4s 精度较高，但服务器端大模型推理速度普遍较慢。将 backboone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度可以大幅提升，但是精度显著降低。在将 backbone 替换为 PPLCNet_x1_0，精度较 MobileNetV3_small_x0_35 提高约 30 个百分点，与此同时速度快 20% 以上。在此基础上，将 PPLCNet_x1_0 的预训练模型替换为 SSLD 预训练模型后，在对推理速度无影响的前提下，精度提升约 4.8 个百分点，进一步地使用 EDA 策略后，精度可以再提升 0.7 个百分点。此时，PPLCNet_x1_0 已经接近了 Res2Net200_vd_26w_4s 模型的精度，但是速度快 70+ 倍。最后，在使用 UDML 知识蒸馏后，精度可以再提升 0.5 个百分点。此时，PPLCNet_x1_0 已经超过了 Res2Net200_vd_26w_4s 模型的精度，但速度是其 70 余倍。下面详细介绍关于 PULC 安全帽模型的训练方法和推理部署方法。
|
||||
|
||||
**备注：**
|
||||
|
||||
* `Tpr`指标的介绍可以参考 [3.3小节](#3.3)的备注部分，延时是基于 Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz 测试得到，开启MKLDNN加速策略，线程数为10。
|
||||
* 关于PPLCNet的介绍可以参考[PPLCNet介绍](../models/PP-LCNet.md)，相关论文可以查阅[PPLCNet paper](https://arxiv.org/abs/2109.15099)。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddleclas
|
||||
|
||||
```
|
||||
pip3 install paddlepaddle paddleclas
|
||||
```
|
||||
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
* 使用命令行快速预测
|
||||
|
||||
```bash
|
||||
paddleclas --model_name=safety_helmet --infer_imgs=deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
```
|
||||
|
||||
结果如下：
|
||||
```
|
||||
>>> result
|
||||
class_ids: [1], scores: [0.9986255], label_names: ['unwearing_helmet'], filename: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
Predict complete!
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xxx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
* 在 Python 代码中预测
|
||||
```python
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="safety_helmet")
|
||||
result = model.predict(input_data="deploy/images/PULC/safety_helmet/safety_helmet_test_1.png")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="safety_helmet", batch_size=2)`, 使用上述测试代码返回结果示例如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
[{'class_ids': [1], 'scores': [0.9986255], 'label_names': ['unwearing_helmet'], 'filename': 'deploy/images/PULC/safety_helmet/safety_helmet_test_1.png'}]
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境配置
|
||||
|
||||
* 安装：请先参考 [Paddle 安装教程](../installation/install_paddle.md) 以及 [PaddleClas 安装教程](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
|
||||
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
<a name="3.2.1"></a>
|
||||
|
||||
#### 3.2.1 数据集来源
|
||||
|
||||
本案例中所使用的所有数据集均为开源数据，数据集基于[Safety-Helmet-Wearing-Dataset](https://github.com/njvisionpower/Safety-Helmet-Wearing-Dataset)、[hard-hat-detection](https://www.kaggle.com/datasets/andrewmvd/hard-hat-detection)与[Large-scale CelebFaces Attributes (CelebA) Dataset](https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html)处理整合而来。
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
|
||||
#### 3.2.2 数据集获取
|
||||
|
||||
在公开数据集的基础上经过后处理即可得到本案例需要的数据，具体处理方法如下：
|
||||
|
||||
* 对于 Safety-Helmet-Wearing-Dataset 数据集：根据 bbox 标签数据，对其宽、高放大 3 倍作为 bbox 对图像进行裁剪，其中带有安全帽的图像类别为0，不戴安全帽的图像类别为1；
|
||||
* 对于 hard-hat-detection 数据集：仅使用其中类别标签为 “hat” 的图像，并使用 bbox 标签进行裁剪，图像类别为0；
|
||||
* 对于 CelebA 数据集：仅使用其中类别标签为 “Wearing_Hat” 的图像，并使用 bbox 标签进行裁剪，图像类别为0。
|
||||
|
||||
在整合上述数据后，可得到共约 15 万数据，其中戴安全帽与不戴安全帽的图像数量分别约为 2.8 万与 12.1 万，然后在两个类别上分别随机选取 0.56 万张图像作为测试集，共约 1.12 万张图像，其他约 13.8 万张图像作为训练集。
|
||||
|
||||
处理后的数据集部分数据可视化如下：
|
||||
|
||||
|
||||
|
||||
此处提供了经过上述方法处理好的数据，可以直接下载得到。
|
||||
|
||||
进入 PaddleClas 目录。
|
||||
|
||||
```
|
||||
cd path_to_PaddleClas
|
||||
```
|
||||
|
||||
进入 `dataset/` 目录，下载并解压安全帽场景的数据。
|
||||
|
||||
```shell
|
||||
cd dataset
|
||||
wget https://paddleclas.bj.bcebos.com/data/PULC/safety_helmet.tar
|
||||
tar -xf safety_helmet.tar
|
||||
cd ../
|
||||
```
|
||||
|
||||
执行上述命令后，`dataset/` 下存在 `safety_helmet` 目录，该目录中具有以下数据：
|
||||
|
||||
```
|
||||
├── images
|
||||
│ ├── VOC2028_part2_001209_1.jpg
|
||||
│ ├── HHD_hard_hat_workers23_1.jpg
|
||||
│ ├── CelebA_077809.jpg
|
||||
│ ├── ...
|
||||
│ └── ...
|
||||
├── train_list.txt
|
||||
└── val_list.txt
|
||||
```
|
||||
|
||||
其中，`train_list.txt` 和 `val_list.txt` 分别为训练集和验证集的标签文件，所有的图像数据在 `images/` 目录下。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 关于 `train_list.txt`、`val_list.txt`的格式说明，可以参考[PaddleClas分类数据集格式说明](../data_preparation/classification_dataset.md#1-数据集格式说明) 。
|
||||
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 模型训练
|
||||
|
||||
在 `ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml` 中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
验证集的最佳指标在 `0.975-0.985` 之间（数据集较小，容易造成波动）。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 此时使用的指标为Tpr，该指标描述了在假正类率（Fpr）小于某一个指标时的真正类率（Tpr），是产业中二分类问题常用的指标之一。在本案例中，Fpr 为万分之一。关于 Fpr 和 Tpr 的更多介绍，可以参考[这里](https://baike.baidu.com/item/AUC/19282953)。
|
||||
|
||||
* 在eval时，会打印出来当前最佳的 TprAtFpr 指标，具体地，其会打印当前的 `Fpr`、`Tpr` 值，以及当前的 `threshold`值，`Tpr` 值反映了在当前 `Fpr` 值下的召回率，该值越高，代表模型越好。`threshold` 表示当前最佳 `Fpr` 所对应的分类阈值，可用于后续模型部署落地等。
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 模型评估
|
||||
|
||||
训练好模型之后，可以通过以下命令实现对模型指标的评估。
|
||||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/PPLCNet_x1_0/best_model
|
||||
```
|
||||
|
||||
其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了训练过程中的最佳参数权重文件所在的路径，如需指定其他权重文件，只需替换对应的路径即可。
|
||||
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测
|
||||
|
||||
模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测：
|
||||
|
||||
```python
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/PPLCNet_x1_0/best_model
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'class_ids': [1], 'scores': [0.9524797], 'label_names': ['unwearing_helmet'], 'file_name': 'deploy/images/PULC/safety_helmet/safety_helmet_test_1.png'}]
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
* 这里`-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
* 默认是对 `deploy/images/PULC/safety_helmet/safety_helmet_test_1.png` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
|
||||
* 二分类默认的阈值为0.5， 如果需要指定阈值，可以重写 `Infer.PostProcess.threshold` ，如 `-o Infer.PostProcess.threshold=0.9167`，该值需要根据实际应用场景来确定，在 safety_helmet 数据集的 val 验证集上，在万分之一 Fpr 下得到的最佳 Tpr 时，该值为 0.9167。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型压缩
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 UDML 知识蒸馏
|
||||
|
||||
UDML 知识蒸馏是一种简单有效的知识蒸馏方法，关于该方法的介绍，可以参考[UDML 知识蒸馏](@ruoyu)。
|
||||
|
||||
<a name="4.1.1"></a>
|
||||
|
||||
#### 4.1.1 蒸馏训练
|
||||
|
||||
配置文件 `ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_distillation.yaml` 提供了 `UDML知识蒸馏策略` 的配置。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_distillation.yaml
|
||||
```
|
||||
|
||||
验证集的最佳指标为 `0.990-0.993` 之间，当前模型最好的权重保存在 `output/DistillationModel/best_model_student.pdparams`。
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 超参搜索
|
||||
|
||||
在 [3.2 节](#3.2)和 [4.1 节](#4.1)所使用的超参数是根据 PaddleClas 提供的 `SHAS 超参数搜索策略` 搜索得到的，如果希望在自己的数据集上得到更好的结果，可以参考[SHAS 超参数搜索策略](#TODO)来获得更好的训练超参数。
|
||||
|
||||
**备注**：此部分内容是可选内容，搜索过程需要较长的时间，您可以根据自己的硬件情况来选择执行。如果没有更换数据集，可以忽略此节内容。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 模型推理部署
|
||||
|
||||
<a name="6.1"></a>
|
||||
|
||||
### 6.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference 可使用 MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于 Paddle Inference 推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
当使用 Paddle Inference 推理时，加载的模型类型为 inference 模型。本案例提供了两种获得 inference 模型的方法，如果希望得到和文档相同的结果，请选择[直接下载 inference 模型](#6.1.2)的方式。
|
||||
|
||||
<a name="6.1.1"></a>
|
||||
|
||||
### 6.1.1 基于训练得到的权重导出 inference 模型
|
||||
|
||||
此处，我们提供了将权重和模型转换的脚本，执行该脚本可以得到对应的 inference 模型：
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model_student \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_safety_helmet_infer
|
||||
```
|
||||
|
||||
执行完该脚本后会在 `deploy/models/` 下生成 `PPLCNet_x1_0_safety_helmet_infer` 目录，该目录下有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_safety_helmet_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
**备注：** 此处的最佳权重是经过知识蒸馏后的权重路径，如果没有执行知识蒸馏的步骤，最佳模型保存在 `output/PPLCNet_x1_0/best_model.pdparams` 中。
|
||||
|
||||
<a name="6.1.2"></a>
|
||||
|
||||
### 6.1.2 直接下载 inference 模型
|
||||
|
||||
[6.1.1 小节](#6.1.1)提供了导出 inference 模型的方法，此处也提供了该场景可以下载的 inference 模型，可以直接下载体验。
|
||||
|
||||
```
|
||||
cd deploy/models
|
||||
# 下载 inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/safety_helmet_infer.tar && tar -xf safety_helmet_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── safety_helmet_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
<a name="6.2"></a>
|
||||
|
||||
### 6.2 基于 Python 预测引擎推理
|
||||
|
||||
<a name="6.2.1"></a>
|
||||
|
||||
#### 6.2.1 预测单张图像
|
||||
|
||||
返回 `deploy` 目录：
|
||||
|
||||
```
|
||||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/safety_helmet/safety_helmet_test_1.png` 进行是否佩戴安全帽分类。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
||||
```
|
||||
safety_helmet_test_1.png: class id(s): [1], score(s): [1.00], label_name(s): ['unwearing_helmet']
|
||||
```
|
||||
|
||||
**备注：** 二分类默认的阈值为0.5， 如果需要指定阈值，可以重写 `Infer.PostProcess.threshold` ，如 `-o Infer.PostProcess.threshold=0.9167`，该值需要根据实际应用场景来确定，在 safety_helmet 数据集的 val 验证集上，在万分之一 Fpr 下得到的最佳 Tpr 时，该值为 0.9167。该阈值的确定方法可以参考[3.3节](#3.3)备注部分。
|
||||
|
||||
<a name="6.2.2"></a>
|
||||
|
||||
#### 6.2.2 基于文件夹的批量预测
|
||||
|
||||
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/safety_helmet/inference_safety_helmet.yaml -o Global.infer_imgs="./images/PULC/safety_helmet/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的分类结果，如下所示。
|
||||
|
||||
```
|
||||
safety_helmet_test_1.png: class id(s): [1], score(s): [1.00], label_name(s): ['unwearing_helmet']
|
||||
safety_helmet_test_2.png: class id(s): [0], score(s): [1.00], label_name(s): ['wearing_helmet']
|
||||
```
|
||||
|
||||
其中，`wearing_helmet` 表示该图中的人佩戴了安全帽，`unwearing_helmet` 表示该图中的人未佩戴安全帽。
|
||||
|
||||
<a name="6.3"></a>
|
||||
|
||||
### 6.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="6.4"></a>
|
||||
|
||||
### 6.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.5"></a>
|
||||
|
||||
### 6.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.6"></a>
|
||||
|
||||
### 6.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
|
@ -0,0 +1,439 @@
|
|||
# PULC含文字图像方向分类模型
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 模型和应用场景介绍](#1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [2.1 安装 paddleclas](#2.1)
|
||||
- [2.2 预测](#2.2)
|
||||
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [3.1 环境配置](#3.1)
|
||||
- [3.2 数据准备](#3.2)
|
||||
- [3.2.1 数据集来源](#3.2.1)
|
||||
- [3.2.2 数据集获取](#3.2.2)
|
||||
- [3.3 模型训练](#3.3)
|
||||
- [3.4 模型评估](#3.4)
|
||||
- [3.5 模型预测](#3.5)
|
||||
- [4. 模型压缩](#4)
|
||||
- [4.1 SKL-UGI 知识蒸馏](#4.1)
|
||||
- [4.1.1 教师模型训练](#4.1.1)
|
||||
- [4.1.2 蒸馏训练](#4.1.2)
|
||||
- [5. 超参搜索](#5)
|
||||
- [6. 模型推理部署](#6)
|
||||
- [6.1 推理模型准备](#6.1)
|
||||
- [6.1.1 基于训练得到的权重导出 inference 模型](#6.1.1)
|
||||
- [6.1.2 直接下载 inference 模型](#6.1.2)
|
||||
- [6.2 基于 Python 预测引擎推理](#6.2)
|
||||
- [6.2.1 预测单张图片](#6.2.1)
|
||||
- [6.2.2 基于文件夹的批量预测](#6.2.2)
|
||||
- [6.3 基于 C++ 预测引擎推理](#6.3)
|
||||
- [6.4 服务化部署](#6.4)
|
||||
- [6.5 端侧部署](#6.5)
|
||||
- [6.6 Paddle2ONNX 模型转换与预测](#6.6)
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 模型和应用场景介绍
|
||||
|
||||
在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的图片也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断含文字图像的方向，并将其进行方向调整，从而提高OCR处理的准确性。该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的含文字图像方向的分类模型。该模型可以广泛应用于金融、政务等行业的旋转图片的OCR处理场景中。
|
||||
|
||||
下表列出了判断含文字图像方向分类模型的相关指标，前两行展现了使用 SwinTranformer_tiny 和 MobileNetV3_small_x0_35 作为 backbone 训练得到的模型的相关指标，第三行至第五行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SHAS 超参数搜索策略训练得到的模型的相关指标。
|
||||
|
||||
| 模型 | 精度（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
| ----------------------- | --------- | ---------- | --------- | ------------------------------------- |
|
||||
| SwinTranformer_tiny | 99.12 | 89.65 | 107 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 83.61 | 2.95 | 17 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 97.85 | 2.16 | 6.5 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 98.02 | 2.16 | 6.5 | 使用SSLD预训练模型 |
|
||||
| **PPLCNet_x1_0** | **99.06** | **2.16** | **6.5** | 使用SSLD预训练模型+SHAS超参数搜索策略 |
|
||||
|
||||
从表中可以看出，backbone 为 SwinTranformer_tiny 时精度比较高，但是推理速度较慢。将 backboone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度提升明显，但精度有了大幅下降。将 backbone 替换为 PPLCNet_x1_0 时，速度略为提升，同时精度较 MobileNetV3_small_x0_35 高了 14.24 个百分点。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升 0.17 个百分点，进一步地，当使用SHAS超参数搜索策略搜索最优超参数后，精度可以再提升 1.04 个百分点。此时，PPLCNet_x1_0 与 SwinTranformer_tiny 的精度差别不大，但是速度明显变快。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**关于PPLCNet的介绍可以参考[PPLCNet介绍](../models/PP-LCNet.md)，相关论文可以查阅[PPLCNet paper](https://arxiv.org/abs/2109.15099)。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddleclas
|
||||
|
||||
```
|
||||
pip3 install paddleclas
|
||||
```
|
||||
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
- 使用命令行快速预测
|
||||
|
||||
```
|
||||
paddleclas --model_name=text_image_orientation --infer_imgs=deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
```
|
||||
|
||||
结果如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
class_ids: [0, 2], scores: [0.85615, 0.05046], label_names: ['0', '180'], filename: deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
Predict complete!
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
- 在 Python 代码中预测
|
||||
|
||||
```
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="text_image_orientation")
|
||||
result = model.predict(input_data="deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="text_image_orientation", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
[{'class_ids': [0, 2], 'scores': [0.85615, 0.05046], 'label_names': ['0', '180'], 'filename': 'deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg'}]
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境配置
|
||||
|
||||
- 安装：请先参考 [Paddle 安装教程](../installation/install_paddle.md) 以及 [PaddleClas 安装教程](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
|
||||
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
<a name="3.2.1"></a>
|
||||
|
||||
#### 3.2.1 数据集来源
|
||||
|
||||
[第1节](#1)中提供的模型使用内部数据训练得到，该数据集暂时不方便公开。这里基于 [ICDAR2019-ArT](https://ai.baidu.com/broad/introduction?dataset=art)、 [XFUND](https://github.com/doc-analysis/XFUND) 和 [ICDAR2015](https://rrc.cvc.uab.es/?ch=4&com=introduction) 三个公开数据集构造了一个小规模含文字图像方向分类数据集，用于体验本案例。
|
||||
|
||||
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
|
||||
#### 3.2.2 数据集获取
|
||||
|
||||
在公开数据集的基础上经过后处理即可得到本案例需要的数据，具体处理方法如下：
|
||||
|
||||
考虑到原始图片的分辨率较高，模型训练时间较长，这里将所有数据预先进行了缩放处理，在保持长宽比不变的前提下，将短边缩放到384。然后将数据进行顺时针旋转处理，分别生成90度、180度和270度的合成数据。其中，将 ICDAR2019-ArT 和 XFUND 生成的41460张数据按照 9:1 的比例随机划分成了训练集和验证集， ICDAR2015 生成的6000张数据作为`SKL-UGI知识蒸馏策略`实验中的补充数据。
|
||||
|
||||
处理后的数据集部分数据可视化如下：
|
||||
|
||||
|
||||
|
||||
此处提供了经过上述方法处理好的数据，可以直接下载得到。
|
||||
|
||||
进入 PaddleClas 目录。
|
||||
|
||||
```
|
||||
cd path_to_PaddleClas
|
||||
```
|
||||
|
||||
进入 `dataset/` 目录，下载并解压含文字图像方向场景的数据。
|
||||
|
||||
```shell
|
||||
cd dataset
|
||||
wget https://paddleclas.bj.bcebos.com/data/PULC/text_image_orientation.tar
|
||||
tar -xf text_image_orientation.tar
|
||||
cd ../
|
||||
```
|
||||
|
||||
执行上述命令后，`dataset/`下存在`text_image_orientation`目录，该目录中具有以下数据：
|
||||
|
||||
```
|
||||
├── img_0
|
||||
│ ├── img_rot0_0.jpg
|
||||
│ ├── img_rot0_1.png
|
||||
...
|
||||
├── img_90
|
||||
│ ├── img_rot90_0.jpg
|
||||
│ ├── img_rot90_1.png
|
||||
...
|
||||
├── img_180
|
||||
│ ├── img_rot180_0.jpg
|
||||
│ ├── img_rot180_1.png
|
||||
...
|
||||
├── img_270
|
||||
│ ├── img_rot270_0.jpg
|
||||
│ ├── img_rot270_1.png
|
||||
...
|
||||
├── distill_data
|
||||
│ ├── gt_7060_0.jpg
|
||||
│ ├── gt_7060_90.jpg
|
||||
...
|
||||
├── train_list.txt
|
||||
├── train_list.txt.debug
|
||||
├── train_list_for_distill.txt
|
||||
├── test_list.txt
|
||||
├── test_list.txt.debug
|
||||
└── label_list.txt
|
||||
```
|
||||
|
||||
其中`img_0/`、`img_90/`、`img_180/`和`img_270/`分别存放了4个角度的训练集和验证集数据。`train_list.txt`和`test_list.txt`分别为训练集和验证集的标签文件，`train_list.txt.debug`和`test_list.txt.debug`分别为训练集和验证集的`debug`标签文件，其分别是`train_list.txt`和`test_list.txt`的子集，用该文件可以快速体验本案例的流程。`distill_data/`是补充文字数据，该集合和`train`集合的混合数据用于本案例的`SKL-UGI知识蒸馏策略`，对应的训练标签文件为`train_list_for_distill.txt`。关于如何得到蒸馏的标签可以参考[知识蒸馏标签获得](@ruoyu)。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 关于 `train_list.txt`、`val_list.txt`的格式说明，可以参考[PaddleClas分类数据集格式说明](../data_preparation/classification_dataset.md#1-数据集格式说明) 。
|
||||
|
||||
* 关于如何得到蒸馏的标签文件可以参考[知识蒸馏标签获得方法](@ruoyu)。
|
||||
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 模型训练
|
||||
|
||||
在`ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml`中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
验证集的最佳指标在0.99左右。
|
||||
|
||||
**备注**：本文档中提到的训练指标均为在大规模内部数据上的训练指标，使用demo数据训练时，由于数据集规模较小且分布与大规模内部数据不同，无法达到该指标。可以进一步扩充自己的数据并且使用本案例中介绍的优化方法进行调优，从而达到更高的精度。
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 模型评估
|
||||
|
||||
训练好模型之后，可以通过以下命令实现对模型指标的评估。
|
||||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测
|
||||
|
||||
模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测：
|
||||
|
||||
```bash
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'class_ids': [0, 2], 'scores': [0.85615, 0.05046], 'file_name': 'deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg', 'label_names': ['0', '180']}]
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
- 其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
- 默认是对 `deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
- 输出为top2的预测结果，`0` 表示该图文本方向为0度，`90` 表示该图文本方向为顺时针90度，`180` 表示该图文本方向为顺时针180度，`270` 表示该图文本方向为顺时针270度。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型压缩
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 SKL-UGI 知识蒸馏
|
||||
|
||||
SKL-UGI 知识蒸馏是 PaddleClas 提出的一种简单有效的知识蒸馏方法，关于该方法的介绍，可以参考[SKL-UGI 知识蒸馏](@ruoyu)。
|
||||
|
||||
<a name="4.1.1"></a>
|
||||
|
||||
#### 4.1.1 教师模型训练
|
||||
|
||||
复用`ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml`中的超参数，训练教师模型，训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Arch.name=ResNet101_vd
|
||||
```
|
||||
|
||||
验证集的最佳指标为0.996左右，当前教师模型最好的权重保存在`output/ResNet101_vd/best_model.pdparams`。
|
||||
|
||||
**备注：** 训练ResNet101_vd模型需要的显存较多，如果机器显存不够，可以将学习率和 batch size 同时缩小一定的倍数进行训练。
|
||||
|
||||
<a name="4.1.2"></a>
|
||||
|
||||
#### 4.1.2 蒸馏训练
|
||||
|
||||
配置文件`ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型，使用[3.2.2节](#3.2.2)中介绍的蒸馏数据作为新增的无标签数据。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_distillation.yaml \
|
||||
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model
|
||||
```
|
||||
|
||||
验证集的最佳指标为0.99左右，当前模型最好的权重保存在`output/DistillationModel/best_model_student.pdparams`。
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 超参搜索
|
||||
|
||||
在 [3.2 节](#3.2)和 [4.1 节](#4.1)所使用的超参数是根据 PaddleClas 提供的 `SHAS 超参数搜索策略` 搜索得到的，如果希望在自己的数据集上得到更好的结果，可以参考[SHAS 超参数搜索策略](#TODO)来获得更好的训练超参数。
|
||||
|
||||
**备注：** 此部分内容是可选内容，搜索过程需要较长的时间，您可以根据自己的硬件情况来选择执行。如果没有更换数据集，可以忽略此节内容。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 模型推理部署
|
||||
|
||||
<a name="6.1"></a>
|
||||
|
||||
### 6.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference可使用MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于Paddle Inference推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
当使用 Paddle Inference 推理时，加载的模型类型为 inference 模型。本案例提供了两种获得 inference 模型的方法，如果希望得到和文档相同的结果，请选择[直接下载 inference 模型](#6.1.2)的方式。
|
||||
|
||||
<a name="6.1.1"></a>
|
||||
|
||||
#### 6.1.1 基于训练得到的权重导出 inference 模型
|
||||
|
||||
此处，我们提供了将权重和模型转换的脚本，执行该脚本可以得到对应的 inference 模型：
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model_student \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_text_image_orientation_infer
|
||||
```
|
||||
|
||||
执行完该脚本后会在`deploy/models/`下生成`PPLCNet_x1_0_text_image_orientation_infer`文件夹，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_text_image_orientation_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
**备注：** 此处的最佳权重是经过知识蒸馏后的权重路径，如果没有执行知识蒸馏的步骤，最佳模型保存在`output/PPLCNet_x1_0/best_model.pdparams`中。
|
||||
|
||||
<a name="6.1.2"></a>
|
||||
|
||||
#### 6.1.2 直接下载 inference 模型
|
||||
|
||||
[6.1.1 小节](#6.1.1)提供了导出 inference 模型的方法，此处也提供了该场景可以下载的 inference 模型，可以直接下载体验。
|
||||
|
||||
```
|
||||
cd deploy/models
|
||||
# 下载inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/text_image_orientation_infer.tar && tar -xf text_image_orientation_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── text_image_orientation_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
<a name="6.2"></a>
|
||||
|
||||
### 6.2 基于 Python 预测引擎推理
|
||||
|
||||
<a name="6.2.1"></a>
|
||||
|
||||
#### 6.2.1 预测单张图像
|
||||
|
||||
返回 `deploy` 目录：
|
||||
|
||||
```
|
||||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/text_image_orientation/img_rot0_demo.png` 进行含文字图像方向分类。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/text_image_orientation/inference_text_image_orientation.yaml
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/text_image_orientation/inference_text_image_orientation.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
||||
```
|
||||
img_rot0_demo.jpg: class id(s): [0, 2], score(s): [0.86, 0.05], label_name(s): ['0', '180']
|
||||
```
|
||||
|
||||
其中，输出为top2的预测结果，`0` 表示该图文本方向为0度，`90` 表示该图文本方向为顺时针90度，`180` 表示该图文本方向为顺时针180度，`270` 表示该图文本方向为顺时针270度。
|
||||
|
||||
<a name="6.2.2"></a>
|
||||
|
||||
#### 6.2.2 基于文件夹的批量预测
|
||||
|
||||
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/text_image_orientation/inference_text_image_orientation.yaml -o Global.infer_imgs="./images/PULC/text_image_orientation/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的分类结果，如下所示。
|
||||
|
||||
```
|
||||
img_rot0_demo.jpg: class id(s): [0, 2], score(s): [0.86, 0.05], label_name(s): ['0', '180']
|
||||
img_rot180_demo.jpg: class id(s): [2, 1], score(s): [0.88, 0.04], label_name(s): ['180', '90']
|
||||
```
|
||||
|
||||
<a name="6.3"></a>
|
||||
|
||||
### 6.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="6.4"></a>
|
||||
|
||||
### 6.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.5"></a>
|
||||
|
||||
### 6.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.6"></a>
|
||||
|
||||
### 6.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
|
@ -0,0 +1,439 @@
|
|||
# PULC 文本行方向分类模型
|
||||
|
||||
------
|
||||
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 模型和应用场景介绍](#1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [3.1 环境配置](#3.1)
|
||||
- [3.2 数据准备](#3.2)
|
||||
- [3.2.1 数据集来源](#3.2.1)
|
||||
- [3.2.2 数据集获取](#3.2.2)
|
||||
- [3.3 模型训练](#3.3)
|
||||
- [3.4 模型评估](#3.4)
|
||||
- [3.5 模型预测](#3.5)
|
||||
- [4. 模型压缩](#4)
|
||||
- [4.1 SKL-UGI 知识蒸馏](#4.1)
|
||||
- [4.1.1 教师模型训练](#4.1.1)
|
||||
- [4.1.2 蒸馏训练](#4.1.2)
|
||||
- [5. 超参搜索](#5)
|
||||
- [6. 模型推理部署](#6)
|
||||
- [6.1 推理模型准备](#6.1)
|
||||
- [6.1.1 基于训练得到的权重导出 inference 模型](#6.1.1)
|
||||
- [6.1.2 直接下载 inference 模型](#6.1.2)
|
||||
- [6.2 基于 Python 预测引擎推理](#6.2)
|
||||
- [6.2.1 预测单张图像](#6.2.1)
|
||||
- [6.2.2 基于文件夹的批量预测](#6.2.2)
|
||||
- [6.3 基于 C++ 预测引擎推理](#6.3)
|
||||
- [6.4 服务化部署](#6.4)
|
||||
- [6.5 端侧部署](#6.5)
|
||||
- [6.6 Paddle2ONNX 模型转换与预测](#6.6)
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 模型和应用场景介绍
|
||||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的文本行方向分类模型。该模型可以广泛应用于如文字矫正、文字识别等场景。
|
||||
|
||||
下表列出了文本行方向分类模型的相关指标，前两行展现了使用 Res2Net200_vd 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
|
||||
|
||||
| 模型 | Top-1 Acc（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
|-------|-----------|----------|---------------|---------------|
|
||||
| SwinTranformer_tiny | 93.61 | 89.64 | 107 | 使用 ImageNet 预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 81.40 | 2.96 | 17 | 使用 ImageNet 预训练模型 |
|
||||
| PPLCNet_x1_0 | 89.99 | 2.11 | 6.5 | 使用 ImageNet 预训练模型 |
|
||||
| PPLCNet_x1_0* | 94.06 | 2.68 | 6.5 | 使用 ImageNet 预训练模型 |
|
||||
| PPLCNet_x1_0* | 94.11 | 2.68 | 6.5 | 使用 SSLD 预训练模型 |
|
||||
| <b>PPLCNet_x1_0**<b> | <b>96.01<b> | <b>2.72<b> | <b>6.5<b> | 使用 SSLD 预训练模型+EDA 策略|
|
||||
| PPLCNet_x1_0** | 95.86 | 2.72 | 6.5 | 使用 SSLD 预训练模型+EDA 策略+SKL-UGI 知识蒸馏策略|
|
||||
|
||||
从表中可以看出，backbone 为 SwinTranformer_tiny 时精度较高，但是推理速度较慢。将 backboone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度可以大幅提升，精度下降也比较明显。将 backbone 替换为 PPLCNet_x1_0 时，精度较 MobileNetV3_small_x0_35 高 8.6 个百分点，速度快10%左右。在此基础上，更改分辨率和stride， 速度变慢 27%，但是精度可以提升 4.5%（采用[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)的方案），使用 SSLD 预训练模型后，精度可以继续提升约 0.05% ，进一步地，当融合EDA策略后，精度可以再提升 1.9 个百分点。最后，融合SKL-UGI 知识蒸馏策略后，在该场景无效。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 其中不带\*的模型表示分辨率为224x224，带\*的模型表示分辨率为48x192（h*w）,数据增强从网络中的 stride 改为 `[2, [2, 1], [2, 1], [2, 1], [2, 1]]`，其中，外层列表中的每一个元素代表网络结构下采样层的stride，该策略为 [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) 提供的文本行方向分类器方案。带\*\*的模型表示分辨率为80x160（h*w）, 网络中的 stride 改为 `[2, [2, 1], [2, 1], [2, 1], [2, 1]]`，其中，外层列表中的每一个元素代表网络结构下采样层的stride，此分辨率是经过[SHAS 超参数搜索策略](#TODO)搜索得到的。
|
||||
* 延时是基于 Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz 测试得到，开启 MKLDNN 加速策略，线程数为10。
|
||||
* 关于PPLCNet的介绍可以参考[PPLCNet介绍](../models/PP-LCNet.md)，相关论文可以查阅[PPLCNet paper](https://arxiv.org/abs/2109.15099)。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddlepaddle, paddleclas
|
||||
|
||||
```
|
||||
pip3 install paddlepaddle paddleclas
|
||||
```
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
* 使用命令行快速预测
|
||||
|
||||
```bash
|
||||
paddleclas --model_name=textline_orientation --infer_imgs=deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
```
|
||||
|
||||
结果如下：
|
||||
```
|
||||
>>> result
|
||||
class_ids: [0], scores: [1.00], label_names: ['0_degree'], filename: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
Predict complete!
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
|
||||
* 在 Python 代码中预测
|
||||
```python
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="textline_orientation")
|
||||
result = model.predict(input_data="deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="person_exists", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
>>> result
|
||||
[{'class_ids': [0], 'scores': [1.00], 'label_names': ['0_degree'], 'filename': 'deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png'}]
|
||||
```
|
||||
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境配置
|
||||
|
||||
* 安装：请先参考 [Paddle 安装教程](../installation/install_paddle.md) 以及 [PaddleClas 安装教程](../installation/install_paddleclas.md) 配置 PaddleClas 运行环境。
|
||||
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
<a name="3.2.1"></a>
|
||||
|
||||
#### 3.2.1 数据集来源
|
||||
|
||||
本案例中所使用的所有数据集来源于内部数据，如果您希望体验训练过程，可以使用开源数据如[ICDAR2019-LSVT 文本行识别数据](https://aistudio.baidu.com/aistudio/datasetdetail/8429)。
|
||||
|
||||
<a name="3.2.2"></a>
|
||||
|
||||
#### 3.2.2 数据集获取
|
||||
|
||||
在公开数据集的基础上经过后处理即可得到本案例需要的数据，具体处理方法如下：
|
||||
|
||||
本案例处理了 ICDAR2019-LSVT 文本行识别数据，将其中的 id 号为 0-1999 作为本案例的数据集合，经过旋转处理成 0 类 和 1 类，其中 0 类代表文本行为正，即 0 度，1 类代表文本行为反，即 180 度。
|
||||
|
||||
- 训练集合，id号为 0-1799 作为训练集合，0 类和 1 类共 3600 张。
|
||||
|
||||
- 验证集合，id号为 1800-1999 作为验证集合，0 类和 1 类共 400 张。
|
||||
|
||||
处理后的数据集部分数据可视化如下：
|
||||
|
||||
|
||||
|
||||
|
||||
此处提供了经过上述方法处理好的数据，可以直接下载得到。
|
||||
|
||||
|
||||
进入 PaddleClas 目录。
|
||||
|
||||
```
|
||||
cd path_to_PaddleClas
|
||||
```
|
||||
|
||||
进入 `dataset/` 目录，下载并解压有人/无人场景的数据。
|
||||
|
||||
```shell
|
||||
cd dataset
|
||||
wget https://paddleclas.bj.bcebos.com/data/PULC/textline_orientation.tar
|
||||
tar -xf textline_orientation.tar
|
||||
cd ../
|
||||
```
|
||||
|
||||
执行上述命令后，`dataset/` 下存在 `textline_orientation` 目录，该目录中具有以下数据：
|
||||
|
||||
```
|
||||
|
||||
├── 0
|
||||
│ ├── img_0.jpg
|
||||
│ ├── img_1.jpg
|
||||
...
|
||||
├── 1
|
||||
│ ├── img_0.jpg
|
||||
│ ├── img_1.jpg
|
||||
...
|
||||
├── train_list.txt
|
||||
└── val_list.txt
|
||||
```
|
||||
|
||||
其中 `0/` 和 `1/` 分别存放 0 类和 1 类的数据。`train_list.txt` 和 `val_list.txt` 分别为训练集和验证集的标签文件。
|
||||
|
||||
**备注：**
|
||||
|
||||
* 关于 `train_list.txt`、`val_list.txt` 的格式说明，可以参考[PaddleClas分类数据集格式说明](../data_preparation/classification_dataset.md#1-数据集格式说明) 。
|
||||
|
||||
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 模型训练
|
||||
|
||||
|
||||
在 `ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml` 中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
|
||||
**备注：**
|
||||
|
||||
* 由于此时使用的数据集并非内部非开源数据集，此处不能直接复现提供的模型的指标，如果希望得到更高的精度，可以根据需要处理[ICDAR2019-LSVT 文本行识别数据](https://aistudio.baidu.com/aistudio/datasetdetail/8429)。
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 模型评估
|
||||
|
||||
训练好模型之后，可以通过以下命令实现对模型指标的评估。
|
||||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
其中 `-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测
|
||||
|
||||
模型训练完成之后，可以加载训练得到的预训练模型，进行模型预测。在模型库的 `tools/infer.py` 中提供了完整的示例，只需执行下述命令即可完成模型预测：
|
||||
|
||||
```python
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'class_ids': [0], 'scores': [1.0], 'file_name': 'deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png', 'label_names': ['0_degree']}]
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
* 这里`-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
* 默认是对 `deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型压缩
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 SKL-UGI 知识蒸馏
|
||||
|
||||
SKL-UGI 知识蒸馏是 PaddleClas 提出的一种简单有效的知识蒸馏方法，关于该方法的介绍，可以参考[SKL-UGI 知识蒸馏](@ruoyu)。
|
||||
|
||||
<a name="4.1.1"></a>
|
||||
|
||||
#### 4.1.1 教师模型训练
|
||||
|
||||
复用 `./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml` 中的超参数，训练教师模型，训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Arch.name=ResNet101_vd
|
||||
```
|
||||
|
||||
当前教师模型最好的权重保存在 `output/ResNet101_vd/best_model.pdparams`。
|
||||
|
||||
<a name="4.1.2"></a>
|
||||
|
||||
#### 4.1.2 蒸馏训练
|
||||
|
||||
配置文件`ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0_distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0_distillation.yaml \
|
||||
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model
|
||||
```
|
||||
|
||||
当前模型最好的权重保存在 `output/DistillationModel/best_model_student.pdparams`。
|
||||
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 超参搜索
|
||||
|
||||
在 [3.2 节](#3.2)和 [4.1 节](#4.1)所使用的超参数是根据 PaddleClas 提供的 `SHAS 超参数搜索策略` 搜索得到的，如果希望在自己的数据集上得到更好的结果，可以参考[SHAS 超参数搜索策略](#TODO)来获得更好的训练超参数。
|
||||
|
||||
**备注：** 此部分内容是可选内容，搜索过程需要较长的时间，您可以根据自己的硬件情况来选择执行。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 模型推理部署
|
||||
|
||||
<a name="6.1"></a>
|
||||
|
||||
### 6.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference可使用MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于Paddle Inference推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
当使用 Paddle Inference 推理时，加载的模型类型为 inference 模型。本案例提供了两种获得 inference 模型的方法，如果希望得到和文档相同的结果，请选择[直接下载 inference 模型](#6.1.2)的方式。
|
||||
|
||||
<a name="6.1.1"></a>
|
||||
|
||||
### 6.1.1 基于训练得到的权重导出 inference 模型
|
||||
|
||||
此处，我们提供了将权重和模型转换的脚本，执行该脚本可以得到对应的 inference 模型：
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/textline_orientation/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/PPLCNet_x1_0/best_model \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_textline_orientation_infer
|
||||
```
|
||||
执行完该脚本后会在 `deploy/models/` 下生成 `PPLCNet_x1_0_textline_orientation_infer` 文件夹，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_textline_orientation_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
**备注：** 此处的最佳权重可以根据实际情况来选择，如果希望导出知识蒸馏后的权重，则最佳权重保存在`output/DistillationModel/best_model_student.pdparams`，在导出命令中更改`-o Global.pretrained_model=xx`中的字段为`output/DistillationModel/best_model_student`即可。
|
||||
|
||||
<a name="6.1.2"></a>
|
||||
|
||||
### 6.1.2 直接下载 inference 模型
|
||||
|
||||
[6.1.1 小节](#6.1.1)提供了导出 inference 模型的方法，此处也提供了该场景可以下载的 inference 模型，可以直接下载体验。
|
||||
|
||||
```
|
||||
cd deploy/models
|
||||
# 下载 inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/textline_orientation_infer.tar && tar -xf textline_orientation_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── textline_orientation_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
<a name="6.2"></a>
|
||||
|
||||
### 6.2 基于 Python 预测引擎推理
|
||||
|
||||
|
||||
<a name="6.2.1"></a>
|
||||
|
||||
#### 6.2.1 预测单张图像
|
||||
|
||||
返回 `deploy` 目录：
|
||||
|
||||
```
|
||||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/textline_orientation/textline_orientation_test_0_0.png` 进行文字方向cd分类。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/textline_orientation/inference_textline_orientation.yaml
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/textline_orientation/inference_textline_orientation.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
||||
```
|
||||
textline_orientation_test_0_0.png: class id(s): [0], score(s): [1.00], label_name(s): ['0_degree']
|
||||
```
|
||||
|
||||
<a name="6.2.2"></a>
|
||||
|
||||
#### 6.2.2 基于文件夹的批量预测
|
||||
|
||||
如果希望预测文件夹内的图像，可以直接修改配置文件中的 `Global.infer_imgs` 字段，也可以通过下面的 `-o` 参数修改对应的配置。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/textline_orientation/inference_textline_orientation.yaml -o Global.infer_imgs="./images/PULC/textline_orientation/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的分类结果，如下所示。
|
||||
|
||||
```
|
||||
textline_orientation_test_0_0.png: class id(s): [0], score(s): [1.00], label_name(s): ['0_degree']
|
||||
textline_orientation_test_0_1.png: class id(s): [0], score(s): [1.00], label_name(s): ['0_degree']
|
||||
textline_orientation_test_1_0.png: class id(s): [1], score(s): [1.00], label_name(s): ['180_degree']
|
||||
textline_orientation_test_1_1.png: class id(s): [1], score(s): [1.00], label_name(s): ['180_degree']
|
||||
```
|
||||
|
||||
其中，`0_degree` 表示该文本行为 0 度，`180_degree` 表示该文本行为 180 度。
|
||||
|
||||
<a name="6.3"></a>
|
||||
|
||||
### 6.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="6.4"></a>
|
||||
|
||||
### 6.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.5"></a>
|
||||
|
||||
### 6.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="6.6"></a>
|
||||
|
||||
### 6.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
|
@ -39,19 +39,19 @@
|
|||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的交通标志分类模型。该模型可以广泛应用于自动驾驶、道路监控等场景。
|
||||
|
||||
下表列出了不同交通标志分类模型的相关指标，前两行展现了使用 SwinTranformer_tiny 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
下表列出了不同交通标志分类模型的相关指标，前两行展现了使用 SwinTranformer_tiny 和 MobileNetV3_small_x0_35 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
|
||||
|
||||
| 模型 | Top-1 Acc（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
|-------|-----------|----------|---------------|---------------|
|
||||
| SwinTranformer_tiny | 98.11 | 89.45 | 111 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_large_x1_0 | 97.79 | 4.81 | 23 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 93.88 | 3.01 | 3.9 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 97.78 | 2.10 | 8.2 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 97.84 | 2.10 | 8.2 | 使用SSLD预训练模型 |
|
||||
| PPLCNet_x1_0 | 98.14 | 2.10 | 8.2 | 使用SSLD预训练模型+EDA策略|
|
||||
| <b>PPLCNet_x1_0<b> | <b>98.35<b> | <b>2.10<b> | <b>8.2<b> | 使用SSLD预训练模型+EDA策略+SKL-UGI知识蒸馏策略|
|
||||
|
||||
从表中可以看出，backbone 为 SwinTranformer_tiny 时精度较高，但是推理速度较慢。将 backbone 替换为轻量级模型 MobileNetV3_large_x1_0 后，速度可以大幅提升，但是精度下降明显。将 backbone 替换为 PPLCNet_x1_0 时，精度低0.01%，但是速度提升 1 倍左右。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升约 0.06%，进一步地，当融合EDA策略后，精度可以再提升 0.3%，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.21%。此时，PPLCNet_x1_0 的精度超越了SwinTranformer_tiny，速度快 41 倍。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
从表中可以看出，backbone 为 SwinTranformer_tiny 时精度较高，但是推理速度较慢。将 backbone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度可以大幅提升，但是精度下降明显。将 backbone 替换为 PPLCNet_x1_0 时，精度低3.9%，同时速度提升 43% 左右。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升约 0.06%，进一步地，当融合EDA策略后，精度可以再提升 0.3%，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.21%。此时，PPLCNet_x1_0 的精度超越了 SwinTranformer_tiny，速度快 41 倍。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**
|
||||
|
||||
|
|
@ -62,8 +62,48 @@
|
|||
|
||||
## 2. 模型快速体验
|
||||
|
||||
（pip方式，待补充）
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddlepaddle, paddleclas
|
||||
|
||||
```bash
|
||||
pip3 install paddlepaddle paddleclas
|
||||
```
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
* 使用命令行快速预测
|
||||
|
||||
```bash
|
||||
paddleclas --model_name traffic_sign --infer_imgs PaddleClas/deploy/images/PULC/traffic_sign/100999_83928.jpg
|
||||
```
|
||||
|
||||
结果如下：
|
||||
```
|
||||
>>> result
|
||||
class_ids: [182, 179, 162, 128, 24], scores: [0.98623, 0.01255, 0.00022, 0.00021, 0.00012], label_names: ['pl110', 'pl100', 'pl120', 'p26', 'pm10'], filename: PaddleClas/deploy/images/PULC/traffic_sign/100999_83928.jpg
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
|
||||
* 在 Python 代码中预测
|
||||
```python
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="traffic_sign")
|
||||
result = model.predict(input_data="PaddleClas/deploy/images/PULC/traffic_sign/100999_83928.jpg")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="person_exists", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
result
|
||||
[{'class_ids': [182, 179, 162, 128, 24], 'scores': [0.98623, 0.01255, 0.00022, 0.00021, 0.00012], 'label_names': ['pl110', 'pl100', 'pl120', 'p26', 'pm10'], 'filename': 'PaddleClas/deploy/images/PULC/traffic_sign/100999_83928.jpg'}]
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,96 @@
|
|||
## 超轻量图像分类方案PULC
|
||||
### 0. PULC方案简介
|
||||
图像分类是计算机视觉的基础算法之一，是企业应用中最常见的算法，也是许多CV应用的重要组成部分。
|
||||
近年来，骨干网络模型发展迅速，Imagenet的精度纪录被不断刷新。然而，这些模型在实用场景的表现有时却不尽如人意。
|
||||
一方面，精度高的模型往往体积大，运算慢，常常难以满足实际部署需求；另一方面，选择了合适的模型之后，往往还需要经验丰富的工程师进行调参，
|
||||
费时费力。PaddleClas为了解决企业应用难题，让分类模型的训练和调参更加容易，总结推出了实用轻量图像分类解决方案PULC。
|
||||
PULC融合了骨干网络、数据增广、蒸馏等多种前沿算法，可以自动训练得到轻量且高精度的图像分类模型。
|
||||
方案在人、车、OCR等方向的多个场景中均验证有效，用超轻量模型就可实现与SwinTransformer模型接近的精度，预测速度提高50倍。
|
||||
<div align="center">
|
||||
<img src="https://user-images.githubusercontent.com/19523330/172054976-e12d2c9b-439f-469d-b520-56bb5c3e6215.png"/>
|
||||
</div>
|
||||
|
||||
方案主要包括4部分，分别是：PP-LCNet轻量级骨干网络、SSLD预训练权重、数据增强策略集成和SKL-UGI知识蒸馏算法。此外，我们还采用了超参搜索的方法，高效优化训练中的超参数。
|
||||
下面，我们以有人/无人场景为例，对方案进行说明。
|
||||
|
||||
**注**：针对一些特定场景，我们提供了基础的训练文档供参考，例如[有人/无人分类模型](PULC_person_exists.md)等，您可以在[这里]()找到这些文档。
|
||||
如果这些文档中的方法不能满足您的需求，或者您需要自定义训练任务，您可以参考本文档。
|
||||
|
||||
### 1. 数据准备
|
||||
#### 1.1 数据集格式说明
|
||||
|
||||
PaddleClas 使用 `txt` 格式文件指定训练集和测试集，以有人无人场景为例，其中 `train_list.txt` 和 `val_list.txt` 的格式形如：
|
||||
|
||||
```shell
|
||||
# 每一行采用"空格"分隔图像路径与标注
|
||||
train/1.jpg 0
|
||||
train/10.jpg 1
|
||||
...
|
||||
```
|
||||
如果您想获取更多常用分类数据集的信息，可以参考文档[常见分类说明](../data_preparation/classification_dataset.md)。
|
||||
|
||||
// todo@cuicheng v2.4.1 1.2有人无人场景数据获取代码。整理obj365数据提取的数据并说明。
|
||||
|
||||
|
||||
#### 1.2 标注文件生成
|
||||
如果您已经有实际场景中的数据，那么按照上节的格式进行标注即可。这里，我们提供了一个快速生成数据的脚本，您只需要将不同类别的数据分别放在文件夹中，运行脚本即可生成标注文件。
|
||||
// todo 数据脚本。
|
||||
|
||||
### 2. 使用标准分类配置进行训练
|
||||
#### 2.1 骨干网络PP-LCNet
|
||||
PULC采用了轻量骨干网络PP-LCNet，相比同精度竞品速度快50%，您可以在[这里](../models/PP-LCNet.md)找到详细介绍。
|
||||
直接使用PP-LCNet训练的命令为：
|
||||
|
||||
**todo**
|
||||
|
||||
为了方便性能对比，我们也提供了大模型SwinTransformer和轻量模型MobileNet的配置文件，您可以使用命令训练：
|
||||
|
||||
**todo**
|
||||
|
||||
训练得到的模型精度对比如下表。从中可以看出，LCNet的速度比SwinTransformer快很多，但是精度也略低。
|
||||
下面我们通过一系列优化来提高PP-LCNet模型的精度。
|
||||
|
||||
#### 2.2 SSLD预训练权重
|
||||
SSLD是百度自研的半监督蒸馏算法，在ImageNet数据集上，模型精度可以提升3-7个点，您可以在[这里](../algorithm_introduction/#2)找到详细介绍。
|
||||
我们发现，使用SSLD预训练权重，可以提升应用分类模型的精度。此外，使用SSLD预训练权重也有助于其他策略精度提升。
|
||||
此外，根据**todo**，在训练中使用略低一点的分辨率，可以有效提升模型精度。同时，我们也对学习率进行了优化。
|
||||
基于以上三点改进，我们训练得到模型精度为**todo**，提升**todo**。
|
||||
|
||||
#### 2.3 EDA数据增广策略
|
||||
数据增广是视觉算法中常用的优化策略，可以对模型精度有明显提升。除了传统的RandomCrop，RandomFlip等方法之外，我们还应用了RandomAugment和RandomErasing。
|
||||
您可以在[这里](../advanced_tutorials/DataAugmentation.md)找到详细介绍。
|
||||
由于这两种数据增强对图片的修改较大，使任务变难，在一些小数据集上可能会导致模型欠拟合，我们将这两种方法启用的概率设为10%。
|
||||
基于以上改进，我们训练得到模型精度为**todo**，提升**todo**。
|
||||
|
||||
#### 2.4 SKL-UGI模型蒸馏
|
||||
模型蒸馏是一种可以有效提升小模型精度的方法，您可以在[这里](todo@ruoyu)找到详细介绍。
|
||||
我们选择ResNet101作为教师模型进行蒸馏。
|
||||
**todo @cuicheng，对lr_mult进行说明**
|
||||
基于以上改进，我们训练得到模型精度为**todo**，提升:**todo。
|
||||
#### 2.5 总结
|
||||
经过以上方法优化，PP-LCNet最终精度达到**todo**，达到了大模型的精度水平。我们将实验结果总结如下表：
|
||||
**todo**
|
||||
我们在其他9个场景中也使用了同样的优化策略，得到如下结果：
|
||||
**todo**
|
||||
|
||||
从结果可以看出，PULC优化方法在多个应用场景中均可提升模型精度。虽然并非每种方法都有正向收益，但是使用PULC可以大大减少模型优化的工作量，快速得到精度较高的模型。
|
||||
|
||||
### 3. 超参搜索
|
||||
在上述训练过程中，我们调节了学习率、数据增广方法开启概率、分阶段学习率倍数等参数。
|
||||
这些参数在不同场景中最优值可能并不相同。我们提供了一个快速超参搜索的脚本，将超参调优的过程自动化。
|
||||
这个脚本会遍历搜索值列表中的参数来替代默认配置中的参数，依次训练，最终选择精度最高的模型所对应的参数作为搜索结果。
|
||||
|
||||
#### 3.1 基于默认配置搜索
|
||||
配置文件[search.yaml](todo)定义了有人/无人场景超参搜索的配置，使用命令**todo**，可以使用默认的超参数搜索配置进行训练，最终可得训练结果为：
|
||||
**todo**
|
||||
#### 3.2 自定义搜索配置
|
||||
您也可以根据训练结果或调参经验，修改超参搜索的配置。
|
||||
修改**todo**字段，可以修改学习率搜索值列表；
|
||||
|
||||
修改**todo**字段，可以修改RandAugment开启概率的搜索值列表；
|
||||
|
||||
修改**todo**字段，可以修改RnadomErasing开启概率的搜索值列表；
|
||||
|
||||
修改**todo**字段，可以修改lr_mult搜索值列表；
|
||||
|
||||
修改**todo**字段，可以修改教师模型的搜索列表。
|
||||
|
|
@ -39,20 +39,20 @@
|
|||
|
||||
该案例提供了用户使用 PaddleClas 的超轻量图像分类方案（PULC，Practical Ultra Lightweight Classification）快速构建轻量级、高精度、可落地的车辆属性识别模型。该模型可以广泛应用于车辆识别、道路监控等场景。
|
||||
|
||||
下表列出了不同车辆属性识别模型的相关指标，前两行展现了使用 Res2Net200_vd_26w_4s 和 MobileNetV3_large_x1_0 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
下表列出了不同车辆属性识别模型的相关指标，前两行展现了使用 Res2Net200_vd_26w_4s 和 MobileNetV3_small_x0_35 作为 backbone 训练得到的模型的相关指标，第三行至第六行依次展现了替换 backbone 为 PPLCNet_x1_0、使用 SSLD 预训练模型、使用 SSLD 预训练模型 + EDA 策略、使用 SSLD 预训练模型 + EDA 策略 + SKL-UGI 知识蒸馏策略训练得到的模型的相关指标。
|
||||
|
||||
|
||||
| 模型 | ma（%） | 延时（ms） | 存储（M） | 策略 |
|
||||
|-------|-----------|----------|---------------|---------------|
|
||||
| Res2Net200_vd_26w_4s | 91.36 | 79.46 | 293 | 使用ImageNet预训练模型 |
|
||||
| ResNet50 | 89.98 | 12.83 | 92 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_large_x1_0 | 89.77 | 5.09 | 23 | 使用ImageNet预训练模型 |
|
||||
| MobileNetV3_small_x0_35 | 87.41 | 2.91 | 2.8 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 89.57 | 2.36 | 8.2 | 使用ImageNet预训练模型 |
|
||||
| PPLCNet_x1_0 | 90.07 | 2.36 | 8.2 | 使用SSLD预训练模型 |
|
||||
| PPLCNet_x1_0 | 90.59 | 2.36 | 8.2 | 使用SSLD预训练模型+EDA策略|
|
||||
| <b>PPLCNet_x1_0<b> | <b>90.81<b> | <b>2.36<b> | <b>8.2<b> | 使用SSLD预训练模型+EDA策略+SKL-UGI知识蒸馏策略|
|
||||
|
||||
从表中可以看出，backbone 为 Res2Net200_vd_26w_4s 时精度较高，但是推理速度较慢。将 backbone 替换为轻量级模型 MobileNetV3_large_x1_0 后，速度可以大幅提升，但是精度下降明显。将 backbone 替换为 PPLCNet_x1_0 时，精度低0.2%，但是速度提升 1 倍左右。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升约 0.5%，进一步地，当融合EDA策略后，精度可以再提升 0.52%，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.23%。此时，PPLCNet_x1_0 的精度与 Res2Net200_vd_26w_4s 仅相差0.55%，但是速度快32倍。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
从表中可以看出，backbone 为 Res2Net200_vd_26w_4s 时精度较高，但是推理速度较慢。将 backbone 替换为轻量级模型 MobileNetV3_small_x0_35 后，速度可以大幅提升，但是精度下降明显。将 backbone 替换为 PPLCNet_x1_0 时，精度提升 2.16%，同时速度也提升 23% 左右。在此基础上，使用 SSLD 预训练模型后，在不改变推理速度的前提下，精度可以提升约 0.5%，进一步地，当融合EDA策略后，精度可以再提升 0.52%，最后，在使用 SKL-UGI 知识蒸馏后，精度可以继续提升 0.23%。此时，PPLCNet_x1_0 的精度与 Res2Net200_vd_26w_4s 仅相差0.55%，但是速度快32倍。关于 PULC 的训练方法和推理部署方法将在下面详细介绍。
|
||||
|
||||
**备注：**
|
||||
|
||||
|
|
@ -63,8 +63,48 @@
|
|||
|
||||
## 2. 模型快速体验
|
||||
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 安装 paddleclas
|
||||
|
||||
使用如下命令快速安装 paddlepaddle, paddleclas
|
||||
|
||||
```bash
|
||||
pip3 install paddlepaddle paddleclas
|
||||
```
|
||||
（pip方式，待补充）
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 预测
|
||||
|
||||
* 使用命令行快速预测
|
||||
|
||||
```bash
|
||||
paddleclas --model_name vehicle_attribute --infer_imgs PaddleClas/deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg
|
||||
```
|
||||
|
||||
结果如下：
|
||||
```
|
||||
>>> result
|
||||
attributes: Color: (yellow, prob: 0.9893476963043213), Type: (hatchback, prob: 0.9734097719192505), output: [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], filename: PaddleClas/deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg
|
||||
ppcls INFO: Predict complete!
|
||||
```
|
||||
|
||||
**备注**： 更换其他预测的数据时，只需要改变 `--infer_imgs=xx` 中的字段即可，支持传入整个文件夹。
|
||||
|
||||
|
||||
* 在 Python 代码中预测
|
||||
```python
|
||||
import paddleclas
|
||||
model = paddleclas.PaddleClas(model_name="vehicle_attribute")
|
||||
result = model.predict(input_data="PaddleClas/deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg")
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
**备注**：`model.predict()` 为可迭代对象（`generator`），因此需要使用 `next()` 函数或 `for` 循环对其迭代调用。每次调用将以 `batch_size` 为单位进行一次预测，并返回预测结果, 默认 `batch_size` 为 1，如果需要更改 `batch_size`，实例化模型时，需要指定 `batch_size`，如 `model = paddleclas.PaddleClas(model_name="person_exists", batch_size=2)`, 使用默认的代码返回结果示例如下：
|
||||
|
||||
```
|
||||
result
|
||||
[{'attributes': 'Color: (yellow, prob: 0.9893476963043213), Type: (hatchback, prob: 0.9734097719192505)', 'output': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], 'filename': 'PaddleClas/deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg'}]
|
||||
```
|
||||
|
||||
<a name="3"></a>
|
||||
|
|
@ -94,10 +134,10 @@
|
|||
部分数据可视化如下所示。
|
||||
|
||||
<div align="center">
|
||||
<img src="../../images/PULC/docs/vehicle_attr_data_demo.png" width = "500" />
|
||||
<img src="../../images/PULC/docs/vehicle_attribute_data_demo.png" width = "500" />
|
||||
</div>
|
||||
|
||||
首先从[VeRi数据集官网](https://www.v7labs.com/open-datasets/veri-dataset)中申请并下载数据，放在PaddleClas的`dataset`目录下，数据集目录名为`VeRi`，使用下面的命令进入该文件夹。
|
||||
首先从[VeRi数据集官网](https://www.v7labs.com/open-datasets/veri-dataset)中申请并下载数据，放在PaddleClas的`dataset`目录下，数据集目录名为`VeRi`，使用下面的命令进入该文件夹。
|
||||
|
||||
```shell
|
||||
cd PaddleClas/dataset/VeRi/
|
||||
|
|
@ -172,17 +212,17 @@ VeRi
|
|||
### 3.3 模型训练
|
||||
|
||||
|
||||
在 `ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml` 中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
在 `ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml` 中提供了基于该场景的训练配置，可以通过如下脚本启动训练：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml
|
||||
```
|
||||
|
||||
验证集的最佳指标在 `90.07%` 左右（数据集较小，一般有0.3%左右的波动）。
|
||||
验证集的最佳指标在 `90.59%` 左右（数据集较小，一般有0.3%左右的波动）。
|
||||
|
||||
|
||||
<a name="3.4"></a>
|
||||
|
|
@ -193,7 +233,7 @@ python3 -m paddle.distributed.launch \
|
|||
|
||||
```bash
|
||||
python3 tools/eval.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml \
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"
|
||||
```
|
||||
|
||||
|
|
@ -207,21 +247,21 @@ python3 tools/eval.py \
|
|||
|
||||
```bash
|
||||
python3 tools/infer.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml \
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model
|
||||
```
|
||||
|
||||
输出结果如下：
|
||||
|
||||
```
|
||||
[{'attr': 'Color: (yellow, prob: 0.9893478155136108), Type: (hatchback, prob: 0.9734100103378296)', 'pred': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], 'file_name': './deploy/images/PULC/vehicle_attr/0002_c002_00030670_0.jpg'}]
|
||||
[{'attr': 'Color: (yellow, prob: 0.9893478155136108), Type: (hatchback, prob: 0.9734100103378296)', 'pred': [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0], 'file_name': './deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg'}]
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
* 这里`-o Global.pretrained_model="output/PPLCNet_x1_0/best_model"` 指定了当前最佳权重所在的路径，如果指定其他权重，只需替换对应的路径即可。
|
||||
|
||||
* 默认是对 `./deploy/images/PULC/vehicle_attr/0002_c002_00030670_0.jpg` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
* 默认是对 `./deploy/images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg` 进行预测，此处也可以通过增加字段 `-o Infer.infer_imgs=xxx` 对其他图片预测。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
|
|
@ -237,14 +277,14 @@ SKL-UGI 知识蒸馏是 PaddleClas 提出的一种简单有效的知识蒸馏方
|
|||
|
||||
#### 4.1.1 教师模型训练
|
||||
|
||||
复用 `ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml` 中的超参数，训练教师模型，训练脚本如下：
|
||||
复用 `ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml` 中的超参数，训练教师模型，训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml \
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Arch.name=ResNet101_vd
|
||||
```
|
||||
|
||||
|
|
@ -254,14 +294,14 @@ python3 -m paddle.distributed.launch \
|
|||
|
||||
#### 4.1.2 蒸馏训练
|
||||
|
||||
配置文件`ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0_distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型。训练脚本如下：
|
||||
配置文件`ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0_distillation.yaml`提供了`SKL-UGI知识蒸馏策略`的配置。该配置将`ResNet101_vd`当作教师模型，`PPLCNet_x1_0`当作学生模型。训练脚本如下：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" \
|
||||
tools/train.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0_distillation.yaml \
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0_distillation.yaml \
|
||||
-o Arch.models.0.Teacher.pretrained=output/ResNet101_vd/best_model
|
||||
```
|
||||
|
||||
|
|
@ -296,14 +336,14 @@ Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端
|
|||
|
||||
```bash
|
||||
python3 tools/export_model.py \
|
||||
-c ./ppcls/configs/PULC/vehicle_attr/PPLCNet_x1_0.yaml \
|
||||
-c ./ppcls/configs/PULC/vehicle_attribute/PPLCNet_x1_0.yaml \
|
||||
-o Global.pretrained_model=output/DistillationModel/best_model_student \
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attr_infer
|
||||
-o Global.save_inference_dir=deploy/models/PPLCNet_x1_0_vehicle_attribute_infer
|
||||
```
|
||||
执行完该脚本后会在 `deploy/models/` 下生成 `PPLCNet_x1_0_vehicle_attr_infer` 文件夹，`models` 文件夹下应有如下文件结构：
|
||||
执行完该脚本后会在 `deploy/models/` 下生成 `PPLCNet_x1_0_vehicle_attributeibute_infer` 文件夹，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── PPLCNet_x1_0_vehicle_attr_infer
|
||||
├── PPLCNet_x1_0_vehicle_attribute_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
|
|
@ -320,13 +360,13 @@ python3 tools/export_model.py \
|
|||
```
|
||||
cd deploy/models
|
||||
# 下载 inference 模型并解压
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/vehicle_attr_infer.tar && tar -xf vehicle_attr_infer.tar
|
||||
wget https://paddleclas.bj.bcebos.com/models/PULC/vehicle_attribute_infer.tar && tar -xf vehicle_attribute_infer.tar
|
||||
```
|
||||
|
||||
解压完毕后，`models` 文件夹下应有如下文件结构：
|
||||
|
||||
```
|
||||
├── vehicle_attr_infer
|
||||
├── vehicle_attribute_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
|
|
@ -347,13 +387,13 @@ wget https://paddleclas.bj.bcebos.com/models/PULC/vehicle_attr_infer.tar && tar
|
|||
cd ../
|
||||
```
|
||||
|
||||
运行下面的命令，对图像 `./images/PULC/vehicle_attr/0002_c002_00030670_0.jpg` 进行车辆属性识别。
|
||||
运行下面的命令，对图像 `./images/PULC/vehicle_attribute/0002_c002_00030670_0.jpg` 进行车辆属性识别。
|
||||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attr/inference_vehicle_attr.yaml -o Global.use_gpu=True
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.use_gpu=True
|
||||
# 使用下面的命令使用 CPU 进行预测
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attr/inference_vehicle_attr.yaml -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.use_gpu=False
|
||||
```
|
||||
|
||||
输出结果如下。
|
||||
|
|
@ -371,7 +411,7 @@ predict output: [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
|
|||
|
||||
```shell
|
||||
# 使用下面的命令使用 GPU 进行预测，如果希望使用 CPU 预测，可以在命令后面添加 -o Global.use_gpu=False
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attr/inference_vehicle_attr.yaml -o Global.infer_imgs="./images/PULC/vehicle_attr/"
|
||||
python3.7 python/predict_cls.py -c configs/PULC/vehicle_attribute/inference_vehicle_attribute.yaml -o Global.infer_imgs="./images/PULC/vehicle_attribute/"
|
||||
```
|
||||
|
||||
终端中会输出该文件夹内所有图像的属性识别结果，如下所示。
|
||||
|
|
@ -11,8 +11,9 @@
|
|||
- [1.2 PaddleClas支持的知识蒸馏算法](#1.2)
|
||||
- [1.2.1 SSLD](#1.2.1)
|
||||
- [1.2.2 DML](#1.2.2)
|
||||
- [1.2.3 AFD](#1.2.3)
|
||||
- [1.2.4 DKD](#1.2.4)
|
||||
- [1.2.3 UDML](#1.2.3)
|
||||
- [1.2.4 AFD](#1.2.4)
|
||||
- [1.2.5 DKD](#1.2.5)
|
||||
- [2. 使用方法](#2)
|
||||
- [2.1 环境配置](#2.1)
|
||||
- [2.2 数据准备](#2.2)
|
||||
|
|
@ -196,9 +197,80 @@ Loss:
|
|||
|
||||
<a name='1.2.3'></a>
|
||||
|
||||
#### 1.2.3 AFD
|
||||
#### 1.2.3 UDML
|
||||
|
||||
##### 1.2.3.1 AFD 算法介绍
|
||||
##### 1.2.3.1 UDML 算法介绍
|
||||
|
||||
论文信息：
|
||||
|
||||
UDML 是百度飞桨视觉团队提出的无需依赖教师模型的知识蒸馏算法，它基于DML进行改进，在蒸馏的过程中，除了考虑两个模型的输出信息，也考虑两个模型的中间层特征信息，从而进一步提升知识蒸馏的精度。更多关于UDML的说明与应用，请参考[PP-ShiTu论文](https://arxiv.org/abs/2111.00775)以及[PP-OCRv3论文](https://arxiv.org/abs/2109.03144)。
|
||||
|
||||
|
||||
|
||||
在ImageNet1k公开数据集上，效果如下所示。
|
||||
|
||||
| 策略 | 骨干网络 | 配置文件 | Top-1 acc | 下载链接 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| baseline | PPLCNet_x2_5 | [PPLCNet_x2_5.yaml](../../../ppcls/configs/ImageNet/PPLCNet/PPLCNet_x2_5.yaml) | 74.93% | - |
|
||||
| UDML | PPLCNet_x2_5 | [PPLCNet_x2_5_dml.yaml](../../../ppcls/configs/ImageNet/Distillation/PPLCNet_x2_5_udml.yaml) | 76.74%(**+1.81%**) | - |
|
||||
|
||||
|
||||
##### 1.2.3.2 UDML 配置
|
||||
|
||||
|
||||
```yaml
|
||||
Arch:
|
||||
name: "DistillationModel"
|
||||
class_num: &class_num 1000
|
||||
# if not null, its lengths should be same as models
|
||||
pretrained_list:
|
||||
# if not null, its lengths should be same as models
|
||||
freeze_params_list:
|
||||
- False
|
||||
- False
|
||||
models:
|
||||
- Teacher:
|
||||
name: PPLCNet_x2_5
|
||||
class_num: *class_num
|
||||
pretrained: False
|
||||
# return_patterns表示除了返回输出的logits，也会返回对应名称的中间层feature map
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
- Student:
|
||||
name: PPLCNet_x2_5
|
||||
class_num: *class_num
|
||||
pretrained: False
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- DistillationGTCELoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_names: ["Student", "Teacher"]
|
||||
- DistillationDMLLoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
- DistillationDistanceLoss: # 基于蒸馏结果的距离loss，这里默认使用l2 loss计算block5之间的损失函数
|
||||
weight: 1.0
|
||||
key: "blocks5"
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
```
|
||||
|
||||
**注意(：** 上述在网络中指定`return_patterns`，返回中间层特征的功能是基于TheseusLayer，更多关于TheseusLayer的使用说明，请参考：[TheseusLayer 使用说明](./theseus_layer.md)。
|
||||
|
||||
|
||||
<a name='1.2.4'></a>
|
||||
|
||||
#### 1.2.4 AFD
|
||||
|
||||
##### 1.2.4.1 AFD 算法介绍
|
||||
|
||||
论文信息：
|
||||
|
||||
|
|
@ -220,7 +292,7 @@ AFD提出在蒸馏的过程中，利用基于注意力的元网络学习特征
|
|||
|
||||
注意：这里为了与论文的训练配置保持对齐，设置训练的迭代轮数为100epoch，因此baseline精度低于PaddleClas中开源出的模型精度（71.0%）
|
||||
|
||||
##### 1.2.3.2 AFD 配置
|
||||
##### 1.2.4.2 AFD 配置
|
||||
|
||||
AFD配置如下所示。在模型构建Arch字段中，需要同时定义学生模型与教师模型，固定教师模型的权重。这里需要对从教师模型获取的特征进行变换，进而与学生模型进行损失函数的计算。在损失函数Loss字段中，需要定义`DistillationKLDivLoss`（学生与教师之间的KL-Div loss）、`AFDLoss`（学生与教师之间的AFD loss）以及`DistillationGTCELoss`（学生与教师关于真值标签的CE loss），作为训练的损失函数。
|
||||
|
||||
|
|
@ -305,11 +377,11 @@ Loss:
|
|||
|
||||
**注意(：** 上述在网络中指定`return_patterns`，返回中间层特征的功能是基于TheseusLayer，更多关于TheseusLayer的使用说明，请参考：[TheseusLayer 使用说明](./theseus_layer.md)。
|
||||
|
||||
<a name='1.2.4'></a>
|
||||
<a name='1.2.5'></a>
|
||||
|
||||
#### 1.2.4 DKD
|
||||
#### 1.2.5 DKD
|
||||
|
||||
##### 1.2.4.1 DKD 算法介绍
|
||||
##### 1.2.5.1 DKD 算法介绍
|
||||
|
||||
论文信息：
|
||||
|
||||
|
|
@ -330,7 +402,7 @@ DKD将蒸馏中常用的 KD Loss 进行了解耦成为Target Class Knowledge Dis
|
|||
| AFD | ResNet18 | [resnet34_distill_resnet18_dkd.yaml](../../../ppcls/configs/ImageNet/Distillation/resnet34_distill_resnet18_dkd.yaml) | 72.59%(**+1.79%**) | - |
|
||||
|
||||
|
||||
##### 1.2.4.2 DKD 配置
|
||||
##### 1.2.5.2 DKD 配置
|
||||
|
||||
DKD 配置如下所示。在模型构建Arch字段中，需要同时定义学生模型与教师模型，教师模型固定参数，且需要加载预训练模型。在损失函数Loss字段中，需要定义`DistillationDKDLoss`（学生与教师之间的DKD loss）以及`DistillationGTCELoss`（学生与教师关于真值标签的CE loss），作为训练的损失函数。
|
||||
|
||||
|
|
|
|||
|
|
@ -1,182 +1,247 @@
|
|||
简体中文|[English](../../en/image_recognition_pipeline/feature_extraction_en.md)
|
||||
# 特征提取
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 简介](#1)
|
||||
- [2. 网络结构](#2)
|
||||
- [3. 通用识别模型](#3)
|
||||
- [4. 自定义特征提取](#4)
|
||||
- [4.1 数据准备](#4.1)
|
||||
- [4.2 模型训练](#4.2)
|
||||
- [4.3 模型评估](#4.3)
|
||||
- [4.4 模型推理](#4.4)
|
||||
- [4.4.1 导出推理模型](#4.4.1)
|
||||
- [4.4.2 获取特征向量](#4.4.2)
|
||||
- [1. 摘要](#1-摘要)
|
||||
- [2. 介绍](#2-介绍)
|
||||
- [3. 方法](#3-方法)
|
||||
- [3.1 Backbone](#31-backbone)
|
||||
- [3.2 Neck](#32-neck)
|
||||
- [3.3 Head](#33-head)
|
||||
- [3.4 Loss](#34-loss)
|
||||
- [4. 实验部分](#4-实验部分)
|
||||
- [5. 自定义特征提取](#5-自定义特征提取)
|
||||
- [5.1 数据准备](#51-数据准备)
|
||||
- [5.2 模型训练](#52-模型训练)
|
||||
- [5.3 模型评估](#53-模型评估)
|
||||
- [5.4 模型推理](#54-模型推理)
|
||||
- [5.4.1 导出推理模型](#541-导出推理模型)
|
||||
- [5.4.2 获取特征向量](#542-获取特征向量)
|
||||
- [6. 总结](#6-总结)
|
||||
- [7. 参考文献](#7-参考文献)
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 简介
|
||||
## 1. 摘要
|
||||
|
||||
特征提取是图像识别中的关键一环，它的作用是将输入的图片转化为固定维度的特征向量，用于后续的[向量检索](./vector_search.md)。好的特征需要具备相似度保持性，即在特征空间中，相似度高的图片对其特征相似度要比较高（距离比较近），相似度低的图片对，其特征相似度要比较小（距离比较远）。[Deep Metric Learning](../algorithm_introduction/metric_learning.md)用以研究如何通过深度学习的方法获得具有强表征能力的特征。
|
||||
特征提取是图像识别中的关键一环，它的作用是将输入的图片转化为固定维度的特征向量，用于后续的[向量检索](./vector_search.md)。一个好的特征需要具备“相似度保持性”，即相似度高的图片对，其特征的相似度也比较高（特征空间中的距离比较近），相似度低的图片对，其特征相似度要比较低（特征空间中的距离比较远）。为此[Deep Metric Learning](../algorithm_introduction/metric_learning.md)领域内提出了不少方法用以研究如何通过深度学习来获得具有强表征能力的特征。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 网络结构
|
||||
## 2. 介绍
|
||||
|
||||
为了图像识别任务的灵活定制，我们将整个网络分为 Backbone、 Neck、 Head 以及 Loss 部分，整体结构如下图所示:
|
||||
|
||||
图中各个模块的功能为:
|
||||
|
||||
- **Backbone**: 指定所使用的骨干网络。 值得注意的是，PaddleClas 提供的基于 ImageNet 的预训练模型，最后一层的输出为 1000，我们需要依据所需的特征维度定制最后一层的输出。
|
||||
- **Neck**: 用以特征增强及特征维度变换。这儿的 Neck，可以是一个简单的 Linear Layer，用来做特征维度变换；也可以是较复杂的 FPN 结构，用以做特征增强。
|
||||
- **Head**: 用来将 feature 转化为 logits。除了常用的 Fc Layer 外，还可以替换为 cosmargin, arcmargin, circlemargin 等模块。
|
||||
- **Loss**: 指定所使用的 Loss 函数。我们将 Loss 设计为组合 loss 的形式，可以方便地将 Classification Loss 和 Pair_wise Loss 组合在一起。
|
||||
- **Backbone**: 用于提取输入图像初步特征的骨干网络，一般由配置文件中的 [`Backbone`](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml#L26-L29) 以及 [`BackboneStopLayer`](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml#L30-L31) 字段共同指定。
|
||||
- **Neck**: 用以特征增强及特征维度变换。可以是一个简单的 FC Layer，用来做特征维度变换；也可以是较复杂的 FPN 结构，用以做特征增强，一般由配置文件中的 [`Neck`](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml#L32-L35)字段指定。
|
||||
- **Head**: 用来将 feature 转化为 logits，让模型在训练阶段能以分类任务的形式进行训练。除了常用的 FC Layer 外，还可以替换为 cosmargin, arcmargin, circlemargin 等模块，一般由配置文件中的 [`Head`](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml#L36-L41)字段指定。
|
||||
- **Loss**: 指定所使用的 Loss 函数。我们将 Loss 设计为组合 loss 的形式，可以方便地将 Classification Loss 和 Metric learning Loss 组合在一起，一般由配置文件中的 [`Loss`](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml#L44-L50)字段指定。
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 通用识别模型
|
||||
## 3. 方法
|
||||
|
||||
在 PP-Shitu 中, 我们采用 [PP_LCNet_x2_5](../models/PP-LCNet.md) 作为骨干网络 Neck 部分选用 Linear Layer, Head 部分选用 [ArcMargin](../../../ppcls/arch/gears/arcmargin.py)，Loss 部分选用 CELoss，详细的配置文件见[通用识别配置文件](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml)。其中，训练数据为如下 7 个公开数据集的汇总：
|
||||
### 3.1 Backbone
|
||||
|
||||
| 数据集 | 数据量 | 类别数 | 场景 | 数据集地址 |
|
||||
| :------------: | :-------------: | :-------: | :-------: | :--------: |
|
||||
| Aliproduct | 2498771 | 50030 | 商品 | [地址](https://retailvisionworkshop.github.io/recognition_challenge_2020/) |
|
||||
| GLDv2 | 1580470 | 81313 | 地标 | [地址](https://github.com/cvdfoundation/google-landmark) |
|
||||
| VeRI-Wild | 277797 | 30671 | 车辆 | [地址](https://github.com/PKU-IMRE/VERI-Wild)|
|
||||
| LogoDet-3K | 155427 | 3000 | Logo | [地址](https://github.com/Wangjing1551/LogoDet-3K-Dataset) |
|
||||
| iCartoonFace | 389678 | 5013 | 动漫人物 | [地址](http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d) |
|
||||
| SOP | 59551 | 11318 | 商品 | [地址](https://cvgl.stanford.edu/projects/lifted_struct/) |
|
||||
| Inshop | 25882 | 3997 | 商品 | [地址](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html) |
|
||||
| **Total** | **5M** | **185K** | ---- | ---- |
|
||||
Backbone 部分采用了 [PP_LCNet_x2_5](../models/PP-LCNet.md)，其针对Intel CPU端的性能优化探索了多个有效的结构设计方案，最终实现了在不增加推理时间的情况下，进一步提升模型的性能，最终大幅度超越现有的 SOTA 模型。
|
||||
|
||||
最终的模型效果如下表所示:
|
||||
### 3.2 Neck
|
||||
|
||||
| 模型 | Aliproduct | VeRI-Wild | LogoDet-3K | iCartoonFace | SOP | Inshop | Latency(ms) |
|
||||
| :----------: | :---------: | :-------: | :-------: | :--------: | :--------: | :--------: | :--------: |
|
||||
PP-LCNet-2.5x | 0.839 | 0.888 | 0.861 | 0.841 | 0.793 | 0.892 | 5.0
|
||||
Neck 部分采用了 [FC Layer](../../../ppcls/arch/gears/fc.py)，对 Backbone 抽取得到的特征进行降维，减少了特征存储的成本与计算量。
|
||||
|
||||
* 采用的评测指标为：`Recall@1`
|
||||
* 速度评测机器的 CPU 具体信息为：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`
|
||||
* 速度指标的评测条件为： 开启 MKLDNN, 线程数设置为 10
|
||||
* 预训练模型地址：[通用识别预训练模型](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/general_PPLCNet_x2_5_pretrained_v1.0.pdparams)
|
||||
### 3.3 Head
|
||||
|
||||
Head 部分选用 [ArcMargin](../../../ppcls/arch/gears/arcmargin.py)，在训练时通过指定margin，增大同类特征之间的角度差异再进行分类，进一步提升抽取特征的表征能力。
|
||||
|
||||
### 3.4 Loss
|
||||
|
||||
Loss 部分选用 [Cross entropy loss](../../../ppcls/loss/celoss.py)，在训练时以分类任务的损失函数来指导网络进行优化。详细的配置文件见[通用识别配置文件](../../../ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml)。
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 自定义特征提取
|
||||
## 4. 实验部分
|
||||
|
||||
自定义特征提取，是指依据自己的任务，重新训练特征提取模型。主要包含四个步骤：1）数据准备；2）模型训练；3）模型评估；4）模型推理。
|
||||
训练数据为如下 7 个公开数据集的汇总：
|
||||
|
||||
<a name="4.1"></a>
|
||||
| 数据集 | 数据量 | 类别数 | 场景 | 数据集地址 |
|
||||
| :----------: | :-----: | :------: | :------: | :--------------------------------------------------------------------------: |
|
||||
| Aliproduct | 2498771 | 50030 | 商品 | [地址](https://retailvisionworkshop.github.io/recognition_challenge_2020/) |
|
||||
| GLDv2 | 1580470 | 81313 | 地标 | [地址](https://github.com/cvdfoundation/google-landmark) |
|
||||
| VeRI-Wild | 277797 | 30671 | 车辆 | [地址](https://github.com/PKU-IMRE/VERI-Wild) |
|
||||
| LogoDet-3K | 155427 | 3000 | Logo | [地址](https://github.com/Wangjing1551/LogoDet-3K-Dataset) |
|
||||
| iCartoonFace | 389678 | 5013 | 动漫人物 | [地址](http://challenge.ai.iqiyi.com/detail?raceId=5def69ace9fcf68aef76a75d) |
|
||||
| SOP | 59551 | 11318 | 商品 | [地址](https://cvgl.stanford.edu/projects/lifted_struct/) |
|
||||
| Inshop | 25882 | 3997 | 商品 | [地址](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion.html) |
|
||||
| **Total** | **5M** | **185K** | ---- | ---- |
|
||||
|
||||
### 4.1 数据准备
|
||||
最终的模型效果如下表所示:
|
||||
|
||||
首先，需要基于任务定制自己的数据集。数据集格式参见[格式说明](https://github.com/PaddlePaddle/PaddleClas/blob/develop/docs/zh_CN/data_preparation/recognition_dataset.md#%E6%95%B0%E6%8D%AE%E9%9B%86%E6%A0%BC%E5%BC%8F%E8%AF%B4%E6%98%8E)。在启动模型训练之前，需要在配置文件中修改数据配置相关的内容, 主要包括数据集的地址以及类别数量。对应到配置文件中的位置如下所示：
|
||||
```
|
||||
Head:
|
||||
name: ArcMargin
|
||||
embedding_size: 512
|
||||
class_num: 185341 #此处表示类别数
|
||||
```
|
||||
```
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/ #此处表示train数据所在的目录
|
||||
cls_label_path: ./dataset/train_reg_all_data.txt #此处表示train数据集label文件的地址
|
||||
```
|
||||
```
|
||||
Query:
|
||||
| 模型 | Aliproduct | VeRI-Wild | LogoDet-3K | iCartoonFace | SOP | Inshop | Latency(ms) |
|
||||
| :-----------------------------: | :--------: | :-------: | :--------: | :----------: | :---: | :----: | :---------: |
|
||||
| GeneralRecognition_PPLCNet_x2_5 | 0.839 | 0.888 | 0.861 | 0.841 | 0.793 | 0.892 | 5.0 |
|
||||
|
||||
* 预训练模型地址：[通用识别预训练模型](https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/pretrain/general_PPLCNet_x2_5_pretrained_v1.0.pdparams)
|
||||
* 采用的评测指标为：`Recall@1`
|
||||
* 速度评测机器的 CPU 具体信息为：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`
|
||||
* 速度指标的评测条件为： 开启 MKLDNN, 线程数设置为 10
|
||||
|
||||
<a name="5"></a>
|
||||
|
||||
## 5. 自定义特征提取
|
||||
|
||||
自定义特征提取，是指依据自己的任务，重新训练特征提取模型。
|
||||
|
||||
下面基于`GeneralRecognition_PPLCNet_x2_5.yaml`配置文件，介绍主要的四个步骤：1）数据准备；2）模型训练；3）模型评估；4）模型推理
|
||||
|
||||
|
||||
<a name="5.1"></a>
|
||||
|
||||
### 5.1 数据准备
|
||||
|
||||
首先需要基于任务定制自己的数据集。数据集格式与文件结构详见[数据集格式说明](../data_preparation/recognition_dataset.md)。
|
||||
|
||||
准备完毕之后还需要在配置文件中修改数据配置相关的内容, 主要包括数据集的地址以及类别数量。对应到配置文件中的位置如下所示：
|
||||
|
||||
- 修改类别数：
|
||||
```yaml
|
||||
Head:
|
||||
name: ArcMargin
|
||||
embedding_size: 512
|
||||
class_num: 185341 # 此处表示类别数
|
||||
```
|
||||
- 修改训练数据集配置：
|
||||
```yaml
|
||||
Train:
|
||||
dataset:
|
||||
name: VeriWild
|
||||
image_root: ./dataset/Aliproduct/. #此处表示query数据集所在的目录
|
||||
cls_label_path: ./dataset/Aliproduct/val_list.txt. #此处表示query数据集label文件的地址
|
||||
```
|
||||
```
|
||||
Gallery:
|
||||
dataset:
|
||||
name: VeriWild
|
||||
image_root: ./dataset/Aliproduct/ #此处表示gallery数据集所在的目录
|
||||
cls_label_path: ./dataset/Aliproduct/val_list.txt. #此处表示gallery数据集label文件的地址
|
||||
```
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/ # 此处表示train数据所在的目录
|
||||
cls_label_path: ./dataset/train_reg_all_data.txt # 此处表示train数据集label文件的地址
|
||||
```
|
||||
- 修改评估数据集中query数据配置：
|
||||
```yaml
|
||||
Query:
|
||||
dataset:
|
||||
name: VeriWild
|
||||
image_root: ./dataset/Aliproduct/ # 此处表示query数据集所在的目录
|
||||
cls_label_path: ./dataset/Aliproduct/val_list.txt # 此处表示query数据集label文件的地址
|
||||
```
|
||||
- 修改评估数据集中gallery数据配置：
|
||||
```yaml
|
||||
Gallery:
|
||||
dataset:
|
||||
name: VeriWild
|
||||
image_root: ./dataset/Aliproduct/ # 此处表示gallery数据集所在的目录
|
||||
cls_label_path: ./dataset/Aliproduct/val_list.txt # 此处表示gallery数据集label文件的地址
|
||||
```
|
||||
|
||||
<a name="4.2"></a>
|
||||
<a name="5.2"></a>
|
||||
|
||||
### 4.2 模型训练
|
||||
### 5.2 模型训练
|
||||
|
||||
模型训练主要包括启动训练和断点恢复训练的功能
|
||||
|
||||
- 单机单卡训练
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python tools/train.py -c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
|
||||
```
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python3.7 tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
|
||||
```
|
||||
- 单机多卡训练
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
|
||||
```
|
||||
**注意:**
|
||||
配置文件中默认采用`在线评估`的方式，如果你想加快训练速度，去除`在线评估`，只需要在上述命令后面，增加 `-o eval_during_train=False`。训练完毕后，在 output 目录下会生成最终模型文件 `latest`，`best_model` 和训练日志文件 `train.log`。其中，`best_model` 用来存储当前评测指标下的最佳模型；`latest` 用来存储最新生成的模型, 方便在任务中断的情况下从断点位置启动训练。
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3.7 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml
|
||||
```
|
||||
**注意：**
|
||||
配置文件中默认采用`在线评估`的方式，如果你想加快训练速度，可以关闭`在线评估`功能，只需要在上述命令的后面，增加 `-o Global.eval_during_train=False`。
|
||||
|
||||
- 断点续训：
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.checkpoint="output/RecModel/latest"
|
||||
```
|
||||
训练完毕后，在 output 目录下会生成最终模型文件 `latest.pdparams`，`best_model.pdarams` 和训练日志文件 `train.log`。其中，`best_model` 保存了当前评测指标下的最佳模型，`latest` 用来保存最新生成的模型, 方便在任务中断的情况下从断点位置恢复训练。通过在上述训练命令的末尾加上`-o Global.checkpoint="path_to_resume_checkpoint"`即可从断点恢复训练，示例如下。
|
||||
|
||||
<a name="4.3"></a>
|
||||
- 单机单卡断点恢复训练
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python3.7 tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.checkpoint="output/RecModel/latest"
|
||||
```
|
||||
- 单机多卡断点恢复训练
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3.7 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/train.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.checkpoint="output/RecModel/latest"
|
||||
```
|
||||
|
||||
### 4.3 模型评估
|
||||
<a name="5.3"></a>
|
||||
|
||||
### 5.3 模型评估
|
||||
|
||||
除了训练过程中对模型进行的在线评估，也可以手动启动评估程序来获得指定的模型的精度指标。
|
||||
|
||||
- 单卡评估
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python tools/eval.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.pretrained_model="output/RecModel/best_model"
|
||||
```
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python3.7 tools/eval.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.pretrained_model="output/RecModel/best_model"
|
||||
```
|
||||
|
||||
- 多卡评估
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python3.7 -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/eval.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.pretrained_model="output/RecModel/best_model"
|
||||
```
|
||||
**注：** 建议使用多卡评估。该方式可以利用多卡并行计算快速得到全部数据的特征，能够加速评估的过程。
|
||||
|
||||
<a name="5.4"></a>
|
||||
|
||||
### 5.4 模型推理
|
||||
|
||||
推理过程包括两个步骤： 1）导出推理模型；2）模型推理以获取特征向量
|
||||
|
||||
#### 5.4.1 导出推理模型
|
||||
|
||||
首先需要将 `*.pdparams` 模型文件转换成 inference 格式，转换命令如下。
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python -m paddle.distributed.launch \
|
||||
--gpus="0,1,2,3" tools/eval.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.pretrained_model="output/RecModel/best_model"
|
||||
```
|
||||
**推荐：** 建议使用多卡评估。多卡评估方式可以利用多卡并行计算快速得到整体数据集的特征集合，能够加速评估的过程。
|
||||
|
||||
<a name="4.4"></a>
|
||||
|
||||
### 4.4 模型推理
|
||||
|
||||
推理过程包括两个步骤： 1)导出推理模型; 2)获取特征向量
|
||||
|
||||
<a name="4.4.1"></a>
|
||||
|
||||
#### 4.4.1 导出推理模型
|
||||
|
||||
```
|
||||
python tools/export_model.py \
|
||||
python3.7 tools/export_model.py \
|
||||
-c ppcls/configs/GeneralRecognition/GeneralRecognition_PPLCNet_x2_5.yaml \
|
||||
-o Global.pretrained_model="output/RecModel/best_model"
|
||||
```
|
||||
生成的推理模型位于 `inference` 目录，里面包含三个文件，分别为 `inference.pdmodel`、`inference.pdiparams`、`inference.pdiparams.info`。
|
||||
其中: `inference.pdmodel` 用来存储推理模型的结构, `inference.pdiparams` 和 `inference.pdiparams.info` 用来存储推理模型相关的参数信息。
|
||||
生成的推理模型默认位于 `PaddleClas/inference` 目录，里面包含三个文件，分别为 `inference.pdmodel`、`inference.pdiparams`、`inference.pdiparams.info`。
|
||||
其中`inference.pdmodel` 用来存储推理模型的结构, `inference.pdiparams` 和 `inference.pdiparams.info` 用来存储推理模型相关的参数信息。
|
||||
|
||||
<a name="4.4.2"></a>
|
||||
#### 5.4.2 获取特征向量
|
||||
|
||||
#### 4.4.2 获取特征向量
|
||||
使用上一步转换得到的 inference 格式模型，将输入图片转换为对应的特征向量，推理命令如下。
|
||||
|
||||
```
|
||||
```shell
|
||||
cd deploy
|
||||
python python/predict_rec.py \
|
||||
python3.7 python/predict_rec.py \
|
||||
-c configs/inference_rec.yaml \
|
||||
-o Global.rec_inference_model_dir="../inference"
|
||||
```
|
||||
得到的特征输出格式如下图所示：
|
||||
|
||||
|
||||
在实际使用过程中，单纯得到特征往往并不能够满足业务的需求。如果想进一步通过特征检索来进行图像识别，可以参照文档[向量检索](./vector_search.md)。
|
||||
在实际使用过程中，仅仅得到特征可能并不能满足业务需求。如果想进一步通过特征检索来进行图像识别，可以参照文档[向量检索](./vector_search.md)。
|
||||
|
||||
<a name="6"></a>
|
||||
|
||||
## 6. 总结
|
||||
|
||||
特征提取模块作为图像识别中的关键一环，在网络结构的设计，损失函数的选取上有很大的改进空间。不同的数据集类型有各自不同的特点，如行人重识别、商品识别、人脸识别数据集的分布、图片内容都不尽相同。学术界根据这些特点提出了各种各样的方法，如PCB、MGN、ArcFace、CircleLoss、TripletLoss等，围绕的还是增大类间差异、减少类内差异的最终目标，从而有效地应对各种真实场景数据。
|
||||
|
||||
<a name="7"></a>
|
||||
|
||||
## 7. 参考文献
|
||||
|
||||
1. [PP-LCNet: A Lightweight CPU Convolutional Neural Network](https://arxiv.org/pdf/2109.15099.pdf)
|
||||
2. [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://arxiv.org/abs/1801.07698)
|
||||
|
|
|
|||
|
|
@ -19,9 +19,13 @@
|
|||
- [3.3 配置文件改动和说明](#3.3)
|
||||
- [3.4 启动训练](#3.4)
|
||||
- [3.5 模型预测与调试](#3.5)
|
||||
- [3.6 模型导出与预测部署](#3.6)
|
||||
- [4. 模型推理部署](#4)
|
||||
- [4.1 推理模型准备](#4.1)
|
||||
- [4.2 基于python预测引擎推理](#4.2)
|
||||
- [4.3 其他推理方式](#4.3)
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
<a name="1"></a>
|
||||
|
||||
## 1. 数据集
|
||||
|
||||
|
|
@ -37,7 +41,7 @@
|
|||
|
||||
在实际训练的过程中，将所有数据集混合在一起。由于是主体检测，这里将所有标注出的检测框对应的类别都修改为 `前景` 的类别，最终融合的数据集中只包含 1 个类别，即前景。
|
||||
|
||||
<a name="2"></a>
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型选择
|
||||
|
||||
|
|
@ -55,7 +59,7 @@
|
|||
* 速度评测机器的 CPU 具体信息为：`Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz`，速度指标为开启 mkldnn，线程数设置为 10 测试得到。
|
||||
* 主体检测的预处理过程较为耗时，平均每张图在上述机器上的时间在 40~55 ms 左右，没有包含在上述的预测耗时统计中。
|
||||
|
||||
<a name="2.1"></a>
|
||||
<a name="2.1"></a>
|
||||
|
||||
### 2.1 轻量级主体检测模型
|
||||
|
||||
|
|
@ -72,7 +76,7 @@ PicoDet 由 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)
|
|||
|
||||
在轻量级主体检测任务中，为了更好地兼顾检测速度与效果，我们使用 PPLCNet_x2_5 作为主体检测模型的骨干网络，同时将训练与预测的图像尺度修改为了 640x640，其余配置与 [picodet_lcnet_1_5x_416_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/picodet/more_config/picodet_lcnet_1_5x_416_coco.yml) 完全一致。将数据集更换为自定义的主体检测数据集，进行训练，最终得到检测模型。
|
||||
|
||||
<a name="2.2"></a>
|
||||
<a name="2.2"></a>
|
||||
|
||||
### 2.2 服务端主体检测模型
|
||||
|
||||
|
|
@ -93,13 +97,13 @@ PP-YOLO 由 [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)
|
|||
|
||||
在服务端主体检测任务中，为了保证检测效果，我们使用 ResNet50vd-DCN 作为检测模型的骨干网络，使用配置文件 [ppyolov2_r50vd_dcn_365e_coco.yml](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml)，更换为自定义的主体检测数据集，进行训练，最终得到检测模型。
|
||||
|
||||
<a name="3"></a>
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练
|
||||
|
||||
本节主要介绍怎样基于 PaddleDetection，基于自己的数据集，训练主体检测模型。
|
||||
|
||||
<a name="3.1"></a>
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 环境准备
|
||||
|
||||
|
|
@ -116,7 +120,7 @@ pip install -r requirements.txt
|
|||
|
||||
更多安装教程，请参考: [安装文档](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/docs/tutorials/INSTALL_cn.md)
|
||||
|
||||
<a name="3.2"></a>
|
||||
<a name="3.2"></a>
|
||||
|
||||
### 3.2 数据准备
|
||||
|
||||
|
|
@ -128,7 +132,7 @@ pip install -r requirements.txt
|
|||
[{u'id': 1, u'name': u'foreground', u'supercategory': u'foreground'}]
|
||||
```
|
||||
|
||||
<a name="3.3"></a>
|
||||
<a name="3.3"></a>
|
||||
|
||||
### 3.3 配置文件改动和说明
|
||||
|
||||
|
|
@ -154,7 +158,7 @@ ppyolov2_reader.yml：主要说明数据读取器配置，如 batch size，并
|
|||
|
||||
此外，也可以根据实际情况，修改上述文件，比如，如果显存溢出，可以将 batch size 和学习率等比缩小等。
|
||||
|
||||
<a name="3.4"></a>
|
||||
<a name="3.4"></a>
|
||||
|
||||
### 3.4 启动训练
|
||||
|
||||
|
|
@ -198,7 +202,7 @@ python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/ppy
|
|||
|
||||
注意：如果遇到 "`Out of memory error`" 问题, 尝试在 `ppyolov2_reader.yml` 文件中调小 `batch_size`，同时等比例调小学习率。
|
||||
|
||||
<a name="3.5"></a>
|
||||
<a name="3.5"></a>
|
||||
|
||||
### 3.5 模型预测与调试
|
||||
|
||||
|
|
@ -211,9 +215,11 @@ python tools/infer.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml --infer
|
|||
|
||||
`--draw_threshold` 是个可选参数. 根据 [NMS](https://ieeexplore.ieee.org/document/1699659) 的计算，不同阈值会产生不同的结果 `keep_top_k` 表示设置输出目标的最大数量，默认值为 100，用户可以根据自己的实际情况进行设定。
|
||||
|
||||
<a name="3.6"></a>
|
||||
<a name="4"></a>
|
||||
## 4. 模型推理部署
|
||||
|
||||
### 3.6 模型导出与预测部署。
|
||||
<a name="4.1"></a>
|
||||
### 4.1 推理模型准备
|
||||
|
||||
执行导出模型脚本：
|
||||
|
||||
|
|
@ -225,15 +231,21 @@ python tools/export_model.py -c configs/ppyolo/ppyolov2_r50vd_dcn_365e_coco.yml
|
|||
|
||||
注意： `PaddleDetection` 导出的 inference 模型的文件格式为 `model.xxx`，这里如果希望与 PaddleClas 的 inference 模型文件格式保持一致，需要将其 `model.xxx` 文件修改为 `inference.xxx` 文件，用于后续主体检测的预测部署。
|
||||
|
||||
更多模型导出教程，请参考： [EXPORT_MODEL](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/deploy/EXPORT_MODEL.md)
|
||||
更多模型导出教程，请参考： [EXPORT_MODEL](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/EXPORT_MODEL.md)
|
||||
|
||||
最终，目录 `inference/ppyolov2_r50vd_dcn_365e_coco` 中包含 `inference.pdiparams`, `inference.pdiparams.info` 以及 `inference.pdmodel` 文件，其中 `inference.pdiparams` 为保存的 inference 模型权重文件，`inference.pdmodel` 为保存的 inference 模型结构文件。
|
||||
|
||||
<a name="4.2"></a>
|
||||
### 4.2 基于python预测引擎推理
|
||||
|
||||
导出模型之后，在主体检测与识别任务中，就可以将检测模型的路径更改为该 inference 模型路径，完成预测。
|
||||
|
||||
以商品识别为例，其配置文件为 [inference_product.yaml](../../../deploy/configs/inference_product.yaml)，修改其中的 `Global.det_inference_model_dir` 字段为导出的主体检测 inference 模型目录，参考[图像识别快速开始教程](../quick_start/quick_start_recognition.md)，即可完成商品检测与识别过程。
|
||||
|
||||
<a name="4.3"></a>
|
||||
### 4.3 其他推理方式
|
||||
其他推理方法，如C++推理部署、PaddleServing部署等请参考[检测模型推理部署](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/deploy/README.md)。
|
||||
|
||||
|
||||
### FAQ
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,21 @@
|
|||
# 向量检索
|
||||
|
||||
## 目录
|
||||
|
||||
- [1. 向量检索应用场景介绍](#1)
|
||||
- [2. 向量检索算法介绍](#2)
|
||||
- [2.1 HNSW](#2.1)
|
||||
- [2.2 IVF](#2.2)
|
||||
- [2.3 FLAT](#2.3)
|
||||
- [3. 检索库安装](#3)
|
||||
- [4. 使用及配置文档介绍](#4)
|
||||
- [4.1 建库及配置文件参数](#4.1)
|
||||
- [4.2 检索配置文件参数](#4.2)
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. 向量检索应用场景介绍
|
||||
|
||||
向量检索技术在图像识别、图像检索中应用比较广泛。其主要目标是，对于给定的查询向量，在已经建立好的向量库中，与库中所有的待查询向量，进行特征向量的相似度或距离计算，得到相似度排序。在图像识别系统中，我们使用 [Faiss](https://github.com/facebookresearch/faiss) 对此部分进行支持，具体信息请详查 [Faiss 官网](https://github.com/facebookresearch/faiss)。`Faiss` 主要有以下优势
|
||||
|
||||
- 适配性好：支持 Windos、Linux、MacOS 系统
|
||||
|
|
@ -20,17 +36,33 @@
|
|||
|
||||
--------------------------
|
||||
|
||||
## 目录
|
||||
<a name="2"></a>
|
||||
## 2. 使用的检索算法
|
||||
|
||||
- [1. 检索库安装](#1)
|
||||
- [2. 使用的检索算法](#2)
|
||||
- [3. 使用及配置文档介绍](#3)
|
||||
- [3.1 建库及配置文件参数](#3.1)
|
||||
- [3.2 检索配置文件参数](#3.2)
|
||||
目前 `PaddleClas` 中检索模块，支持三种检索算法**HNSW32**、**IVF**、**FLAT**。每种检索算法，满足不同场景。其中 `HNSW32` 为默认方法，此方法的检索精度、检索速度可以取得一个较好的平衡，具体算法介绍可以查看[官方文档](https://github.com/facebookresearch/faiss/wiki)。
|
||||
|
||||
<a name="1"></a>
|
||||
<a name="2.1"></a>
|
||||
### 2.1 HNSW方法
|
||||
|
||||
## 1. 检索库安装
|
||||
此方法为图索引方法，如下图所示，在建立索引的时候，分为不同的层，所以检索精度较高，速度较快，但是特征库只支持添加图像功能，不支持删除图像特征功能。基于图的向量检索算法在向量检索的评测中性能都是比较优异的。如果比较在乎检索算法的效率，而且可以容忍一定的空间成本，多数场景下比较推荐基于图的检索算法。而HNSW是一种典型的，应用广泛的图算法，很多分布式检索引擎都对HNSW算法进行了分布式改造，以应用于高并发，大数据量的线上查询。此方法为默认方法。
|
||||
<div align="center">
|
||||
<img src="../../images/algorithm_introduction/hnsw.png" width = "400" />
|
||||
</div>
|
||||
|
||||
<a name="2.2"></a>
|
||||
### 2.2 IVF
|
||||
|
||||
一种倒排索引检索方法。速度较快，但是精度略低。特征库支持增加、删除图像特征功能。IVF主要利用倒排的思想保存每个聚类中心下的向量，每次查询向量的时候找到最近的几个中心，分别搜索这几个中心下的向量。通过减小搜索范围，大大提升搜索效率。
|
||||
|
||||
<a name="2.3"></a>
|
||||
### 2.3 FLAT
|
||||
|
||||
暴力检索算法。精度最高，但是数据量大时，检索速度较慢。特征库支持增加、删除图像特征功能。
|
||||
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 检索库安装
|
||||
|
||||
`Faiss` 具体安装方法如下：
|
||||
|
||||
|
|
@ -40,27 +72,16 @@ pip install faiss-cpu==1.7.1post2
|
|||
|
||||
若使用时，不能正常引用，则 `uninstall` 之后，重新 `install`，尤其是 `windows` 下。
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 使用的检索算法
|
||||
<a name="4"></a>
|
||||
|
||||
目前 `PaddleClas` 中检索模块，支持如下三种检索算法
|
||||
## 4. 使用及配置文档介绍
|
||||
|
||||
- **HNSW32**: 一种图索引方法。检索精度较高，速度较快。但是特征库只支持添加图像功能，不支持删除图像特征功能。（默认方法）
|
||||
- **IVF**：倒排索引检索方法。速度较快，但是精度略低。特征库支持增加、删除图像特征功能。
|
||||
- **FLAT**： 暴力检索算法。精度最高，但是数据量大时，检索速度较慢。特征库支持增加、删除图像特征功能。
|
||||
涉及检索模块配置文件位于：`deploy/configs/` 下，其中 `inference_*.yaml` 是检索或者分类的推理配置文件,同时也是建立特征库的相关配置文件。
|
||||
|
||||
每种检索算法，满足不同场景。其中 `HNSW32` 为默认方法，此方法的检索精度、检索速度可以取得一个较好的平衡，具体算法介绍可以查看[官方文档](https://github.com/facebookresearch/faiss/wiki)。
|
||||
<a name="4.1"></a>
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 使用及配置文档介绍
|
||||
|
||||
涉及检索模块配置文件位于：`deploy/configs/` 下，其中 `build_*.yaml` 是建立特征库的相关配置文件，`inference_*.yaml` 是检索或者分类的推理配置文件。
|
||||
|
||||
<a name="3.1"></a>
|
||||
|
||||
### 3.1 建库及配置文件参数
|
||||
### 4.1 建库及配置文件参数
|
||||
|
||||
建库的具体操作如下：
|
||||
|
||||
|
|
@ -68,14 +89,14 @@ pip install faiss-cpu==1.7.1post2
|
|||
# 进入 deploy 目录
|
||||
cd deploy
|
||||
# yaml 文件根据需要改成自己所需的具体 yaml 文件
|
||||
python python/build_gallery.py -c configs/build_***.yaml
|
||||
python python/build_gallery.py -c configs/inference_***.yaml
|
||||
```
|
||||
|
||||
其中 `yaml` 文件的建库的配置如下，在运行时，请根据实际情况进行修改。建库操作会将根据 `data_file` 的图像列表，将 `image_root` 下的图像进行特征提取，并在 `index_dir` 下进行存储，以待后续检索使用。
|
||||
|
||||
其中 `data_file` 文件存储的是图像文件的路径和标签，每一行的格式为：`image_path label`。中间间隔以 `yaml` 文件中 `delimiter` 参数作为间隔。
|
||||
|
||||
关于特征提取的具体模型参数，可查看 `yaml` 文件。
|
||||
关于特征提取的具体模型参数，可查看 `yaml` 文件。注意下面的配置参数只列举了建立索引库相关部分。
|
||||
|
||||
```yaml
|
||||
# indexing engine config
|
||||
|
|
@ -88,6 +109,7 @@ IndexProcess:
|
|||
delimiter: "\t"
|
||||
dist_type: "IP"
|
||||
embedding_size: 512
|
||||
batch_size: 32
|
||||
```
|
||||
|
||||
- **index_method**：使用的检索算法。目前支持三种，HNSW32、IVF、Flat
|
||||
|
|
@ -98,23 +120,29 @@ IndexProcess:
|
|||
- **delimiter**：**data_file** 中每一行的间隔符
|
||||
- **dist_type**: 特征匹配过程中使用的相似度计算方式。例如 `IP` 内积相似度计算方式，`L2` 欧式距离计算方法
|
||||
- **embedding_size**：特征维度
|
||||
- **batch_size**：建立特征库时，特征提取的`batch_size`
|
||||
|
||||
<a name="3.2"></a>
|
||||
<a name="4.2"></a>
|
||||
|
||||
### 4.2 检索配置文件参数
|
||||
|
||||
### 3.2 检索配置文件参数
|
||||
|
||||
将检索的过程融合到 `PP-ShiTu` 的整体流程中，请参考 [README](../../../README_ch.md) 中 `PP-ShiTu 图像识别系统介绍` 部分。检索具体使用操作请参考[识别快速开始文档](../quick_start/quick_start_recognition.md)。
|
||||
|
||||
其中，检索部分配置如下，整体检索配置文件，请参考 `deploy/configs/inference_*.yaml` 文件。
|
||||
|
||||
注意：此部分参数只是列举了离线检索相关部分参数。
|
||||
|
||||
```yaml
|
||||
IndexProcess:
|
||||
index_dir: "./recognition_demo_data_v1.1/gallery_logo/index/"
|
||||
return_k: 5
|
||||
score_thres: 0.5
|
||||
hamming_radius: 100
|
||||
```
|
||||
|
||||
与建库配置文件不同，新参数主要如下：
|
||||
|
||||
- `return_k`: 检索结果返回 `k` 个结果
|
||||
- `score_thres`: 检索匹配的阈值
|
||||
- `hamming_radius`: 汉明距离半径。此参数只有在使用二值特征模型，`dist_type`设置为`hamming`时才能生效。具体二值特征模型使用方法请参考[哈希编码](./deep_hashing.md)
|
||||
|
|
|
|||
|
|
@ -6,10 +6,11 @@
|
|||
|
||||
## 目录
|
||||
|
||||
- [1. 图像分类推理](#1)
|
||||
- [2. 主体检测模型推理](#2)
|
||||
- [3. 特征提取模型推理](#3)
|
||||
- [4. 主体检测、特征提取和向量检索串联](#4)
|
||||
- [1. 图像分类模型推理](#1)
|
||||
- [2. PP-ShiTu模型推理](#2)
|
||||
- [2.1 主体检测模型推理](#2.1)
|
||||
- [2.2 特征提取模型推理](#2.2)
|
||||
- [2.3 PP-ShiTu PipeLine推理](#2.3)
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. 图像分类推理
|
||||
|
|
@ -42,7 +43,12 @@ python python/predict_cls.py -c configs/inference_cls.yaml
|
|||
* 如果你希望提升评测模型速度，使用 GPU 评测时，建议开启 TensorRT 加速预测，使用 CPU 评测时，建议开启 MKL-DNN 加速预测。
|
||||
|
||||
<a name="2"></a>
|
||||
## 2. 主体检测模型推理
|
||||
## 2. PP-ShiTu模型推理
|
||||
|
||||
PP-ShiTu整个Pipeline包含三部分：主体检测、特提取模型、特征检索。其中主体检测、特征模型可以单独推理使用。单独主体检测详见[2.1](#2.1)，特征提取模型单独推理详见[2.2](#2.2)， PP-ShiTu整体推理详见[2.3](#2.3)。
|
||||
|
||||
<a name="2.1"></a>
|
||||
### 2.1 主体检测模型推理
|
||||
|
||||
进入 PaddleClas 的 `deploy` 目录下：
|
||||
|
||||
|
|
@ -70,8 +76,8 @@ python python/predict_det.py -c configs/inference_det.yaml
|
|||
* `Global.use_gpu`： 是否使用 GPU 预测，默认为 `True`。
|
||||
|
||||
|
||||
<a name="3"></a>
|
||||
## 3. 特征提取模型推理
|
||||
<a name="2.2"></a>
|
||||
### 2.2 特征提取模型推理
|
||||
|
||||
下面以商品特征提取为例，介绍特征提取模型推理。首先进入 PaddleClas 的 `deploy` 目录下：
|
||||
|
||||
|
|
@ -90,7 +96,7 @@ tar -xf ./models/product_ResNet50_vd_aliproduct_v1.0_infer.tar -C ./models/
|
|||
|
||||
上述预测命令可以得到一个 512 维的特征向量，直接输出在在命令行中。
|
||||
|
||||
<a name="4"></a>
|
||||
## 4. 主体检测、特征提取和向量检索串联
|
||||
<a name="2.3"></a>
|
||||
### 2.3. PP-ShiTu PipeLine推理
|
||||
|
||||
主体检测、特征提取和向量检索的串联预测，可以参考图像识别[快速体验](../quick_start/quick_start_recognition.md)。
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ PaddleClas 支持 Python Whl 包方式进行预测，目前 Whl 包方式仅支
|
|||
- [4.6 对 `NumPy.ndarray` 格式数据进行预测](#4.6)
|
||||
- [4.7 保存预测结果](#4.7)
|
||||
- [4.8 指定 label name](#4.8)
|
||||
|
||||
|
||||
|
||||
<a name="1"></a>
|
||||
## 1. 安装 paddleclas
|
||||
|
|
@ -212,14 +212,14 @@ print(next(result))
|
|||
```python
|
||||
from paddleclas import PaddleClas
|
||||
clas = PaddleClas(model_name='ResNet50', save_dir='./output_pre_label/')
|
||||
infer_imgs = 'docs/images/whl/' # it can be infer_imgs folder path which contains all of images you want to predict.
|
||||
infer_imgs = 'docs/images/' # it can be infer_imgs folder path which contains all of images you want to predict.
|
||||
result=clas.predict(infer_imgs)
|
||||
print(next(result))
|
||||
```
|
||||
|
||||
* CLI
|
||||
```bash
|
||||
paddleclas --model_name='ResNet50' --infer_imgs='docs/images/whl/' --save_dir='./output_pre_label/'
|
||||
paddleclas --model_name='ResNet50' --infer_imgs='docs/images/' --save_dir='./output_pre_label/'
|
||||
```
|
||||
|
||||
<a name="4.8"></a>
|
||||
|
|
|
|||
|
|
@ -1,21 +1,37 @@
|
|||
# SwinTransformer
|
||||
---
|
||||
|
||||
-----
|
||||
## 目录
|
||||
|
||||
* [1. 概述](#1)
|
||||
* [2. 精度、FLOPS 和参数量](#2)
|
||||
* [3. 基于V100 GPU 的预测速度](#3)
|
||||
- [1. 模型介绍](#1)
|
||||
- [1.1 模型简介](#1.1)
|
||||
- [1.2 模型指标](#1.2)
|
||||
- [1.3 Benchmark](#1.3)
|
||||
- [1.3.1 基于 V100 GPU 的预测速度](#1.3.1)
|
||||
- [2. 模型快速体验](#2)
|
||||
- [3. 模型训练、评估和预测](#3)
|
||||
- [4. 模型推理部署](#4)
|
||||
- [4.1 推理模型准备](#4.1)
|
||||
- [4.2 基于 Python 预测引擎推理](#4.2)
|
||||
- [4.3 基于 C++ 预测引擎推理](#4.3)
|
||||
- [4.4 服务化部署](#4.4)
|
||||
- [4.5 端侧部署](#4.5)
|
||||
- [4.6 Paddle2ONNX 模型转换与预测](#4.6)
|
||||
|
||||
|
||||
<a name='1'></a>
|
||||
|
||||
## 1. 概述
|
||||
## 1. 模型介绍
|
||||
|
||||
### 1.1 模型简介
|
||||
|
||||
Swin Transformer 是一种新的视觉 Transformer 网络，可以用作计算机视觉领域的通用骨干网路。SwinTransformer 由移动窗口（shifted windows）表示的层次 Transformer 结构组成。移动窗口将自注意计算限制在非重叠的局部窗口上，同时允许跨窗口连接，从而提高了网络性能。[论文地址](https://arxiv.org/abs/2103.14030)。
|
||||
|
||||
<a name='2'></a>
|
||||
|
||||
## 2. 精度、FLOPS 和参数量
|
||||
### 1.2 模型指标
|
||||
|
||||
| Models | Top1 | Top5 | Reference<br>top1 | Reference<br>top5 | FLOPS<br>(G) | Params<br>(M) |
|
||||
| Models | Top1 | Top5 | Reference<br>top1 | Reference<br>top5 | FLOPs<br>(G) | Params<br>(M) |
|
||||
|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
|
||||
| SwinTransformer_tiny_patch4_window7_224 | 0.8069 | 0.9534 | 0.812 | 0.955 | 4.5 | 28 |
|
||||
| SwinTransformer_small_patch4_window7_224 | 0.8275 | 0.9613 | 0.832 | 0.962 | 8.7 | 50 |
|
||||
|
|
@ -32,17 +48,87 @@ Swin Transformer 是一种新的视觉 Transformer 网络，可以用作计算
|
|||
|
||||
<a name='3'></a>
|
||||
|
||||
## 3. 基于 V100 GPU 的预测速度
|
||||
### 1.3 Benchmark
|
||||
|
||||
| Models | Crop Size | Resize Short Size | FP32<br/>Batch Size=1<br/>(ms) | FP32<br/>Batch Size=4<br/>(ms) | FP32<br/>Batch Size=8<br/>(ms) |
|
||||
| ------------------------------------------------------- | --------- | ----------------- | ------------------------------ | ------------------------------ | ------------------------------ |
|
||||
| SwinTransformer_tiny_patch4_window7_224 | 224 | 256 | 6.59 | 9.68 | 16.32 |
|
||||
| SwinTransformer_small_patch4_window7_224 | 224 | 256 | 12.54 | 17.07 | 28.08 |
|
||||
| SwinTransformer_base_patch4_window7_224 | 224 | 256 | 13.37 | 23.53 | 39.11 |
|
||||
| SwinTransformer_base_patch4_window12_384 | 384 | 384 | 19.52 | 64.56 | 123.30 |
|
||||
| SwinTransformer_base_patch4_window7_224<sup>[1]</sup> | 224 | 256 | 13.53 | 23.46 | 39.13 |
|
||||
| SwinTransformer_base_patch4_window12_384<sup>[1]</sup> | 384 | 384 | 19.65 | 64.72 | 123.42 |
|
||||
| SwinTransformer_large_patch4_window7_224<sup>[1]</sup> | 224 | 256 | 15.74 | 38.57 | 71.49 |
|
||||
| SwinTransformer_large_patch4_window12_384<sup>[1]</sup> | 384 | 384 | 32.61 | 116.59 | 223.23 |
|
||||
#### 1.3.1 基于 V100 GPU 的预测速度
|
||||
|
||||
| Models | Size | Latency(ms)<br>bs=1 | Latency(ms)<br>bs=4 | Latency(ms)<br>bs=8 |
|
||||
|:--:|:--:|:--:|:--:|:--:|
|
||||
| SwinTransformer_tiny_patch4_window7_224 | 224 | 6.59 | 9.68 | 16.32 |
|
||||
| SwinTransformer_small_patch4_window7_224 | 224 | 12.54 | 17.07 | 28.08 |
|
||||
| SwinTransformer_base_patch4_window7_224 | 224 | 13.37 | 23.53 | 39.11 |
|
||||
| SwinTransformer_base_patch4_window12_384 | 384 | 19.52 | 64.56 | 123.30 |
|
||||
| SwinTransformer_base_patch4_window7_224<sup>[1]</sup> | 224 | 13.53 | 23.46 | 39.13 |
|
||||
| SwinTransformer_base_patch4_window12_384<sup>[1]</sup> | 384 | 19.65 | 64.72 | 123.42 |
|
||||
| SwinTransformer_large_patch4_window7_224<sup>[1]</sup> | 224 | 15.74 | 38.57 | 71.49 |
|
||||
| SwinTransformer_large_patch4_window12_384<sup>[1]</sup> | 384 | 32.61 | 116.59 | 223.23 |
|
||||
|
||||
[1]：基于 ImageNet22k 数据集预训练，然后在 ImageNet1k 数据集迁移学习得到。
|
||||
|
||||
**备注：** 精度类型为 FP32，推理过程使用 TensorRT。
|
||||
|
||||
|
||||
<a name="2"></a>
|
||||
|
||||
## 2. 模型快速体验
|
||||
|
||||
安装 paddlepaddle 和 paddleclas 即可快速对图片进行预测，体验方法可以参考[ResNet50 模型快速体验](./ResNet.md#2-模型快速体验)。
|
||||
|
||||
<a name="3"></a>
|
||||
|
||||
## 3. 模型训练、评估和预测
|
||||
|
||||
|
||||
此部分内容包括训练环境配置、ImageNet数据的准备、SwinTransformer 在 ImageNet 上的训练、评估、预测等内容。在 `ppcls/configs/ImageNet/SwinTransformer/` 中提供了 SwinTransformer 的训练配置，可以通过如下脚本启动训练：此部分内容可以参考[ResNet50 模型训练、评估和预测](./ResNet.md#3-模型训练评估和预测)。
|
||||
|
||||
**备注：** 由于 SwinTransformer 系列模型默认使用的 GPU 数量为 8 个，所以在训练时，需要指定8个GPU，如`python3 -m paddle.distributed.launch --gpus="0,1,2,3,4,5,6,7" tools/train.py -c xxx.yaml`, 如果使用 4 个 GPU 训练，默认学习率需要减小一半，精度可能有损。
|
||||
|
||||
|
||||
<a name="4"></a>
|
||||
|
||||
## 4. 模型推理部署
|
||||
|
||||
<a name="4.1"></a>
|
||||
|
||||
### 4.1 推理模型准备
|
||||
|
||||
Paddle Inference 是飞桨的原生推理库， 作用于服务器端和云端，提供高性能的推理能力。相比于直接基于预训练模型进行预测，Paddle Inference可使用 MKLDNN、CUDNN、TensorRT 进行预测加速，从而实现更优的推理性能。更多关于Paddle Inference推理引擎的介绍，可以参考[Paddle Inference官网教程](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/infer/inference/inference_cn.html)。
|
||||
|
||||
Inference 的获取可以参考 [ResNet50 推理模型准备](./ResNet.md#41-推理模型准备) 。
|
||||
|
||||
<a name="4.2"></a>
|
||||
|
||||
### 4.2 基于 Python 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 python 预测引擎推理的示例。您可以参考[ResNet50 基于 Python 预测引擎推理](./ResNet.md#42-基于-python-预测引擎推理) 对 SwinTransformer 完成推理预测。
|
||||
|
||||
<a name="4.3"></a>
|
||||
|
||||
### 4.3 基于 C++ 预测引擎推理
|
||||
|
||||
PaddleClas 提供了基于 C++ 预测引擎推理的示例，您可以参考[服务器端 C++ 预测](../inference_deployment/cpp_deploy.md)来完成相应的推理部署。如果您使用的是 Windows 平台，可以参考[基于 Visual Studio 2019 Community CMake 编译指南](../inference_deployment/cpp_deploy_on_windows.md)完成相应的预测库编译和模型预测工作。
|
||||
|
||||
<a name="4.4"></a>
|
||||
|
||||
### 4.4 服务化部署
|
||||
|
||||
Paddle Serving 提供高性能、灵活易用的工业级在线推理服务。Paddle Serving 支持 RESTful、gRPC、bRPC 等多种协议，提供多种异构硬件和多种操作系统环境下推理解决方案。更多关于Paddle Serving 的介绍，可以参考[Paddle Serving 代码仓库](https://github.com/PaddlePaddle/Serving)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Serving 来完成模型服务化部署的示例，您可以参考[模型服务化部署](../inference_deployment/paddle_serving_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="4.5"></a>
|
||||
|
||||
### 4.5 端侧部署
|
||||
|
||||
Paddle Lite 是一个高性能、轻量级、灵活性强且易于扩展的深度学习推理框架，定位于支持包括移动端、嵌入式以及服务器端在内的多硬件平台。更多关于 Paddle Lite 的介绍，可以参考[Paddle Lite 代码仓库](https://github.com/PaddlePaddle/Paddle-Lite)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle Lite 来完成模型端侧部署的示例，您可以参考[端侧部署](../inference_deployment/paddle_lite_deploy.md)来完成相应的部署工作。
|
||||
|
||||
<a name="4.6"></a>
|
||||
|
||||
### 4.6 Paddle2ONNX 模型转换与预测
|
||||
|
||||
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式。通过 ONNX 可以完成将 Paddle 模型到多种推理引擎的部署，包括TensorRT/OpenVINO/MNN/TNN/NCNN，以及其它对 ONNX 开源格式进行支持的推理引擎或硬件。更多关于 Paddle2ONNX 的介绍，可以参考[Paddle2ONNX 代码仓库](https://github.com/PaddlePaddle/Paddle2ONNX)。
|
||||
|
||||
PaddleClas 提供了基于 Paddle2ONNX 来完成 inference 模型转换 ONNX 模型并作推理预测的示例，您可以参考[Paddle2ONNX 模型转换与预测](@shuilong)来完成相应的部署工作。
|
||||
|
||||
|
|
|
|||
329
paddleclas.py
329
paddleclas.py
|
|
@ -1,4 +1,4 @@
|
|||
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
|
||||
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
|
|
@ -24,7 +24,6 @@ import shutil
|
|||
import textwrap
|
||||
import tarfile
|
||||
import requests
|
||||
import warnings
|
||||
from functools import partial
|
||||
from difflib import SequenceMatcher
|
||||
|
||||
|
|
@ -32,24 +31,25 @@ import cv2
|
|||
import numpy as np
|
||||
from tqdm import tqdm
|
||||
from prettytable import PrettyTable
|
||||
import paddle
|
||||
|
||||
from deploy.python.predict_cls import ClsPredictor
|
||||
from deploy.utils.get_image_list import get_image_list
|
||||
from deploy.utils import config
|
||||
|
||||
from ppcls.arch.backbone import *
|
||||
from ppcls.utils.logger import init_logger
|
||||
import ppcls.arch.backbone as backbone
|
||||
from ppcls.utils import logger
|
||||
|
||||
# for building model with loading pretrained weights from backbone
|
||||
init_logger()
|
||||
logger.init_logger()
|
||||
|
||||
__all__ = ["PaddleClas"]
|
||||
|
||||
BASE_DIR = os.path.expanduser("~/.paddleclas/")
|
||||
BASE_INFERENCE_MODEL_DIR = os.path.join(BASE_DIR, "inference_model")
|
||||
BASE_IMAGES_DIR = os.path.join(BASE_DIR, "images")
|
||||
BASE_DOWNLOAD_URL = "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/{}_infer.tar"
|
||||
MODEL_SERIES = {
|
||||
IMN_MODEL_BASE_DOWNLOAD_URL = "https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/inference/{}_infer.tar"
|
||||
IMN_MODEL_SERIES = {
|
||||
"AlexNet": ["AlexNet"],
|
||||
"DarkNet": ["DarkNet53"],
|
||||
"DeiT": [
|
||||
|
|
@ -100,10 +100,17 @@ MODEL_SERIES = {
|
|||
"MobileNetV3_large_x1_0", "MobileNetV3_large_x1_25",
|
||||
"MobileNetV3_small_x1_0_ssld", "MobileNetV3_large_x1_0_ssld"
|
||||
],
|
||||
"PPHGNet": [
|
||||
"PPHGNet_tiny",
|
||||
"PPHGNet_small",
|
||||
"PPHGNet_tiny_ssld",
|
||||
"PPHGNet_small_ssld",
|
||||
],
|
||||
"PPLCNet": [
|
||||
"PPLCNet_x0_25", "PPLCNet_x0_35", "PPLCNet_x0_5", "PPLCNet_x0_75",
|
||||
"PPLCNet_x1_0", "PPLCNet_x1_5", "PPLCNet_x2_0", "PPLCNet_x2_5"
|
||||
],
|
||||
"PPLCNetV2": ["PPLCNetV2_base"],
|
||||
"RedNet": ["RedNet26", "RedNet38", "RedNet50", "RedNet101", "RedNet152"],
|
||||
"RegNet": ["RegNetX_4GF"],
|
||||
"Res2Net": [
|
||||
|
|
@ -168,6 +175,13 @@ MODEL_SERIES = {
|
|||
]
|
||||
}
|
||||
|
||||
PULC_MODEL_BASE_DOWNLOAD_URL = "https://paddleclas.bj.bcebos.com/models/PULC/{}_infer.tar"
|
||||
PULC_MODELS = [
|
||||
"person_exists", "person_attribute", "safety_helmet", "traffic_sign",
|
||||
"vehicle_exists", "vehicle_attr", "textline_orientation",
|
||||
"text_image_orientation", "language_classification"
|
||||
]
|
||||
|
||||
|
||||
class ImageTypeError(Exception):
|
||||
"""ImageTypeError.
|
||||
|
|
@ -185,76 +199,67 @@ class InputModelError(Exception):
|
|||
super().__init__(message)
|
||||
|
||||
|
||||
def init_config(model_name,
|
||||
inference_model_dir,
|
||||
use_gpu=True,
|
||||
batch_size=1,
|
||||
topk=5,
|
||||
**kwargs):
|
||||
imagenet1k_map_path = os.path.join(
|
||||
os.path.abspath(__dir__), "ppcls/utils/imagenet1k_label_list.txt")
|
||||
cfg = {
|
||||
"Global": {
|
||||
"infer_imgs": kwargs["infer_imgs"]
|
||||
if "infer_imgs" in kwargs else False,
|
||||
"model_name": model_name,
|
||||
"inference_model_dir": inference_model_dir,
|
||||
"batch_size": batch_size,
|
||||
"use_gpu": use_gpu,
|
||||
"enable_mkldnn": kwargs["enable_mkldnn"]
|
||||
if "enable_mkldnn" in kwargs else False,
|
||||
"cpu_num_threads": kwargs["cpu_num_threads"]
|
||||
if "cpu_num_threads" in kwargs else 1,
|
||||
"enable_benchmark": False,
|
||||
"use_fp16": kwargs["use_fp16"] if "use_fp16" in kwargs else False,
|
||||
"ir_optim": True,
|
||||
"use_tensorrt": kwargs["use_tensorrt"]
|
||||
if "use_tensorrt" in kwargs else False,
|
||||
"gpu_mem": kwargs["gpu_mem"] if "gpu_mem" in kwargs else 8000,
|
||||
"enable_profile": False
|
||||
},
|
||||
"PreProcess": {
|
||||
"transform_ops": [{
|
||||
"ResizeImage": {
|
||||
"resize_short": kwargs["resize_short"]
|
||||
if "resize_short" in kwargs else 256
|
||||
}
|
||||
}, {
|
||||
"CropImage": {
|
||||
"size": kwargs["crop_size"]
|
||||
if "crop_size" in kwargs else 224
|
||||
}
|
||||
}, {
|
||||
"NormalizeImage": {
|
||||
"scale": 0.00392157,
|
||||
"mean": [0.485, 0.456, 0.406],
|
||||
"std": [0.229, 0.224, 0.225],
|
||||
"order": ''
|
||||
}
|
||||
}, {
|
||||
"ToCHWImage": None
|
||||
}]
|
||||
},
|
||||
"PostProcess": {
|
||||
"main_indicator": "Topk",
|
||||
"Topk": {
|
||||
"topk": topk,
|
||||
"class_id_map_file": imagenet1k_map_path
|
||||
}
|
||||
}
|
||||
}
|
||||
if "save_dir" in kwargs:
|
||||
if kwargs["save_dir"] is not None:
|
||||
cfg["PostProcess"]["SavePreLabel"] = {
|
||||
"save_dir": kwargs["save_dir"]
|
||||
}
|
||||
if "class_id_map_file" in kwargs:
|
||||
if kwargs["class_id_map_file"] is not None:
|
||||
cfg["PostProcess"]["Topk"]["class_id_map_file"] = kwargs[
|
||||
"class_id_map_file"]
|
||||
def init_config(model_type, model_name, inference_model_dir, **kwargs):
|
||||
|
||||
cfg_path = f"deploy/configs/PULC/{model_name}/inference_{model_name}.yaml" if model_type == "pulc" else "deploy/configs/inference_cls.yaml"
|
||||
cfg_path = os.path.join(__dir__, cfg_path)
|
||||
cfg = config.get_config(cfg_path, show=False)
|
||||
|
||||
cfg.Global.inference_model_dir = inference_model_dir
|
||||
|
||||
if "batch_size" in kwargs and kwargs["batch_size"]:
|
||||
cfg.Global.batch_size = kwargs["batch_size"]
|
||||
|
||||
if "use_gpu" in kwargs and kwargs["use_gpu"]:
|
||||
cfg.Global.use_gpu = kwargs["use_gpu"]
|
||||
if cfg.Global.use_gpu and not paddle.device.is_compiled_with_cuda():
|
||||
msg = "The current running environment does not support the use of GPU. CPU has been used instead."
|
||||
logger.warning(msg)
|
||||
cfg.Global.use_gpu = False
|
||||
|
||||
if "infer_imgs" in kwargs and kwargs["infer_imgs"]:
|
||||
cfg.Global.infer_imgs = kwargs["infer_imgs"]
|
||||
if "enable_mkldnn" in kwargs and kwargs["enable_mkldnn"]:
|
||||
cfg.Global.enable_mkldnn = kwargs["enable_mkldnn"]
|
||||
if "cpu_num_threads" in kwargs and kwargs["cpu_num_threads"]:
|
||||
cfg.Global.cpu_num_threads = kwargs["cpu_num_threads"]
|
||||
if "use_fp16" in kwargs and kwargs["use_fp16"]:
|
||||
cfg.Global.use_fp16 = kwargs["use_fp16"]
|
||||
if "use_tensorrt" in kwargs and kwargs["use_tensorrt"]:
|
||||
cfg.Global.use_tensorrt = kwargs["use_tensorrt"]
|
||||
if "gpu_mem" in kwargs and kwargs["gpu_mem"]:
|
||||
cfg.Global.gpu_mem = kwargs["gpu_mem"]
|
||||
if "resize_short" in kwargs and kwargs["resize_short"]:
|
||||
cfg.PreProcess.transform_ops[0]["ResizeImage"][
|
||||
"resize_short"] = kwargs["resize_short"]
|
||||
if "crop_size" in kwargs and kwargs["crop_size"]:
|
||||
cfg.PreProcess.transform_ops[1]["CropImage"]["size"] = kwargs[
|
||||
"crop_size"]
|
||||
|
||||
# TODO(gaotingquan): not robust
|
||||
if "thresh" in kwargs and kwargs[
|
||||
"thresh"] and "ThreshOutput" in cfg.PostProcess:
|
||||
cfg.PostProcess.ThreshOutput.thresh = kwargs["thresh"]
|
||||
if "Topk" in cfg.PostProcess:
|
||||
if "topk" in kwargs and kwargs["topk"]:
|
||||
cfg.PostProcess.Topk.topk = kwargs["topk"]
|
||||
if "class_id_map_file" in kwargs and kwargs["class_id_map_file"]:
|
||||
cfg.PostProcess.Topk.class_id_map_file = kwargs[
|
||||
"class_id_map_file"]
|
||||
else:
|
||||
cfg.PostProcess.Topk.class_id_map_file = os.path.relpath(
|
||||
cfg.PostProcess.Topk.class_id_map_file, "../")
|
||||
if "VehicleAttribute" in cfg.PostProcess:
|
||||
if "color_threshold" in kwargs and kwargs["color_threshold"]:
|
||||
cfg.PostProcess.VehicleAttribute.color_threshold = kwargs[
|
||||
"color_threshold"]
|
||||
if "type_threshold" in kwargs and kwargs["type_threshold"]:
|
||||
cfg.PostProcess.VehicleAttribute.type_threshold = kwargs[
|
||||
"type_threshold"]
|
||||
|
||||
if "save_dir" in kwargs and kwargs["save_dir"]:
|
||||
cfg.PostProcess.SavePreLabel.save_dir = kwargs["save_dir"]
|
||||
|
||||
cfg = config.AttrDict(cfg)
|
||||
config.create_attr_dict(cfg)
|
||||
return cfg
|
||||
|
||||
|
||||
|
|
@ -275,40 +280,48 @@ def args_cfg():
|
|||
type=str,
|
||||
help="The directory of model files. Valid when model_name not specifed."
|
||||
)
|
||||
parser.add_argument("--use_gpu", type=str2bool, help="Whether use GPU.")
|
||||
parser.add_argument(
|
||||
"--use_gpu", type=str, default=True, help="Whether use GPU.")
|
||||
parser.add_argument("--gpu_mem", type=int, default=8000, help="")
|
||||
"--gpu_mem",
|
||||
type=int,
|
||||
help="The memory size of GPU allocated to predict.")
|
||||
parser.add_argument(
|
||||
"--enable_mkldnn",
|
||||
type=str2bool,
|
||||
default=False,
|
||||
help="Whether use MKLDNN. Valid when use_gpu is False")
|
||||
parser.add_argument("--cpu_num_threads", type=int, default=1, help="")
|
||||
parser.add_argument(
|
||||
"--use_tensorrt", type=str2bool, default=False, help="")
|
||||
parser.add_argument("--use_fp16", type=str2bool, default=False, help="")
|
||||
"--cpu_num_threads",
|
||||
type=int,
|
||||
help="The threads number when predicting on CPU.")
|
||||
parser.add_argument(
|
||||
"--batch_size", type=int, default=1, help="Batch size. Default by 1.")
|
||||
"--use_tensorrt",
|
||||
type=str2bool,
|
||||
help="Whether use TensorRT to accelerate. ")
|
||||
parser.add_argument(
|
||||
"--use_fp16", type=str2bool, help="Whether use FP16 to predict.")
|
||||
parser.add_argument("--batch_size", type=int, help="Batch size.")
|
||||
parser.add_argument(
|
||||
"--topk",
|
||||
type=int,
|
||||
default=5,
|
||||
help="Return topk score(s) and corresponding results. Default by 5.")
|
||||
help="Return topk score(s) and corresponding results when Topk postprocess is used."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--class_id_map_file",
|
||||
type=str,
|
||||
help="The path of file that map class_id and label.")
|
||||
parser.add_argument(
|
||||
"--threshold",
|
||||
type=float,
|
||||
help="The threshold of ThreshOutput when postprocess is used.")
|
||||
parser.add_argument("--color_threshold", type=float, help="")
|
||||
parser.add_argument("--type_threshold", type=float, help="")
|
||||
parser.add_argument(
|
||||
"--save_dir",
|
||||
type=str,
|
||||
help="The directory to save prediction results as pre-label.")
|
||||
parser.add_argument(
|
||||
"--resize_short",
|
||||
type=int,
|
||||
default=256,
|
||||
help="Resize according to short size.")
|
||||
parser.add_argument(
|
||||
"--crop_size", type=int, default=224, help="Centor crop size.")
|
||||
"--resize_short", type=int, help="Resize according to short size.")
|
||||
parser.add_argument("--crop_size", type=int, help="Centor crop size.")
|
||||
|
||||
args = parser.parse_args()
|
||||
return vars(args)
|
||||
|
|
@ -317,33 +330,44 @@ def args_cfg():
|
|||
def print_info():
|
||||
"""Print list of supported models in formatted.
|
||||
"""
|
||||
table = PrettyTable(["Series", "Name"])
|
||||
imn_table = PrettyTable(["IMN Model Series", "Model Name"])
|
||||
pulc_table = PrettyTable(["PULC Models"])
|
||||
try:
|
||||
sz = os.get_terminal_size()
|
||||
width = sz.columns - 30 if sz.columns > 50 else 10
|
||||
total_width = sz.columns
|
||||
first_width = 30
|
||||
second_width = total_width - first_width if total_width > 50 else 10
|
||||
except OSError:
|
||||
width = 100
|
||||
for series in MODEL_SERIES:
|
||||
names = textwrap.fill(" ".join(MODEL_SERIES[series]), width=width)
|
||||
table.add_row([series, names])
|
||||
width = len(str(table).split("\n")[0])
|
||||
print("{}".format("-" * width))
|
||||
print("Models supported by PaddleClas".center(width))
|
||||
print(table)
|
||||
print("Powered by PaddlePaddle!".rjust(width))
|
||||
print("{}".format("-" * width))
|
||||
second_width = 100
|
||||
for series in IMN_MODEL_SERIES:
|
||||
names = textwrap.fill(
|
||||
" ".join(IMN_MODEL_SERIES[series]), width=second_width)
|
||||
imn_table.add_row([series, names])
|
||||
|
||||
table_width = len(str(imn_table).split("\n")[0])
|
||||
pulc_table.add_row([
|
||||
textwrap.fill(
|
||||
" ".join(PULC_MODELS), width=total_width).center(table_width - 4)
|
||||
])
|
||||
|
||||
print("{}".format("-" * table_width))
|
||||
print("Models supported by PaddleClas".center(table_width))
|
||||
print(imn_table)
|
||||
print(pulc_table)
|
||||
print("Powered by PaddlePaddle!".rjust(table_width))
|
||||
print("{}".format("-" * table_width))
|
||||
|
||||
|
||||
def get_model_names():
|
||||
def get_imn_model_names():
|
||||
"""Get the model names list.
|
||||
"""
|
||||
model_names = []
|
||||
for series in MODEL_SERIES:
|
||||
model_names += (MODEL_SERIES[series])
|
||||
for series in IMN_MODEL_SERIES:
|
||||
model_names += (IMN_MODEL_SERIES[series])
|
||||
return model_names
|
||||
|
||||
|
||||
def similar_architectures(name="", names=[], thresh=0.1, topk=10):
|
||||
def similar_model_names(name="", names=[], thresh=0.1, topk=5):
|
||||
"""Find the most similar topk model names.
|
||||
"""
|
||||
scores = []
|
||||
|
|
@ -378,12 +402,17 @@ def download_with_progressbar(url, save_path):
|
|||
f"Something went wrong while downloading file from {url}")
|
||||
|
||||
|
||||
def check_model_file(model_name):
|
||||
def check_model_file(model_type, model_name):
|
||||
"""Check the model files exist and download and untar when no exist.
|
||||
"""
|
||||
storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
|
||||
model_name)
|
||||
url = BASE_DOWNLOAD_URL.format(model_name)
|
||||
if model_type == "pulc":
|
||||
storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
|
||||
"PULC", model_name)
|
||||
url = PULC_MODEL_BASE_DOWNLOAD_URL.format(model_name)
|
||||
else:
|
||||
storage_directory = partial(os.path.join, BASE_INFERENCE_MODEL_DIR,
|
||||
"IMN", model_name)
|
||||
url = IMN_MODEL_BASE_DOWNLOAD_URL.format(model_name)
|
||||
|
||||
tar_file_name_list = [
|
||||
"inference.pdiparams", "inference.pdiparams.info", "inference.pdmodel"
|
||||
|
|
@ -393,7 +422,7 @@ def check_model_file(model_name):
|
|||
if not os.path.exists(model_file_path) or not os.path.exists(
|
||||
params_file_path):
|
||||
tmp_path = storage_directory(url.split("/")[-1])
|
||||
print(f"download {url} to {tmp_path}")
|
||||
logger.info(f"download {url} to {tmp_path}")
|
||||
os.makedirs(storage_directory(), exist_ok=True)
|
||||
download_with_progressbar(url, tmp_path)
|
||||
with tarfile.open(tmp_path, "r") as tarObj:
|
||||
|
|
@ -426,9 +455,6 @@ class PaddleClas(object):
|
|||
def __init__(self,
|
||||
model_name: str=None,
|
||||
inference_model_dir: str=None,
|
||||
use_gpu: bool=True,
|
||||
batch_size: int=1,
|
||||
topk: int=5,
|
||||
**kwargs):
|
||||
"""Init PaddleClas with config.
|
||||
|
||||
|
|
@ -440,9 +466,11 @@ class PaddleClas(object):
|
|||
topk (int, optional): Return the top k prediction results with the highest score. Defaults to 5.
|
||||
"""
|
||||
super().__init__()
|
||||
self._config = init_config(model_name, inference_model_dir, use_gpu,
|
||||
batch_size, topk, **kwargs)
|
||||
self._check_input_model()
|
||||
self.model_type, inference_model_dir = self._check_input_model(
|
||||
model_name, inference_model_dir)
|
||||
self._config = init_config(self.model_type, model_name,
|
||||
inference_model_dir, **kwargs)
|
||||
|
||||
self.cls_predictor = ClsPredictor(self._config)
|
||||
|
||||
def get_config(self):
|
||||
|
|
@ -450,24 +478,29 @@ class PaddleClas(object):
|
|||
"""
|
||||
return self._config
|
||||
|
||||
def _check_input_model(self):
|
||||
def _check_input_model(self, model_name, inference_model_dir):
|
||||
"""Check input model name or model files.
|
||||
"""
|
||||
candidate_model_names = get_model_names()
|
||||
input_model_name = self._config.Global.get("model_name", None)
|
||||
inference_model_dir = self._config.Global.get("inference_model_dir",
|
||||
None)
|
||||
if input_model_name is not None:
|
||||
similar_names = similar_architectures(input_model_name,
|
||||
candidate_model_names)
|
||||
similar_names_str = ", ".join(similar_names)
|
||||
if input_model_name not in candidate_model_names:
|
||||
err = f"{input_model_name} is not provided by PaddleClas. \nMaybe you want: [{similar_names_str}]. \nIf you want to use your own model, please specify inference_model_dir!"
|
||||
all_imn_model_names = get_imn_model_names()
|
||||
all_pulc_model_names = PULC_MODELS
|
||||
|
||||
if model_name:
|
||||
if model_name in all_imn_model_names:
|
||||
inference_model_dir = check_model_file("imn", model_name)
|
||||
return "imn", inference_model_dir
|
||||
elif model_name in all_pulc_model_names:
|
||||
inference_model_dir = check_model_file("pulc", model_name)
|
||||
return "pulc", inference_model_dir
|
||||
else:
|
||||
similar_imn_names = similar_model_names(model_name,
|
||||
all_imn_model_names)
|
||||
similar_pulc_names = similar_model_names(model_name,
|
||||
all_pulc_model_names)
|
||||
similar_names_str = ", ".join(similar_imn_names +
|
||||
similar_pulc_names)
|
||||
err = f"{model_name} is not provided by PaddleClas. \nMaybe you want the : [{similar_names_str}]. \nIf you want to use your own model, please specify inference_model_dir!"
|
||||
raise InputModelError(err)
|
||||
self._config.Global.inference_model_dir = check_model_file(
|
||||
input_model_name)
|
||||
return
|
||||
elif inference_model_dir is not None:
|
||||
elif inference_model_dir:
|
||||
model_file_path = os.path.join(inference_model_dir,
|
||||
"inference.pdmodel")
|
||||
params_file_path = os.path.join(inference_model_dir,
|
||||
|
|
@ -476,11 +509,11 @@ class PaddleClas(object):
|
|||
params_file_path):
|
||||
err = f"There is no model file or params file in this directory: {inference_model_dir}"
|
||||
raise InputModelError(err)
|
||||
return
|
||||
return "custom", inference_model_dir
|
||||
else:
|
||||
err = f"Please specify the model name supported by PaddleClas or directory contained model files(inference.pdmodel, inference.pdiparams)."
|
||||
raise InputModelError(err)
|
||||
return
|
||||
return None
|
||||
|
||||
def predict(self, input_data: Union[str, np.array],
|
||||
print_pred: bool=False) -> Generator[list, None, None]:
|
||||
|
|
@ -511,22 +544,21 @@ class PaddleClas(object):
|
|||
os.makedirs(image_storage_dir())
|
||||
image_save_path = image_storage_dir("tmp.jpg")
|
||||
download_with_progressbar(input_data, image_save_path)
|
||||
input_data = image_save_path
|
||||
warnings.warn(
|
||||
logger.info(
|
||||
f"Image to be predicted from Internet: {input_data}, has been saved to: {image_save_path}"
|
||||
)
|
||||
input_data = image_save_path
|
||||
image_list = get_image_list(input_data)
|
||||
|
||||
batch_size = self._config.Global.get("batch_size", 1)
|
||||
topk = self._config.PostProcess.Topk.get('topk', 1)
|
||||
|
||||
img_list = []
|
||||
img_path_list = []
|
||||
cnt = 0
|
||||
for idx, img_path in enumerate(image_list):
|
||||
for idx_img, img_path in enumerate(image_list):
|
||||
img = cv2.imread(img_path)
|
||||
if img is None:
|
||||
warnings.warn(
|
||||
logger.warning(
|
||||
f"Image file failed to read and has been skipped. The path: {img_path}"
|
||||
)
|
||||
continue
|
||||
|
|
@ -535,16 +567,15 @@ class PaddleClas(object):
|
|||
img_path_list.append(img_path)
|
||||
cnt += 1
|
||||
|
||||
if cnt % batch_size == 0 or (idx + 1) == len(image_list):
|
||||
if cnt % batch_size == 0 or (idx_img + 1) == len(image_list):
|
||||
preds = self.cls_predictor.predict(img_list)
|
||||
|
||||
if print_pred and preds:
|
||||
for idx, pred in enumerate(preds):
|
||||
pred_str = ", ".join(
|
||||
[f"{k}: {pred[k]}" for k in pred])
|
||||
print(
|
||||
f"filename: {img_path_list[idx]}, top-{topk}, {pred_str}"
|
||||
)
|
||||
if preds:
|
||||
for idx_pred, pred in enumerate(preds):
|
||||
pred["filename"] = img_path_list[idx_pred]
|
||||
if print_pred:
|
||||
logger.info(", ".join(
|
||||
[f"{k}: {pred[k]}" for k in pred]))
|
||||
|
||||
img_list = []
|
||||
img_path_list = []
|
||||
|
|
@ -564,7 +595,7 @@ def main():
|
|||
res = clas_engine.predict(cfg["infer_imgs"], print_pred=True)
|
||||
for _ in res:
|
||||
pass
|
||||
print("Predict complete!")
|
||||
logger.info("Predict complete!")
|
||||
return
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,167 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output_lcnet_x2_5_udml
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 100
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
AMP:
|
||||
scale_loss: 128.0
|
||||
use_dynamic_loss_scaling: True
|
||||
# O1: mixed fp16
|
||||
level: O1
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "DistillationModel"
|
||||
class_num: &class_num 1000
|
||||
# if not null, its lengths should be same as models
|
||||
pretrained_list:
|
||||
# if not null, its lengths should be same as models
|
||||
freeze_params_list:
|
||||
- False
|
||||
- False
|
||||
models:
|
||||
- Teacher:
|
||||
name: PPLCNet_x2_5
|
||||
class_num: *class_num
|
||||
pretrained: False
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
- Student:
|
||||
name: PPLCNet_x2_5
|
||||
class_num: *class_num
|
||||
pretrained: False
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- DistillationGTCELoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_names: ["Student", "Teacher"]
|
||||
- DistillationDMLLoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
- DistillationDistanceLoss:
|
||||
weight: 1.0
|
||||
key: "blocks5"
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.4
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00004
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/ILSVRC2012/
|
||||
cls_label_path: ./dataset/ILSVRC2012/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/ILSVRC2012/
|
||||
cls_label_path: ./dataset/ILSVRC2012/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- DistillationTopkAcc:
|
||||
model_key: "Student"
|
||||
topk: [1, 5]
|
||||
Eval:
|
||||
- DistillationTopkAcc:
|
||||
model_key: "Student"
|
||||
topk: [1, 5]
|
||||
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 30
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
start_eval_epoch: 20
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: MobileNetV3_small_x0_35
|
||||
class_num: 10
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 1.3
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/language_classification_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 30
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: 10
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
stride_list: [2, [2, 1], [2, 1], [2, 1], [2, 1]]
|
||||
lr_mult_list : [0.0, 0.4, 0.4, 0.8, 0.8, 1.0]
|
||||
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.8
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob: 1.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 1.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/language_classification/word_35404.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/language_classification_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 30
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
use_dali: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "DistillationModel"
|
||||
class_num: &class_num 10
|
||||
# if not null, its lengths should be same as models
|
||||
pretrained_list:
|
||||
# if not null, its lengths should be same as models
|
||||
freeze_params_list:
|
||||
- True
|
||||
- False
|
||||
use_sync_bn: True
|
||||
models:
|
||||
- Teacher:
|
||||
name: ResNet101_vd
|
||||
class_num: *class_num
|
||||
- Student:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: *class_num
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
stride_list: [2, [2, 1], [2, 1], [2, 1], [2, 1]]
|
||||
lr_mult_list : [0.0, 0.4, 0.4, 0.8, 0.8, 1.0]
|
||||
|
||||
|
||||
infer_model_name: "Student"
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- DistillationDMLLoss:
|
||||
weight: 1.0
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.8
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/train_list_for_distill.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob: 1.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 1.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/language_classification/word_35404.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [160, 80]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/language_classification_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- DistillationTopkAcc:
|
||||
model_key: "Student"
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,142 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 30
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
start_eval_epoch: 20
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: 10
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
stride_list: [2, [2, 1], [2, 1], [2, 1], [2, 1]]
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.4
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 48]
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob: 0.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: [192, 48]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 48]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 32
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/language_classification/word_35404.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 48]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/language_classification_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,160 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 30
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: SwinTransformer_tiny_patch4_window7_224
|
||||
class_num: 10
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: AdamW
|
||||
beta1: 0.9
|
||||
beta2: 0.999
|
||||
epsilon: 1e-8
|
||||
weight_decay: 0.05
|
||||
no_weight_decay_name: absolute_pos_embed relative_position_bias_table .bias norm
|
||||
one_dim_param_no_weight_decay: True
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 5e-4
|
||||
eta_min: 1e-5
|
||||
warmup_epoch: 5
|
||||
warmup_start_lr: 1e-6
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.25
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
batch_transform_ops:
|
||||
- OpSampler:
|
||||
MixupOperator:
|
||||
alpha: 0.8
|
||||
prob: 0.5
|
||||
CutmixOperator:
|
||||
alpha: 1.0
|
||||
prob: 0.5
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/language_classification/
|
||||
cls_label_path: ./dataset/language_classification/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/language_classification/word_35404.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/language_classification_label_list.txt
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
base_config_file: ppcls/configs/PULC/language_classification/PPLCNet_x1_0_search.yaml
|
||||
distill_config_file: ppcls/configs/PULC/language_classification/PPLCNet_x1_0_distillation.yaml
|
||||
|
||||
gpus: 0,1,2,3
|
||||
output_dir: output/search_language_classification
|
||||
search_times: 1
|
||||
search_dict:
|
||||
- search_key: lrs
|
||||
replace_config:
|
||||
- Optimizer.lr.learning_rate
|
||||
search_values: [0.2, 0.4, 0.8]
|
||||
- search_key: resolutions
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.1.ResizeImage.size
|
||||
- DataLoader.Train.dataset.transform_ops.3.TimmAutoAugment.img_size
|
||||
- DataLoader.Eval.dataset.transform_ops.1.ResizeImage.size
|
||||
search_values: [[192, 48], [180, 60], [160, 80]]
|
||||
- search_key: ra_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.3.TimmAutoAugment.prob
|
||||
search_values: [0.0, 0.5, 1.0]
|
||||
- search_key: re_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.5.RandomErasing.EPSILON
|
||||
search_values: [0.0, 0.5, 1.0]
|
||||
- search_key: lr_mult_list
|
||||
replace_config:
|
||||
- Arch.lr_mult_list
|
||||
search_values:
|
||||
- [0.0, 0.2, 0.4, 0.6, 0.8, 1.0]
|
||||
- [0.0, 0.4, 0.4, 0.8, 0.8, 1.0]
|
||||
- [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
|
||||
teacher:
|
||||
rm_keys:
|
||||
- Arch.lr_mult_list
|
||||
search_values:
|
||||
- ResNet101_vd
|
||||
- ResNet50_vd
|
||||
final_replace:
|
||||
Arch.lr_mult_list: Arch.models.1.Student.lr_mult_list
|
||||
|
|
@ -4,11 +4,11 @@ Global:
|
|||
pretrained_model: null
|
||||
output_dir: "./output/"
|
||||
device: "gpu"
|
||||
save_interval: 5
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 20
|
||||
print_batch_step: 20
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 256, 192]
|
||||
|
|
@ -17,7 +17,7 @@ Global:
|
|||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "MobileNetV3_large_x1_0"
|
||||
name: "MobileNetV3_small_x0_35"
|
||||
pretrained: True
|
||||
class_num: 26
|
||||
|
||||
|
|
@ -52,7 +52,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_val_list.txt"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -85,7 +85,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/test_list.txt"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -107,6 +107,26 @@ DataLoader:
|
|||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
|
|
@ -53,7 +53,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_val_list.txt"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -99,7 +99,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/test_list.txt"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -121,6 +121,26 @@ DataLoader:
|
|||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
|
|
|
|||
|
|
@ -72,14 +72,13 @@ Optimizer:
|
|||
coeff: 0.0005
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_val_list.txt"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -125,7 +124,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/test_list.txt"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -147,7 +146,26 @@ DataLoader:
|
|||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_val_list.txt"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -99,7 +99,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k"
|
||||
cls_label_path: "dataset/pa100k/test_list.txt"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -121,6 +121,26 @@ DataLoader:
|
|||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
|
|
|
|||
|
|
@ -4,11 +4,11 @@ Global:
|
|||
pretrained_model: null
|
||||
output_dir: "./output/"
|
||||
device: "gpu"
|
||||
save_interval: 5
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 20
|
||||
print_batch_step: 20
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 256, 192]
|
||||
|
|
@ -44,7 +44,6 @@ Optimizer:
|
|||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.0005
|
||||
#clip_norm: 10
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
|
|
@ -52,7 +51,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_val_list.txt"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -85,7 +84,7 @@ DataLoader:
|
|||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/test_list.txt"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
|
|
@ -107,6 +106,26 @@ DataLoader:
|
|||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [192, 256]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,135 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: "./output/"
|
||||
device: "gpu"
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 20
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: "./inference"
|
||||
use_multilabel: True
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "SwinTransformer_tiny_patch4_window7_224"
|
||||
pretrained: True
|
||||
class_num: 26
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- MultiLabelLoss:
|
||||
weight: 1.0
|
||||
weight_ratio: True
|
||||
size_sum: True
|
||||
Eval:
|
||||
- MultiLabelLoss:
|
||||
weight: 1.0
|
||||
weight_ratio: True
|
||||
size_sum: True
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.01
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.0005
|
||||
#clip_norm: 10
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/train_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [224, 224]
|
||||
- Padv2:
|
||||
size: [244, 244]
|
||||
pad_mode: 1
|
||||
fill_value: 0
|
||||
- RandomCropImage:
|
||||
size: [224, 224]
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: True
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
Eval:
|
||||
dataset:
|
||||
name: MultiLabelDataset
|
||||
image_root: "dataset/pa100k/"
|
||||
cls_label_path: "dataset/pa100k/val_list.txt"
|
||||
label_ratio: True
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [224, 224]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/person_attribute/090004.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
size: [224, 224]
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: PersonAttribute
|
||||
threshold: 0.5 #default threshold
|
||||
glasses_threshold: 0.3 #threshold only for glasses
|
||||
hold_threshold: 0.6 #threshold only for hold
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
- ATTRMetric:
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: MobileNetV3_small_x0_35
|
||||
pretrained: True
|
||||
class_num: 2
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.4
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
|
@ -0,0 +1,148 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 40
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
class_num: 2
|
||||
use_sync_bn : True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.025
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 176
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob : 0.5
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size : 176
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON : 0.1
|
||||
r1 : 0.3
|
||||
sh : 1.0/3.0
|
||||
sl : 0.02
|
||||
attempt : 10
|
||||
use_log_aspect : True
|
||||
mode : pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
|
@ -0,0 +1,185 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
start_eval_epoch: 1
|
||||
eval_interval: 1
|
||||
epochs: 40
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
use_dali: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "DistillationModel"
|
||||
class_num: &class_num 2
|
||||
# if not null, its lengths should be same as models
|
||||
pretrained_list:
|
||||
# if not null, its lengths should be same as models
|
||||
freeze_params_list:
|
||||
- False
|
||||
- False
|
||||
use_sync_bn: True
|
||||
models:
|
||||
- Teacher:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: *class_num
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
return_stages: True
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
- Student:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: *class_num
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
return_stages: True
|
||||
return_patterns: ["blocks3", "blocks4", "blocks5", "blocks6"]
|
||||
|
||||
infer_model_name: "Student"
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- DistillationGTCELoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_names: ["Student", "Teacher"]
|
||||
- DistillationDMLLoss:
|
||||
weight: 1.0
|
||||
key: logits
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
- DistillationDistanceLoss:
|
||||
weight: 1.0
|
||||
key: "blocks4"
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.015
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 192
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob: 0.5
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 192
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.5
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- DistillationTopkAcc:
|
||||
model_key: "Student"
|
||||
topk: [1]
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
|
@ -0,0 +1,148 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 40
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
class_num: 2
|
||||
use_sync_bn: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.10
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 192
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
prob: 0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 192
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: Res2Net200_vd_26w_4s
|
||||
class_num: 2
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.005
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.0001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
batch_transform_ops:
|
||||
- MixupOperator:
|
||||
alpha: 0.2
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 32
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
||||
|
|
@ -0,0 +1,159 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: SwinTransformer_tiny_patch4_window7_224
|
||||
pretrained: True
|
||||
class_num: 2
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: AdamW
|
||||
beta1: 0.9
|
||||
beta2: 0.999
|
||||
epsilon: 1e-8
|
||||
weight_decay: 0.05
|
||||
no_weight_decay_name: absolute_pos_embed relative_position_bias_table .bias norm
|
||||
one_dim_param_no_weight_decay: True
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 1e-5
|
||||
eta_min: 1e-7
|
||||
warmup_epoch: 5
|
||||
warmup_start_lr: 1e-6
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- TimmAutoAugment:
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.25
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
batch_transform_ops:
|
||||
- OpSampler:
|
||||
MixupOperator:
|
||||
alpha: 0.8
|
||||
prob: 0.5
|
||||
CutmixOperator:
|
||||
alpha: 1.0
|
||||
prob: 0.5
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/safety_helmet/
|
||||
cls_label_path: ./dataset/safety_helmet/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/safety_helmet/safety_helmet_test_1.png
|
||||
batch_size: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: ThreshOutput
|
||||
threshold: 0.5
|
||||
label_0: wearing_helmet
|
||||
label_1: unwearing_helmet
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
- TprAtFpr:
|
||||
max_fpr: 0.0001
|
||||
- TopkAcc:
|
||||
topk: [1]
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
base_config_file: ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_search.yaml
|
||||
distill_config_file: ppcls/configs/PULC/safety_helmet/PPLCNet_x1_0_distillation.yaml
|
||||
|
||||
gpus: 0,1,2,3
|
||||
output_dir: output/search_safety_helmet
|
||||
search_times: 1
|
||||
search_dict:
|
||||
- search_key: lrs
|
||||
replace_config:
|
||||
- Optimizer.lr.learning_rate
|
||||
search_values: [0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.10, 0.11, 0.11, 0.12]
|
||||
- search_key: resolutions
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.1.RandCropImage.size
|
||||
- DataLoader.Train.dataset.transform_ops.3.TimmAutoAugment.img_size
|
||||
search_values: [176, 192, 224]
|
||||
- search_key: ra_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.3.TimmAutoAugment.prob
|
||||
search_values: [0.0, 0.1, 0.5]
|
||||
- search_key: re_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.5.RandomErasing.EPSILON
|
||||
search_values: [0.0, 0.1, 0.5]
|
||||
- search_key: lr_mult_list
|
||||
replace_config:
|
||||
- Arch.lr_mult_list
|
||||
search_values:
|
||||
- [0.0, 0.2, 0.4, 0.6, 0.8, 1.0]
|
||||
- [0.0, 0.4, 0.4, 0.8, 0.8, 1.0]
|
||||
- [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
|
||||
teacher:
|
||||
algorithm: "udml"
|
||||
final_replace:
|
||||
Arch.lr_mult_list: Arch.models.1.Student.lr_mult_list
|
||||
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
start_eval_epoch: 40
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: MobileNetV3_small_x0_35
|
||||
class_num: 4
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 1.3
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- RandFlipImage:
|
||||
flip_code: 1
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 16
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 16
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: ddeploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/text_image_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: 4
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.4
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- TimmAutoAugment:
|
||||
prob: 0.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/text_image_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,164 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
use_dali: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "DistillationModel"
|
||||
class_num: &class_num 4
|
||||
# if not null, its lengths should be same as models
|
||||
pretrained_list:
|
||||
# if not null, its lengths should be same as models
|
||||
freeze_params_list:
|
||||
- True
|
||||
- False
|
||||
use_sync_bn: True
|
||||
models:
|
||||
- Teacher:
|
||||
name: ResNet101_vd
|
||||
class_num: *class_num
|
||||
- Student:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: *class_num
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
|
||||
|
||||
infer_model_name: "Student"
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- DistillationDMLLoss:
|
||||
weight: 1.0
|
||||
model_name_pairs:
|
||||
- ["Student", "Teacher"]
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.4
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/train_list_for_distill.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- TimmAutoAugment:
|
||||
prob: 0.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/text_image_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- DistillationTopkAcc:
|
||||
model_key: "Student"
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,146 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
start_eval_epoch: 40
|
||||
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: 4
|
||||
pretrained: True
|
||||
use_ssld: True
|
||||
|
||||
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.04
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00003
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- TimmAutoAugment:
|
||||
prob: 0.0
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.0
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 64
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 4
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/text_image_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,157 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
eval_interval: 1
|
||||
epochs: 60
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: SwinTransformer_tiny_patch4_window7_224
|
||||
class_num: 4
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
epsilon: 0.1
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
Optimizer:
|
||||
name: AdamW
|
||||
beta1: 0.9
|
||||
beta2: 0.999
|
||||
epsilon: 1e-8
|
||||
weight_decay: 0.05
|
||||
no_weight_decay_name: absolute_pos_embed relative_position_bias_table .bias norm
|
||||
one_dim_param_no_weight_decay: True
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 2.5e-4
|
||||
eta_min: 1e-5
|
||||
warmup_epoch: 20
|
||||
warmup_start_lr: 1e-6
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- TimmAutoAugment:
|
||||
config_str: rand-m9-mstd0.5-inc1
|
||||
interpolation: bicubic
|
||||
img_size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- RandomErasing:
|
||||
EPSILON: 0.25
|
||||
sl: 0.02
|
||||
sh: 1.0/3.0
|
||||
r1: 0.3
|
||||
attempt: 10
|
||||
use_log_aspect: True
|
||||
mode: pixel
|
||||
batch_transform_ops:
|
||||
- OpSampler:
|
||||
MixupOperator:
|
||||
alpha: 0.8
|
||||
prob: 0.5
|
||||
CutmixOperator:
|
||||
alpha: 1.0
|
||||
prob: 0.5
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_image_orientation/
|
||||
cls_label_path: ./dataset/text_image_orientation/test_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/text_image_orientation/img_rot0_demo.jpg
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
interpolation: bicubic
|
||||
backend: pil
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 2
|
||||
class_id_map_file: ppcls/utils/PULC/text_image_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
base_config_file: ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_search.yaml
|
||||
distill_config_file: ppcls/configs/PULC/text_image_orientation/PPLCNet_x1_0_distillation.yaml
|
||||
|
||||
gpus: 0,1,2,3
|
||||
output_dir: output/search_text_image_orientation
|
||||
search_times: 1
|
||||
search_dict:
|
||||
- search_key: lrs
|
||||
replace_config:
|
||||
- Optimizer.lr.learning_rate
|
||||
search_values: [0.1, 0.2, 0.4, 0.8]
|
||||
- search_key: resolutions
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.1.RandCropImage.size
|
||||
- DataLoader.Train.dataset.transform_ops.2.TimmAutoAugment.img_size
|
||||
search_values: [176, 192, 224]
|
||||
- search_key: ra_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.2.TimmAutoAugment.prob
|
||||
search_values: [0.0, 0.1, 0.5]
|
||||
- search_key: re_probs
|
||||
replace_config:
|
||||
- DataLoader.Train.dataset.transform_ops.4.RandomErasing.EPSILON
|
||||
search_values: [0.0, 0.1, 0.5]
|
||||
- search_key: lr_mult_list
|
||||
replace_config:
|
||||
- Arch.lr_mult_list
|
||||
search_values:
|
||||
- [0.0, 0.0, 0.3, 0.5, 0.8, 1.0]
|
||||
- [0.0, 0.2, 0.4, 0.6, 0.8, 1.0]
|
||||
- [0.0, 0.4, 0.4, 0.8, 0.8, 1.0]
|
||||
- [1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
|
||||
teacher:
|
||||
rm_keys:
|
||||
- Arch.lr_mult_list
|
||||
search_values:
|
||||
- ResNet101_vd
|
||||
- ResNet50_vd
|
||||
final_replace:
|
||||
Arch.lr_mult_list: Arch.models.1.Student.lr_mult_list
|
||||
|
||||
|
|
@ -20,7 +20,7 @@ Global:
|
|||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: MobileNetV3_large_x1_0
|
||||
name: MobileNetV3_small_x0_35
|
||||
class_num: 2
|
||||
pretrained: True
|
||||
use_sync_bn: True
|
||||
|
|
@ -45,7 +45,7 @@ Optimizer:
|
|||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00002
|
||||
coeff: 0.00001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
|
|
@ -53,8 +53,8 @@ DataLoader:
|
|||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/train_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -79,8 +79,8 @@ DataLoader:
|
|||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/val_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -104,7 +104,7 @@ DataLoader:
|
|||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
|
|
@ -122,8 +122,8 @@ Infer:
|
|||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
|
|
@ -53,8 +53,8 @@ DataLoader:
|
|||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/train_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -92,8 +92,8 @@ DataLoader:
|
|||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/val_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -115,7 +115,7 @@ DataLoader:
|
|||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
|
|
@ -131,8 +131,8 @@ Infer:
|
|||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
# global configs
|
||||
Global:
|
||||
checkpoints: null
|
||||
pretrained_model: null
|
||||
output_dir: ./output/
|
||||
device: gpu
|
||||
save_interval: 1
|
||||
eval_during_train: True
|
||||
start_eval_epoch: 18
|
||||
eval_interval: 1
|
||||
epochs: 20
|
||||
print_batch_step: 10
|
||||
use_visualdl: False
|
||||
# used for static mode and model export
|
||||
image_shape: [3, 224, 224]
|
||||
save_inference_dir: ./inference
|
||||
# training model under @to_static
|
||||
to_static: False
|
||||
use_dali: False
|
||||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: PPLCNet_x1_0
|
||||
class_num: 2
|
||||
pretrained: True
|
||||
|
||||
# loss function config for traing/eval process
|
||||
Loss:
|
||||
Train:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
Eval:
|
||||
- CELoss:
|
||||
weight: 1.0
|
||||
|
||||
|
||||
Optimizer:
|
||||
name: Momentum
|
||||
momentum: 0.9
|
||||
lr:
|
||||
name: Cosine
|
||||
learning_rate: 0.04
|
||||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00004
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
DataLoader:
|
||||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- RandCropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 256
|
||||
drop_last: False
|
||||
shuffle: True
|
||||
loader:
|
||||
num_workers: 16
|
||||
use_shared_memory: True
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
sampler:
|
||||
name: DistributedBatchSampler
|
||||
batch_size: 128
|
||||
drop_last: False
|
||||
shuffle: False
|
||||
loader:
|
||||
num_workers: 8
|
||||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
channel_first: False
|
||||
- ResizeImage:
|
||||
resize_short: 256
|
||||
- CropImage:
|
||||
size: 224
|
||||
- NormalizeImage:
|
||||
scale: 1.0/255.0
|
||||
mean: [0.485, 0.456, 0.406]
|
||||
std: [0.229, 0.224, 0.225]
|
||||
order: ''
|
||||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
Eval:
|
||||
- TopkAcc:
|
||||
topk: [1, 2]
|
||||
|
|
@ -72,8 +72,8 @@ DataLoader:
|
|||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/train_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -110,8 +110,8 @@ DataLoader:
|
|||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/val_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -133,7 +133,7 @@ DataLoader:
|
|||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
|
|
@ -149,8 +149,8 @@ Infer:
|
|||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
|
|
@ -54,8 +54,8 @@ DataLoader:
|
|||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/train_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -93,8 +93,8 @@ DataLoader:
|
|||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/val_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -116,7 +116,7 @@ DataLoader:
|
|||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
|
|
@ -132,8 +132,8 @@ Infer:
|
|||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
|
|
@ -62,8 +62,8 @@ DataLoader:
|
|||
Train:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/train_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/train_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -109,8 +109,8 @@ DataLoader:
|
|||
Eval:
|
||||
dataset:
|
||||
name: ImageNetDataset
|
||||
image_root: ./dataset/text_direction/
|
||||
cls_label_path: ./dataset/text_direction/val_list.txt
|
||||
image_root: ./dataset/textline_orientation/
|
||||
cls_label_path: ./dataset/textline_orientation/val_list.txt
|
||||
transform_ops:
|
||||
- DecodeImage:
|
||||
to_rgb: True
|
||||
|
|
@ -134,7 +134,7 @@ DataLoader:
|
|||
use_shared_memory: True
|
||||
|
||||
Infer:
|
||||
infer_imgs: docs/images/inference_deployment/whl_demo.jpg
|
||||
infer_imgs: deploy/images/PULC/textline_orientation/textline_orientation_test_0_0.png
|
||||
batch_size: 10
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
|
|
@ -152,8 +152,8 @@ Infer:
|
|||
- ToCHWImage:
|
||||
PostProcess:
|
||||
name: Topk
|
||||
topk: 5
|
||||
class_id_map_file: ppcls/utils/imagenet1k_label_list.txt
|
||||
topk: 1
|
||||
class_id_map_file: ppcls/utils/PULC/textline_orientation_label_list.txt
|
||||
|
||||
Metric:
|
||||
Train:
|
||||
|
|
@ -16,7 +16,7 @@ Global:
|
|||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: MobileNetV3_large_x1_0
|
||||
name: MobileNetV3_small_x0_35
|
||||
class_num: 232
|
||||
pretrained: True
|
||||
|
||||
|
|
@ -40,7 +40,7 @@ Optimizer:
|
|||
warmup_epoch: 5
|
||||
regularizer:
|
||||
name: 'L2'
|
||||
coeff: 0.00002
|
||||
coeff: 0.00001
|
||||
|
||||
|
||||
# data loader for train and eval
|
||||
|
|
@ -17,7 +17,7 @@ Global:
|
|||
|
||||
# model architecture
|
||||
Arch:
|
||||
name: "MobileNetV3_large_x1_0"
|
||||
name: "MobileNetV3_small_x0_35"
|
||||
pretrained: True
|
||||
class_num: 19
|
||||
infer_add_softmax: False
|
||||
|
|
@ -18,7 +18,7 @@ from . import topk, threshoutput
|
|||
|
||||
from .topk import Topk, MultiLabelTopk
|
||||
from .threshoutput import ThreshOutput
|
||||
from .attr_rec import VehicleAttribute
|
||||
from .attr_rec import VehicleAttribute, PersonAttribute
|
||||
|
||||
|
||||
def build_postprocess(config):
|
||||
|
|
|
|||
|
|
@ -69,3 +69,105 @@ class VehicleAttribute(object):
|
|||
"file_name": file_names[idx]
|
||||
})
|
||||
return batch_res
|
||||
|
||||
|
||||
|
||||
class PersonAttribute(object):
|
||||
def __init__(self,
|
||||
threshold=0.5,
|
||||
glasses_threshold=0.3,
|
||||
hold_threshold=0.6):
|
||||
self.threshold = threshold
|
||||
self.glasses_threshold = glasses_threshold
|
||||
self.hold_threshold = hold_threshold
|
||||
|
||||
def __call__(self, x, file_names=None):
|
||||
if isinstance(x, dict):
|
||||
x = x['logits']
|
||||
assert isinstance(x, paddle.Tensor)
|
||||
if file_names is not None:
|
||||
assert x.shape[0] == len(file_names)
|
||||
x = F.sigmoid(x).numpy()
|
||||
|
||||
# postprocess output of predictor
|
||||
age_list = ['AgeLess18', 'Age18-60', 'AgeOver60']
|
||||
direct_list = ['Front', 'Side', 'Back']
|
||||
bag_list = ['HandBag', 'ShoulderBag', 'Backpack']
|
||||
upper_list = ['UpperStride', 'UpperLogo', 'UpperPlaid', 'UpperSplice']
|
||||
lower_list = [
|
||||
'LowerStripe', 'LowerPattern', 'LongCoat', 'Trousers', 'Shorts',
|
||||
'Skirt&Dress'
|
||||
]
|
||||
batch_res = []
|
||||
for idx, res in enumerate(x):
|
||||
res = res.tolist()
|
||||
label_res = []
|
||||
# gender
|
||||
gender = 'Female' if res[22] > self.threshold else 'Male'
|
||||
label_res.append(gender)
|
||||
# age
|
||||
age = age_list[np.argmax(res[19:22])]
|
||||
label_res.append(age)
|
||||
# direction
|
||||
direction = direct_list[np.argmax(res[23:])]
|
||||
label_res.append(direction)
|
||||
# glasses
|
||||
glasses = 'Glasses: '
|
||||
if res[1] > self.glasses_threshold:
|
||||
glasses += 'True'
|
||||
else:
|
||||
glasses += 'False'
|
||||
label_res.append(glasses)
|
||||
# hat
|
||||
hat = 'Hat: '
|
||||
if res[0] > self.threshold:
|
||||
hat += 'True'
|
||||
else:
|
||||
hat += 'False'
|
||||
label_res.append(hat)
|
||||
# hold obj
|
||||
hold_obj = 'HoldObjectsInFront: '
|
||||
if res[18] > self.hold_threshold:
|
||||
hold_obj += 'True'
|
||||
else:
|
||||
hold_obj += 'False'
|
||||
label_res.append(hold_obj)
|
||||
# bag
|
||||
bag = bag_list[np.argmax(res[15:18])]
|
||||
bag_score = res[15 + np.argmax(res[15:18])]
|
||||
bag_label = bag if bag_score > self.threshold else 'No bag'
|
||||
label_res.append(bag_label)
|
||||
# upper
|
||||
upper_res = res[4:8]
|
||||
upper_label = 'Upper:'
|
||||
sleeve = 'LongSleeve' if res[3] > res[2] else 'ShortSleeve'
|
||||
upper_label += ' {}'.format(sleeve)
|
||||
for i, r in enumerate(upper_res):
|
||||
if r > self.threshold:
|
||||
upper_label += ' {}'.format(upper_list[i])
|
||||
label_res.append(upper_label)
|
||||
# lower
|
||||
lower_res = res[8:14]
|
||||
lower_label = 'Lower: '
|
||||
has_lower = False
|
||||
for i, l in enumerate(lower_res):
|
||||
if l > self.threshold:
|
||||
lower_label += ' {}'.format(lower_list[i])
|
||||
has_lower = True
|
||||
if not has_lower:
|
||||
lower_label += ' {}'.format(lower_list[np.argmax(lower_res)])
|
||||
|
||||
label_res.append(lower_label)
|
||||
# shoe
|
||||
shoe = 'Boots' if res[14] > self.threshold else 'No boots'
|
||||
label_res.append(shoe)
|
||||
|
||||
threshold_list = [0.5] * len(res)
|
||||
threshold_list[1] = self.glasses_threshold
|
||||
threshold_list[18] = self.hold_threshold
|
||||
pred_res = (np.array(res) > np.array(threshold_list)
|
||||
).astype(np.int8).tolist()
|
||||
|
||||
batch_res.append({"attributes": label_res, "output": pred_res})
|
||||
return batch_res
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue