13 KiB

Raw Blame History

Quick Start of Recognition

This tutorial contains 3 parts: Environment Preparation, Image Recognition Experience, and Unknown Category Image Recognition Experience.

If the image category already exists in the image index database, then you can take a reference to chapter Image Recognition Experience，to complete the progress of image recognition；If you wish to recognize unknow category image, which is not included in the index database，you can take a reference to chapter Unknown Category Image Recognition Experience，to complete the process of creating an index to recognize it。

Catalogue

1. Enviroment Preparation
2. Image Recognition Experience
- 2.1 Download and Unzip the Inference Model and Demo Data
- 2.2 Product Recognition and Retrieval
  - 2.2.1 Single Image Recognition
  - 2.2.2 Folder-based Batch Recognition
3. Unknown Category Image Recognition Experience

1. Enviroment Preparation

Installation：Please take a reference to Quick Installation to configure the PaddleClas environment.
Using the following command to enter Folder deploy. All content and commands in this section need to be run in folder deploy.
```
cd deploy
```

2. Image Recognition Experience

The detection model with the recognition inference model for the 4 directions (Logo, Cartoon Face, Vehicle, Product), the address for downloading the test data and the address of the corresponding configuration file are as follows.

Models Introduction	Recommended Scenarios	Test Data Address	inference Model	Predict Config File	Config File to Build Index Database
Generic mainbody detection model	General Scenarios	-	Model Download Link	-	-
Logo Recognition Model	Logo Scenario	Data Download Link	Model Download Link	inference_logo.yaml	build_logo.yaml
Cartoon Face Recognition Model	Cartoon Face Scenario	Data Download Link	Model Download Link	inference_cartoon.yaml	build_cartoon.yaml
Vehicle Subclassification Model	Vehicle Scenario	Data Download Link	Model Download Link	inference_vehicle.yaml	build_vehicle.yaml
Product Recignition Model	Product Scenario	Data Download Link	Model Download Link	inference_inshop.yaml	build_inshop.yaml

Attention：If you do not have wget installed on Windows, you can download the model by copying the link into your browser and unzipping it in the appropriate folder; for Linux or macOS users, you can right-click and copy the download link to download it via the wget command.

You can download and unzip the data and models by following the command below

mkdir dataset
cd dataset
# Download the demo data and unzip
wget {Data download link} && tar -xf {Name of the tar archive}
cd ..

mkdir models
cd models
# Download and unzip the inference model
wget {Models download link} && tar -xf {Name of the tar archive}
cd ..

2.1 Download and Unzip the Inference Model and Demo Data

Take the product recognition as an example, download the detection model, recognition model and product recognition demo data with the following commands.

mkdir models
cd models
# Download the generic detection inference model and unzip it
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar && tar -xf ppyolov2_r50vd_dcn_mainbody_v1.0_infer.tar
# Download and unpack the inference model
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/models/inference/product_ResNet50_vd_aliproduct_v1.0_infer.tar && tar -xf product_ResNet50_vd_aliproduct_v1.0_infer.tar

cd ..
mkdir dataset
cd dataset
# Download the demo data and unzip it
wget https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/rec/data/product_demo_data_v1.0.tar && tar -xf product_demo_data_v1.0.tar
cd ..

Once unpacked, the dataset folder should have the following file structure.

├── product_demo_data_v1.0
│   ├── data_file.txt
│   ├── gallery
│   ├── index
│   └── query
├── ...

The data_file.txt is images list used to build the index database, the gallery folder contains all the original images used to build the index database, the index folder contains the index files generated by building the index database, and the query is the demo image used to test the recognition effect.

The models folder should have the following file structure.

├── product_ResNet50_vd_aliproduct_v1.0_infer
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel
├── ppyolov2_r50vd_dcn_mainbody_v1.0_infer
│   ├── inference.pdiparams
│   ├── inference.pdiparams.info
│   └── inference.pdmodel

2.2 Product Recognition and Retrival

Take the product recognition demo as an example to show the recognition and retrieval process (if you wish to try other scenarios of recognition and retrieval, replace the corresponding configuration file after downloading and unzipping the corresponding demo data and model to complete the prediction)。

2.2.1 Single Image Recognition

Run the following command to identify and retrieve the image ./dataset/product_demo_data_v1.0/query/wangzai.jpg for recognition and retrieval

python3.7 python/predict_system.py -c configs/inference_product.yaml

The image to be retrieved is shown below.

The final output is shown below.

[{'bbox': [305, 226, 776, 930], 'rec_docs': ['旺仔牛奶', '旺仔牛奶', '旺仔牛奶', '旺仔牛奶', '康师傅方便面'], 'rec_scores': array([1328.1072998 , 1185.92248535,  846.88220215,  746.28546143 622.2668457 ])}

where bbox indicates the location of the detected subject, rec_docs indicates the labels corresponding to a number of images in the index dabase that are most similar to the detected subject, and rec_scores indicates the corresponding similarity.

There are 4 旺仔牛奶 results in 5, the recognition result is correct.

The detection result is also saved in the folder output, which is shown as follows.

2.2.2 Folder-based Batch Recognition

If you want to predict the images in the folder, you can directly modify the Global.infer_imgs field in the configuration file, or you can also modify the corresponding configuration through the following -o parameter.

python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./dataset/product_demo_data_v1.0/query/"

Furthermore, the recognition inference model path can be changed by modifying the Global.rec_inference_model_dir field, and the path of the index to the index databass can be changed by modifying the IndexProcess.index_path field.

3. Recognize Images of Unknown Category

To recognize the image ./dataset/product_demo_data_v1.0/query/anmuxi.jpg, run the command as follows:

python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./dataset/product_demo_data_v1.0/query/anmuxi.jpg"

The image to be retrieved is shown below.

The output is as follows:

[{'bbox': [243, 80, 523, 522], 'rec_docs': ['娃哈哈AD钙奶', '旺仔牛奶', '娃哈哈AD钙奶', '农夫山泉矿泉水', '红牛'], 'rec_scores': array([548.33282471, 411.85687256, 408.39770508, 400.89404297, 360.41540527])}]

Since the index infomation is not included in the corresponding index databse, the recognition results are not proper. At this time, we can complete the image recognition of unknown categories by constructing a new index database.

When the index database cannot cover the scenes we actually recognise, i.e. when predicting images of unknown categories, we need to add similar images of the corresponding categories to the index databasey, thus completing the recognition of images of unknown categories ，which does not require retraining.

3.1 Prepare for the new images and labels

First, you need to copy the images which are similar with the image to retrieval to the original images for the index database. The command is as follows.

cp -r  ../docs/images/recognition/product_demo/gallery/anmuxi ./dataset/product_demo_data_v1.0/gallery/

Then you need to create a new label file which records the image path and label information. Use the following command to create a new file based on the original one.

# copy the file
cp dataset/product_demo_data_v1.0/data_file.txt dataset/product_demo_data_v1.0/data_file_update.txt

Then add some new lines into the new label file, which is shown as follows.

gallery/anmuxi/001.jpg 安慕希酸奶
gallery/anmuxi/002.jpg 安慕希酸奶
gallery/anmuxi/003.jpg 安慕希酸奶
gallery/anmuxi/004.jpg 安慕希酸奶
gallery/anmuxi/005.jpg 安慕希酸奶
gallery/anmuxi/006.jpg 安慕希酸奶

Each line can be splited into two fields. The first field denotes the relative image path, and the second field denotes its label. The delimiter is space here.

3.2 Build a new Index Base Library

Use the following command to build the index to accelerate the retrieval process after recognition.

python3.7 python/build_gallery.py -c configs/build_product.yaml -o IndexProcess.data_file="./dataset/product_demo_data_v1.0/data_file_update.txt" -o IndexProcess.index_path="./dataset/product_demo_data_v1.0/index_update"

Finally, the new index information is stored in the folder./dataset/product_demo_data_v1.0/index_update. Use the new index database for the above index.

3.2 Recognize the Unknown Category Images

To recognize the image ./dataset/product_demo_data_v1.0/query/anmuxi.jpg, run the command as follows.

python3.7 python/predict_system.py -c configs/inference_product.yaml -o Global.infer_imgs="./dataset/product_demo_data_v1.0/query/anmuxi.jpg" -o IndexProcess.index_path="./dataset/product_demo_data_v1.0/index_update"