> Remember to give it a !
[](https://star-history.com/#thomas-yanxin/Smart_container&Date)
## 😉Recent Update😉
-**Release training code**：Publish model training code and experimental results for adaptive tuning;
-**Fixe Code Bug**： Restart the service after the database is updated in Pipeline deployment mode;
-**Improve Accuracy Greatly**：The test accuracy of the self-collected data set is **98.442%**；
-**Upgrade Document**：Provides detailed documentation of [PP-ShiTu model training and deployment](https://github.com/thomas-yanxin/Smart_container/blob/master/docs/readme_industry_v5.md)；
-**Optimize Deployment Mode**：The predicted speed increase is **65**%, based on the overall CPU flow control at **0.9s**；
-**Upgrade Product Function**：Add inventory information management function, provide one-click data visualization analysis platform;
## 🧁Project Context🧁
<fontsize=3>Currently in the process of actual operations of the retail industry, will produce a great human cost, such as guides, cleaning, settlement, and among them, especially need to spend a lot of labor cost and time cost in the identification of goods and settlement in the process of the price, and in the process, and so the customer need to wait in line. As a result, the retail industry has high labor costs and low work efficiency. Secondly, it also reduces the shopping experience of customers.
With the development of computer vision technology, as well as the unmanned and automated supermarket operation concept, the use of image recognition technology and target detection technology to achieve Automatic product identification and Automatic settlement demand, namely Automatic checkout system (ACO). The automatic checkout system based on computer vision can effectively reduce the operating cost of retail industry, improve the checkout efficiency of customers, so as to further enhance the user experience and happiness in the process of shopping. </font>
## Applicable Scene
>**ColugoMum——Intelligent Retail Rettlement Platform**Committed to provide **the largest offline retail experience store** with retail settlement solution based on vision.
## Pain Problem
1.**Settlement efficiency is highly required**：In the shopping scenario of large offline retail experience stores, if customers buy more goods, the traditional bar code settlement is adopted, which is inefficient and leads to poor shopping experience of customers；
2.**Category update very fast**：For in the new retail industry, new products are almost always updated on an hourly basis. Every time new products are added, the model has to be retrained so hard that it is impossible for a single model to keep pace；
3.**Different products are very similar**：Different flavors of the same drink, for example, are likely to have very similar packaging. And even for the same product, **there are often considerable differences in the product images obtained under different circumstances**;
4.**Tens of thousands of commodity categories**：There is no way to put all categories into the training set beforehand。
## 🍑Realize function🍑
<fontsize=3>**ColugoMumr**ealize automatic settlement of goods purchased by users in the retail process. We take advantage of the PaddleClas team's open source [PP-ShiTu](https://arxiv.org/pdf/2111.00775.pdf) technology, precise positioning of customers to buy goods, and intelligent, automatic price settlement. When customers place their chosen products in the designated area, **ColugoMum** can accurately locate and identify each product, and can return a complete shopping list and the actual total price of goods that customers should pay. When the system has a new product increase, the system only need to update the retrieval database, without retraining the model.
This project is a lightweight general PP - ShiTu image recognition system provides the solid ground application cases, the new one of the retail industry and retail visual intelligent solution provides a very good basis and train of thought, especially for solving many categories, small sample, high similarity, and frequently updated the special image recognition scene pain difficulties provides reference of demonstrations, Greatly reduce the retail industry in the actual operation of the huge human cost, improve the retail industry unmanned, automation, intelligent level. </font>
<fontsize=3>**ColugoMum** Based on PaddleClas as the main feature development suite, leveraging its open source [PP-ShiTu](https://arxiv.org/pdf/2111.00775.pdf) for core feature development. Through PaddleInference, it was deployed in Jetson Nano, and was packaged based on [QPT](https://github.com/QPT-Family/QPT) to develop an industrial-level intelligent retail settlement platform in line with actual application requirements. </font>
PP-ShiTu is a practical lightweight general image recognition system, which is mainly composed of three modules: subject detection, feature learning and vector retrieval. The system from the selection and adjustment of backbone networks, the choice of loss function, vector data, transform strategy, choice of regularization parameter, use the training model and quantitative model cutting eight aspects, use a variety of strategies, optimize the model of the various modules, finally got on the CPU is only 0.2 s to complete 10 w + library image recognition system.
<fontsize=3>The whole image recognition system is divided into three steps（[See PP-ShiTu training module for details](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/recognition.md)）：
(1) The candidate regions of image objects are detected by a target detection model;
(2) Feature extraction for each candidate region;
(3) Feature matching with images in the retrieval database, and extraction of recognition results.
For the new unknown category, there is no need to retrain the model, but only need to add the image of the category in the retrieval database and rebuild the retrieval database to recognize the category. </font>
### Introduction to Data Set
【The first one】:[Products-10K Large Scale Product Recognition Dataset](https://arxiv.org/abs/2006.12634)
【The second one】:[RP2K: A Large-Scale Retail Product Dataset for Fine-Grained Image Classification](https://www.pinlandata.com/rp2k_dataset)
**ColugoMum** based on the above two data sets and combined with the actual characteristics of the retail scene, adaptive processing is carried out.
### List of Commodities
> 东古酱油一品鲜
东古黄豆酱750G
东鹏特饮罐装
中华（硬）
中华（软）
乳酸菌600亿_2
乳酸菌600亿_3
乳酸菌600亿原味
乳酸菌600亿芒果
乳酸菌600亿芦荟
...
The processed dataset is now open source in [AIStudio](https://aistudio.baidu.com/aistudio/datasetdetail/113685). </font>
**ColugoMum** has been connected to**Jetson Nano, Windows, linux** system.
<fontsize=3>
- Windows
[**ColugoMum** provides a relatively simple demo version]
We use [QPT](https://github.com/QPT-Family/QPT) for packaging.
Download the project code, enter the QPT_client folder, and Click the "启动程序.exe".
- Linux
Download the project code, enter the client folder, and run the following code to run it ：
```
python client.py
```
- For details of the image recognition part deployment, you can see [PP-ShiTu Development](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/inference_deployment/python_deploy.md#%E4%B8%BB%E4%BD%93%E6%A3%80%E6%B5%8B%E3%80%81%E7%89%B9%E5%BE%81%E6%8F%90%E5%8F%96%E5%92%8C%E5%90%91%E9%87%8F%E6%A3%80%E7%B4%A2%E4%B8%B2%E8%81%94)
- Wechat applet
Open the wechat developer tool, import the AIContainer folder under the system folder and run it；
- [PaddleClas](https://github.com/PaddlePaddle/PaddleClas) provides the image recognition core function development;
- [QPT](https://github.com/QPT-Family/QPT) provides Windows side package;
- [jkfx](https://github.com/jkfx) fixed some bugs.
## ❤️Welcome to build together❤️
We welcome you to contribute code or provide suggestions for "**ColugoMum**". Whether you have a bug, fix a bug, or add a new feature, feel free to submit Issue or Pull Requests.
