2020-04-15 18:54:00 +08:00
|
|
|
# 数据说明
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## 1.简介
|
2020-04-20 21:19:44 +08:00
|
|
|
本文档介绍ImageNet1k和flowers102数据准备过程。
|
2020-04-15 18:54:00 +08:00
|
|
|
|
|
|
|
|
## 2.数据集准备
|
|
|
|
|
|
|
|
|
|
数据集 | 训练集大小 | 测试集大小 | 类别数 | 备注|
|
|
|
|
|
:------:|:---------------:|:---------------------:|:-----------:|:-----------:
|
2020-04-20 21:19:44 +08:00
|
|
|
[flowers102](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/)|1k | 6k | 102 |
|
|
|
|
|
[ImageNet1k](http://www.image-net.org/challenges/LSVRC/2012/)|1.2M| 50k | 1000 |
|
2020-04-15 18:54:00 +08:00
|
|
|
|
2020-04-20 21:19:44 +08:00
|
|
|
* 数据格式
|
2020-11-20 11:21:36 +08:00
|
|
|
按照如下结构组织数据，其中`train_list.txt`和`val_list.txt`的格式形如：
|
2020-04-15 18:54:00 +08:00
|
|
|
|
2020-04-20 21:19:44 +08:00
|
|
|
```shell
|
|
|
|
|
# 每一行采用"空格"分隔图像路径与标注
|
2020-04-19 16:10:23 +08:00
|
|
|
|
2020-11-20 11:21:36 +08:00
|
|
|
# 下面是train_list.txt中的格式样例
|
|
|
|
|
train/n01440764/n01440764_10026.JPEG 0
|
2020-04-19 16:10:23 +08:00
|
|
|
...
|
|
|
|
|
|
2020-11-20 11:21:36 +08:00
|
|
|
# 下面是val_list.txt中的格式样例
|
|
|
|
|
val/ILSVRC2012_val_00000001.JPEG 65
|
|
|
|
|
...
|
2020-04-19 16:10:23 +08:00
|
|
|
```
|
2020-11-20 11:21:36 +08:00
|
|
|
|
2020-04-17 18:34:58 +08:00
|
|
|
### ImageNet1k
|
|
|
|
|
从官方下载数据后，按如下组织数据
|
2020-04-15 18:54:00 +08:00
|
|
|
|
|
|
|
|
```bash
|
2020-11-20 11:21:36 +08:00
|
|
|
PaddleClas/dataset/ILSVRC2012/
|
2020-04-17 18:34:58 +08:00
|
|
|
|_ train/
|
2020-04-15 18:54:00 +08:00
|
|
|
| |_ n01440764
|
|
|
|
|
| | |_ n01440764_10026.JPEG
|
|
|
|
|
| | |_ ...
|
|
|
|
|
| |_ ...
|
|
|
|
|
| |
|
|
|
|
|
| |_ n15075141
|
|
|
|
|
| |_ ...
|
|
|
|
|
| |_ n15075141_9993.JPEG
|
2020-04-17 18:34:58 +08:00
|
|
|
|_ val/
|
2020-04-15 18:54:00 +08:00
|
|
|
| |_ ILSVRC2012_val_00000001.JPEG
|
|
|
|
|
| |_ ...
|
|
|
|
|
| |_ ILSVRC2012_val_00050000.JPEG
|
|
|
|
|
|_ train_list.txt
|
|
|
|
|
|_ val_list.txt
|
2020-04-15 18:58:10 +08:00
|
|
|
```
|
2020-11-20 11:21:36 +08:00
|
|
|
|
2020-04-20 21:19:44 +08:00
|
|
|
### Flowers102
|
2020-04-19 16:10:23 +08:00
|
|
|
从[VGG官方网站](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/)下载后的数据，解压后包括
|
2020-04-20 23:15:45 +08:00
|
|
|
|
|
|
|
|
```shell
|
2020-04-17 18:34:58 +08:00
|
|
|
jpg/
|
|
|
|
|
setid.mat
|
|
|
|
|
imagelabels.mat
|
2020-04-20 23:15:45 +08:00
|
|
|
```
|
|
|
|
|
|
2020-04-20 21:19:44 +08:00
|
|
|
将以上文件放置在PaddleClas/dataset/flowers102/下
|
2020-04-17 18:34:58 +08:00
|
|
|
|
2020-04-20 21:19:44 +08:00
|
|
|
通过运行generate_flowers102_list.py生成train_list.txt和val_list.txt
|
2020-04-15 18:54:00 +08:00
|
|
|
|
|
|
|
|
```bash
|
2020-04-20 21:19:44 +08:00
|
|
|
python generate_flowers102_list.py jpg train > train_list.txt
|
|
|
|
|
python generate_flowers102_list.py jpg valid > val_list.txt
|
2020-04-17 18:34:58 +08:00
|
|
|
|
|
|
|
|
```
|
|
|
|
|
按照如下结构组织数据：
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-04-20 21:19:44 +08:00
|
|
|
PaddleClas/dataset/flowers102/
|
2020-04-17 18:34:58 +08:00
|
|
|
|_ jpg/
|
2020-04-20 21:19:44 +08:00
|
|
|
| |_ image_03601.jpg
|
2020-04-15 18:54:00 +08:00
|
|
|
| |_ ...
|
|
|
|
|
| |_ image_02355.jpg
|
|
|
|
|
|_ train_list.txt
|
|
|
|
|
|_ val_list.txt
|
|
|
|
|
```
|
