mirrors
/
deep-person-reid
mirror of https://github.com/KaiyangZhou/deep-person-reid.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

update readme & dataset description

pull/119/head
KaiyangZhou 2019-02-27 11:20:50 +00:00
parent 3fe5cb118c
commit 9b1c131e9e
2 changed files with 61 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

96
DATASETS.md
View File

@ -7,31 +7,30 @@ mkdir data/
```
Note that
- if you wanna store datasets in another directory, you need to specify `--root path_to_your/data` when running the training code.
- please follow the instructions below to prepare each dataset. After that, you can simply give the corresponding keys to the training scripts to use the datasets, e.g. `-s market1501` (use Market1501 as the training dataset).
- please follow the instructions below to prepare each dataset. After that, you can simply use pre-defined keys to build the datasets, e.g. `-s market1501` (use Market1501 as the training dataset).
- please do not assign image-reid's dataset keys to video-reid's training scripts, otherwise error would occur, and vice versa. (see [torchreid/data_manager.py](torchreid/data_manager.py))
- please use the suggested names for the dataset folders, otherwise you have to modify the `dataset_dir` attribute in the specific `dataset.py` file in `torchreid/datasets/` accordingly.
- if you find any errors/bugs, please report in the Issues section.
- in the following, we assume that the path to the dataset directory is `data/`.
- in the following, we assume that the path to the dataset directory is `data/`. However, you can store datasets in whatever location you want, all you need is to specify the root path with `--root path/to/your/data`.
## Image ReID
**Market1501**:
1. Download the dataset to `data/` from http://www.liangzheng.org/Project/project_reid.html.
2. Extract the file and rename it to `market1501`. The data structure should look like:
- Download the dataset to `data/` from http://www.liangzheng.org/Project/project_reid.html.
- Extract the file and rename it to `market1501`. The data structure should look like:
```
market1501/
bounding_box_test/
bounding_box_train/
...
```
3. Use `market1501` as the key to load Market1501.
4. To use the extra 500K distractors (i.e. Market1501 + 500K), go to the **Market-1501+500k Dataset** section at http://www.liangzheng.org/Project/project_reid.html, download the zip file (`distractors_500k.zip`), and extract it under `market1501/`. As a result, you will have a folder named `images/`. Use `--market1501-500k` to add these extra images to the gallery set.
- Use `market1501` as the key to load Market1501.
- To use the extra 500K distractors (i.e. Market1501 + 500K), go to the **Market-1501+500k Dataset** section at http://www.liangzheng.org/Project/project_reid.html, download the zip file (`distractors_500k.zip`), and extract it under `market1501/`. As a result, you will have a folder named `images/`. Use `--market1501-500k` to add these extra images to the gallery set when running the code.
**CUHK03**:
1. Create a folder named `cuhk03/` under `data/`.
2. Download the dataset to `data/cuhk03/` from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and extract `cuhk03_release.zip`, so you will have `data/cuhk03/cuhk03_release`.
3. Download the new split (767/700) from [person-re-ranking](https://github.com/zhunzhong07/person-re-ranking/tree/master/evaluation/data/CUHK03). What you need are `cuhk03_new_protocol_config_detected.mat` and `cuhk03_new_protocol_config_labeled.mat`; put these two mat files under `data/cuhk03`. Finally, the data structure should look like
- Create a folder named `cuhk03/` under `data/`.
- Download the dataset to `data/cuhk03/` from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and extract `cuhk03_release.zip`, so you will have `data/cuhk03/cuhk03_release`.
- Download the new split (767/700) from [person-re-ranking](https://github.com/zhunzhong07/person-re-ranking/tree/master/evaluation/data/CUHK03). What you need are `cuhk03_new_protocol_config_detected.mat` and `cuhk03_new_protocol_config_labeled.mat`; put these two mat files under `data/cuhk03`. Finally, the data structure should look like
```
cuhk03/
cuhk03_release/
@ -39,11 +38,11 @@ cuhk03/
cuhk03_new_protocol_config_labeled.mat
...
```
4. Use `cuhk03` as the dataset key. In the default mode, we load data using the new split (767/700). If you wanna use the original (20) splits (1367/100), please specify with `--cuhk03-classic-split`. As the CMC is computed differently from Market1501 for the 1367/100 split (see [here](http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html)), you need to specify `--use-metric-cuhk03` to activate the corresponding metric for fair comparison with some methods that adopt the original splits. In addition, we support both `labeled` and `detected` modes. The default mode loads `detected` images. Specify `--cuhk03-labeled` if you wanna train and test on `labeled` images.
- Use `cuhk03` as the dataset key. In the default mode, we load data using the new split (767/700). If you wanna use the original (20) splits (1367/100), please specify with `--cuhk03-classic-split`. As the CMC is computed differently from Market1501 for the 1367/100 split (see [here](http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html)), you need to specify `--use-metric-cuhk03` to activate the corresponding metric for fair comparison with some methods that adopt the original splits. In addition, we support both `labeled` and `detected` modes. The default mode loads `detected` images. Specify `--cuhk03-labeled` if you wanna train and test on `labeled` images.
**DukeMTMC-reID**:
1. The process is automated so you can simply do `-s dukemtmcreid -t dukemtmcreid`. The final folder structure looks like
- The process is automated so you can simply do `-s dukemtmcreid -t dukemtmcreid`. The final folder structure looks like
```
dukemtmc-reid/
DukeMTMC-reid.zip # (you can delete this zip file, it is ok)
@ -52,8 +51,8 @@ dukemtmc-reid/
**MSMT17**:
1. Create a directory named `msmt17/` under `data/`.
2. Download the dataset (e.g. `MSMT17_V1.tar.gz`) from http://www.pkuvmc.com/publications/msmt17.html to `data/msmt17/`. Extract the file under the same folder, you need to have
- Create a directory named `msmt17/` under `data/`.
- Download the dataset (e.g. `MSMT17_V1.tar.gz`) from http://www.pkuvmc.com/publications/msmt17.html to `data/msmt17/`. Extract the file under the same folder, you need to have
```
msmt17/
MSMT17_V1/ # different versions might differ in folder name
@ -64,10 +63,10 @@ msmt17/
list_gallery.txt
list_val.txt
```
3. Use `msmt17` as the key for this dataset.
- Use `msmt17` as the key for this dataset.
**VIPeR**:
1. The code supports automatic download and formatting. Just use `-s viper -t viper`. The final data structure would look like:
- The code supports automatic download and formatting. Just use `-s viper -t viper`. The final data structure would look like:
```
viper/
VIPeR/
@ -76,7 +75,7 @@ viper/
```
**GRID**:
1. The code supports automatic download and formatting. Just use `-s grid -t grid`. The final data structure would look like:
- The code supports automatic download and formatting. Just use `-s grid -t grid`. The final data structure would look like:
```
grid/
underground_reid/
@ -85,13 +84,13 @@ grid/
```
**CUHK01**:
1. Create `cuhk01/` under `data/`.
2. Download `CUHK01.zip` from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and place it in `cuhk01/`.
3. Do `-s cuhk01 -t cuhk01` to use the data.
- Create `cuhk01/` under `data/`.
- Download `CUHK01.zip` from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and place it in `cuhk01/`.
- Do `-s cuhk01 -t cuhk01` to use the data.
**PRID450S**:
18. The code supports automatic download and formatting. Just use `-s prid450s -t prid450s`. The final data structure would look like:
- The code supports automatic download and formatting. Just use `-s prid450s -t prid450s`. The final data structure would look like:
```
prid450s/
cam_a/
@ -100,36 +99,57 @@ prid450s/
splits.json
```
**QMUL-iLIDS**
- The code can automate download and formatting. The key to use this dataset is `-s ilids -t ilids`. The final data structure would look like:
```
ilids/
i-LIDS_Pedestrian/
Persons/
```
**PRID**
- Under `data/`, do `mkdir prid2011` to create a directory.
- Download the dataset from https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/ and extract it under `data/prid2011`.
- The data structure would look like
```
prid2011/
prid_2011/
single_shot/
multi_shot/
readme.txt
```
- Use `-s prid -t prid` to build the dataset.
**SenseReID**:
1. Create `sensereid/` under `data/`.
2. Download the dataset from this [link](https://drive.google.com/file/d/0B56OfSrVI8hubVJLTzkwV2VaOWM/view) and extract to `sensereid/`. The final folder structure should look like
- Create `sensereid/` under `data/`.
- Download the dataset from this [link](https://drive.google.com/file/d/0B56OfSrVI8hubVJLTzkwV2VaOWM/view) and extract to `sensereid/`. The final folder structure should look like
```
sensereid/
SenseReID/
test_probe/
test_gallery/
```
3. The command for using SenseReID is `-t sensereid`. Note that SenseReID is for test purpose only so training images are unavailable. Please use `--evaluate` along with `-t sensereid`.
- The command for using SenseReID is `-t sensereid`. Note that SenseReID is for test purpose only so training images are unavailable. Please use `--evaluate` along with `-t sensereid`.
## Video ReID
**MARS**:
1. Create a directory named `mars/` under `data/`.
2. Download the dataset to `data/mars/` from http://www.liangzheng.com.cn/Project/project_mars.html.
3. Extract `bbox_train.zip` and `bbox_test.zip`.
4. Download the split metadata from https://github.com/liangzheng06/MARS-evaluation/tree/master/info and put `info/` in `data/mars` (we want to follow the standard split). The data structure should look like:
- Create a directory named `mars/` under `data/`.
- Download the dataset to `data/mars/` from http://www.liangzheng.com.cn/Project/project_mars.html.
- Extract `bbox_train.zip` and `bbox_test.zip`.
- Download the split metadata from https://github.com/liangzheng06/MARS-evaluation/tree/master/info and put `info/` in `data/mars` (we want to follow the standard split). The data structure should look like:
```
mars/
bbox_test/
bbox_train/
info/
```
5. Use `mars` as the dataset key.
- Use `mars` as the dataset key.
**iLIDS-VID**:
1. The code supports automatic download and formatting. Simple use `-s ilidsvid -t ilidsvid`. The data structure would look like:
- The code supports automatic download and formatting. Simple use `-s ilidsvid -t ilidsvid`. The data structure would look like:
```
ilids-vid/
i-LIDS-VID/
@ -137,10 +157,10 @@ ilids-vid/
splits.json
```
**PRID**:
1. Under `data/`, do `mkdir prid2011` to create a directory.
2. Download the dataset from https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/ and extract it under `data/prid2011`.
3. Download the split created by [iLIDS-VID](http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html) from [here](http://www.eecs.qmul.ac.uk/~kz303/deep-person-reid/datasets/prid2011/splits_prid2011.json), and put it under `data/prid2011/`. Note that only 178 persons whose sequences are more than a threshold are used so that results on this dataset can be fairly compared with other approaches. The data structure would look like:
**PRID-2011**:
- Under `data/`, do `mkdir prid2011` to create a directory.
- Download the dataset from https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/ and extract it under `data/prid2011`.
- Download the split created by [iLIDS-VID](http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html) from [here](http://www.eecs.qmul.ac.uk/~kz303/deep-person-reid/datasets/prid2011/splits_prid2011.json), and put it under `data/prid2011/`. Note that only 178 persons whose sequences are more than a threshold are used so that results on this dataset can be fairly compared with other approaches. The data structure would look like:
```
prid2011/
splits_prid2011.json
@ -149,11 +169,11 @@ prid2011/
single_shot/
readme.txt
```
4. Use `-s prid2011 -t prid2011` when running the training code.
- Use `-s prid2011 -t prid2011` when running the training code.
**DukeMTMC-VideoReID**:
1. Use `-s dukemtmcvidreid -t dukemtmcvidreid` directly.
2. If you wanna download the dataset manually, get `DukeMTMC-VideoReID.zip` from https://github.com/Yu-Wu/DukeMTMC-VideoReID. Unzip the file to `data/dukemtmc-vidreid`. Ultimately, you need to have
- Use `-s dukemtmcvidreid -t dukemtmcvidreid` directly.
- If you wanna download the dataset manually, get `DukeMTMC-VideoReID.zip` from https://github.com/Yu-Wu/DukeMTMC-VideoReID. Unzip the file to `data/dukemtmc-vidreid`. Ultimately, you need to have
```
dukemtmc-vidreid/
DukeMTMC-VideoReID/
@ -178,7 +198,7 @@ These two classes are used for [torch.utils.data.DataLoader](http://pytorch.org/
- **CUHK03 (classic split)** has 20 fixed splits, so do `split_id=0~19`.
- **VIPeR** contains 632 identities each with 2 images under two camera views. Evaluation should be done for 10 random splits. Each split randomly divides 632 identities to 316 train ids (632 images) and the other 316 test ids (632 images). Note that, in each random split, there are two sub-splits, one using camera-A as query and camera-B as gallery while the other one using camera-B as query and camera-A as gallery. Thus, there are totally 20 splits generated with `split_id` starting from 0 to 19. Models can be trained on `split_id=[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]` (because `split_id=0` and `split_id=1` share the same train set, and so on and so forth.). At test time, models trained on `split_id=0` can be directly evaluated on `split_id=1`, models trained on `split_id=2` can be directly evaluated on `split_id=3`, and so on and so forth.
- **CUHK01** is similar to VIPeR in the split generation.
- **GRID** and **PRID450S** have 10 random splits, so evaluation should be done by varying `split_id` from 0 to 9.
- **GRID** , **PRID450S**, **iLIDS** and **PRID** have 10 random splits, so evaluation should be done by varying `split_id` from 0 to 9.
- **SenseReID** has no training images and is used for evaluation only.
## Video ReID

3
README.md
View File

@ -19,6 +19,7 @@ It has the following features:
- state-of-the-art training techniques.
## Updates
- 27-02-2019: Added QMUL-iLIDS and PRID. See [this issue](https://github.com/KaiyangZhou/deep-person-reid/issues/115) for details.
- 03-02-2019: [Added random erasing and staged learning rate training](https://github.com/KaiyangZhou/deep-person-reid/issues/110).
- 22-01-2019: Added [Market1501+500K](https://github.com/KaiyangZhou/deep-person-reid/issues/107).
- 06-01-2019: Released [Awesome-ReID](AWESOME_REID.md), a collection of ReID-related research with links to codes and papers.
@ -44,6 +45,8 @@ Image-reid datasets:
- [CUHK01](http://www.ee.cuhk.edu.hk/~xgwang/papers/liZWaccv12.pdf) (`cuhk01`)
- [PRID450S](https://pdfs.semanticscholar.org/f62d/71e701c9fd021610e2076b5e0f5b2c7c86ca.pdf) (`prid450s`)
- [SenseReID](http://openaccess.thecvf.com/content_cvpr_2017/papers/Zhao_Spindle_Net_Person_CVPR_2017_paper.pdf) (`sensereid`)
- [QMUL-iLIDS](http://www.eecs.qmul.ac.uk/~sgg/papers/ZhengGongXiang_BMVC09.pdf) (`ilids`)
- [PRID](https://pdfs.semanticscholar.org/4c1b/f0592be3e535faf256c95e27982db9b3d3d3.pdf) (`prid`)
Video-reid datasets:
- [MARS](http://www.liangzheng.org/1320.pdf) (`mars`)