# Datasets

Here we provide a guide on how to prepare datasets.

Suppose you wanna store the reid data in a directory called “path/to/reid-data/”, you need to specify the `root` as root=’path/to/reid-data/’ when initializing `DataManager`. Below we use `$REID` to denote “path/to/reid-data”.

Please refer to torchreid.data for details regarding the arguments.

Note

Dataset with a \(\dagger\) superscript means that the process is automated, so you can directly call the dataset in `DataManager` (which automatically downloads the dataset and organizes the data structure). However, we also provide a way below to help the manual setup in case the automation fails.

Note

The keys to use specific datasets are enclosed in the parantheses beside the datasets.

Note

You are suggested to use the provided names for dataset folders such as “market1501” for Market1501 and “dukemtmcreid” for DukeMTMC-reID when doing the manual setup, otherwise you need to modify the source code accordingly (i.e. the `dataset_dir` attribute).

- Image Datasets
  - Market1501 \(^\dagger\) (`market1501`)
  - CUHK03 (`cuhk03`)
  - DukeMTMC-reID \(^\dagger\) (`dukemtmcreid`)
  - MSMT17 (`msmt17`)
  - VIPeR \(^\dagger\) (`viper`)
  - GRID \(^\dagger\) (`grid`)
  - CUHK01 (`cuhk01`)
  - SenseReID (`sensereid`)
  - QMUL-iLIDS \(^\dagger\) (`ilids`)
  - PRID (`prid`)
- Video Datasets
  - MARS (`mars`)
  - iLIDS-VID \(^\dagger\) (`ilidsvid`)
  - PRID2011 (`prid2011`)
  - DukeMTMC-VideoReID \(^\dagger\) (`dukemtmcvidreid`)

## Image Datasets

### Market1501 \(^\dagger\) (`market1501`)

- Create a directory named “market1501” under `$REID`.
- Download the dataset to “market1501” from http://www.liangzheng.org/Project/project_reid.html and extract the files.
- The data structure should look like

    market1501/
        Market-1501-v15.09.15/
            query/
            bounding_box_train/
            bounding_box_test/

- To use the extra 500K distractors (i.e. Market1501 + 500K), go to the Market-1501+500k Dataset section at http://www.liangzheng.org/Project/project_reid.html, download the zip file “distractors_500k.zip” and extract it under “market1501/Market-1501-v15.09.15”. The argument to use these 500K distrctors is `market1501_500k` in `ImageDataManager`.

### CUHK03 (`cuhk03`)

- Create a folder named “cuhk03” under `$REID`.
- Download the dataset to “cuhk03/” from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and extract “cuhk03_release.zip”, resulting in “cuhk03/cuhk03_release/”.
- Download the new split (767/700) from person-re-ranking. What you need are “cuhk03_new_protocol_config_detected.mat” and “cuhk03_new_protocol_config_labeled.mat”. Put these two mat files under “cuhk03/”.
- The data structure should look like

    cuhk03/
        cuhk03_release/
        cuhk03_new_protocol_config_detected.mat
        cuhk03_new_protocol_config_labeled.mat

- In the default mode, we load data using the new split (767/700). If you wanna use the original (20) splits (1367/100), please set `cuhk03_classic_split` to True in `ImageDataManager`. As the CMC is computed differently from Market1501 for the 1367/100 split (see here), you need to enable `use_metric_cuhk03` in `ImageDataManager` to activate the single-gallery-shot metric for fair comparison with some methods that adopt the old splits (do not need to report mAP). In addition, we support both labeled and detected modes. The default mode loads detected images. Enable `cuhk03_labeled` in `ImageDataManager` if you wanna train and test on labeled images.

Note

The code will extract images in “cuhk-03.mat” and save them under “cuhk03/images_detected” and “cuhk03/images_labeled”. Also, four json files will be automatically generated, i.e. “splits_classic_detected.json”, “splits_classic_labeled.json”, “splits_new_detected.json” and “splits_new_labeled.json”. If the parent path of `$REID` is changed, these json files should be manually deleted. The code can automatically generate new json files to match the new path.

### DukeMTMC-reID \(^\dagger\) (`dukemtmcreid`)

- Create a directory called “dukemtmc-reid” under `$REID`.
- Download “DukeMTMC-reID” from http://vision.cs.duke.edu/DukeMTMC/ and extract it under “dukemtmc-reid”.
- The data structure should look like

    dukemtmc-reid/
        DukeMTMC-reID/
            query/
            bounding_box_train/
            bounding_box_test/
            ...

### MSMT17 (`msmt17`)

- Create a directory called “msmt17” under `$REID`.
- Download the dataset from http://www.pkuvmc.com/publications/msmt17.html to “msmt17” and extract the files.
- The data structure should look like

    msmt17/
        MSMT17_V1/ # or MSMT17_V2
            train/
            test/
            list_train.txt
            list_query.txt
            list_gallery.txt
            list_val.txt

### VIPeR \(^\dagger\) (`viper`)

- The download link is http://users.soe.ucsc.edu/~manduchi/VIPeR.v1.0.zip.
- Organize the dataset in a folder named “viper” as follows

    viper/
        VIPeR/
            cam_a/
            cam_b/

### GRID \(^\dagger\) (`grid`)

- The download link is http://personal.ie.cuhk.edu.hk/~ccloy/files/datasets/underground_reid.zip.
- Organize the dataset in a folder named “grid” as follows

    grid/
        underground_reid/
            probe/
            gallery/
            ...

### CUHK01 (`cuhk01`)

- Create a folder named “cuhk01” under `$REID`.
- Download “CUHK01.zip” from http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html and place it under “cuhk01/”.
- The code can automatically extract the files, or you can do it yourself.
- The data structure should look like

    cuhk01/
        campus/

### SenseReID (`sensereid`)

- Create “sensereid” under `$REID`.
- Download the dataset from this link and extract it to “sensereid”.
- Organize the data to be like

    sensereid/
        SenseReID/
            test_probe/
            test_gallery/

### QMUL-iLIDS \(^\dagger\) (`ilids`)

- Create a folder named “ilids” under `$REID`.
- Download the dataset from http://www.eecs.qmul.ac.uk/~jason/data/i-LIDS_Pedestrian.tgz and organize it to look like

    ilids/
        i-LIDS_Pedestrian/
            Persons/

### PRID (`prid`)

- Create a directory named “prid2011” under `$REID`.
- Download the dataset from https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/ and extract it under “prid2011”.
- The data structure should end up with

    prid2011/
        prid_2011/
            single_shot/
            multi_shot/

## Video Datasets

### MARS (`mars`)

- Create “mars/” under `$REID`.
- Download the dataset from http://www.liangzheng.com.cn/Project/project_mars.html and place it in “mars/”.
- Extract “bbox_train.zip” and “bbox_test.zip”.
- Download the split metadata from https://github.com/liangzheng06/MARS-evaluation/tree/master/info and put “info/” in “mars/”.
- The data structure should end up with

    mars/
        bbox_test/
        bbox_train/
        info/

### iLIDS-VID \(^\dagger\) (`ilidsvid`)

- Create “ilids-vid” under `$REID`.
- Download the dataset from http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html to “ilids-vid”.
- Organize the data structure to match

    ilids-vid/
        i-LIDS-VID/
        train-test people splits/

### PRID2011 (`prid2011`)

- Create a directory named “prid2011” under `$REID`.
- Download the dataset from https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/ and extract it under “prid2011”.
- Download the split created by iLIDS-VID from here <http://www.eecs.qmul.ac.uk/~kz303/deep-person-reid/datasets/prid2011/splits_prid2011.json> and put it under “prid2011/”. Following the standard protocol, only 178 persons whose sequences are more than a threshold are used.
- The data structure should end up with

    prid2011/
        splits_prid2011.json
        prid_2011/
            single_shot/
            multi_shot/

### DukeMTMC-VideoReID \(^\dagger\) (`dukemtmcvidreid`)

- Create “dukemtmc-vidreid” under `$REID`.
- Download “DukeMTMC-VideoReID” from http://vision.cs.duke.edu/DukeMTMC/ and unzip the file to “dukemtmc-vidreid/”.
- The data structure should look like

    dukemtmc-vidreid/
        DukeMTMC-VideoReID/
            train/
            query/
            gallery/