<h1>Datasets<aclass="headerlink"href="#datasets"title="Permalink to this headline">¶</a></h1>
<p>Here we provide a guide on how to prepare datasets.</p>
<p>Suppose you wanna store the reid data in a directory called “path/to/reid-data/”, you need to specify the <codeclass="docutils literal notranslate"><spanclass="pre">root</span></code> as <em>root=’path/to/reid-data/’</em> when initializing <codeclass="docutils literal notranslate"><spanclass="pre">DataManager</span></code>. Below we use <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code> to denote “path/to/reid-data”.</p>
<p>Please refer to <aclass="reference internal"href="pkg/data.html#torchreid-data"><spanclass="std std-ref">torchreid.data</span></a> for details regarding the arguments.</p>
<divclass="admonition note">
<pclass="first admonition-title">Note</p>
<pclass="last">Dataset with a <spanclass="math notranslate nohighlight">\(\dagger\)</span> superscript means that the process is automated, so you can directly call the dataset in <codeclass="docutils literal notranslate"><spanclass="pre">DataManager</span></code> (which automatically downloads the dataset and organizes the data structure). However, we also provide a way below to help the manual setup in case the automation fails.</p>
</div>
<divclass="admonition note">
<pclass="first admonition-title">Note</p>
<pclass="last">The keys to use specific datasets are enclosed in the parantheses beside the datasets.</p>
</div>
<divclass="admonition note">
<pclass="first admonition-title">Note</p>
<pclass="last">You are suggested to use the provided names for dataset folders such as “market1501” for Market1501 and “dukemtmcreid” for DukeMTMC-reID when doing the manual setup, otherwise you need to modify the source code accordingly (i.e. the <codeclass="docutils literal notranslate"><spanclass="pre">dataset_dir</span></code> attribute).</p>
<h3><aclass="toc-backref"href="#id2">Market1501 <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">market1501</span></code>)</a><aclass="headerlink"href="#market1501-dagger-market1501"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a directory named “market1501” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset to “market1501” from <aclass="reference external"href="http://www.liangzheng.org/Project/project_reid.html">http://www.liangzheng.org/Project/project_reid.html</a> and extract the files.</li>
<li>To use the extra 500K distractors (i.e. Market1501 + 500K), go to the <strong>Market-1501+500k Dataset</strong> section at <aclass="reference external"href="http://www.liangzheng.org/Project/project_reid.html">http://www.liangzheng.org/Project/project_reid.html</a>, download the zip file “distractors_500k.zip” and extract it under “market1501/Market-1501-v15.09.15”. The argument to use these 500K distrctors is <codeclass="docutils literal notranslate"><spanclass="pre">market1501_500k</span></code> in <codeclass="docutils literal notranslate"><spanclass="pre">ImageDataManager</span></code>.</li>
</ul>
</div>
<divclass="section"id="cuhk03-cuhk03">
<h3><aclass="toc-backref"href="#id3">CUHK03 (<codeclass="docutils literal notranslate"><spanclass="pre">cuhk03</span></code>)</a><aclass="headerlink"href="#cuhk03-cuhk03"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a folder named “cuhk03” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset to “cuhk03/” from <aclass="reference external"href="http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html">http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html</a> and extract “cuhk03_release.zip”, resulting in “cuhk03/cuhk03_release/”.</li>
<li>Download the new split (767/700) from <aclass="reference external"href="https://github.com/zhunzhong07/person-re-ranking/tree/master/evaluation/data/CUHK03">person-re-ranking</a>. What you need are “cuhk03_new_protocol_config_detected.mat” and “cuhk03_new_protocol_config_labeled.mat”. Put these two mat files under “cuhk03/”.</li>
<li>In the default mode, we load data using the new split (767/700). If you wanna use the original (20) splits (1367/100), please set <codeclass="docutils literal notranslate"><spanclass="pre">cuhk03_classic_split</span></code> to True in <codeclass="docutils literal notranslate"><spanclass="pre">ImageDataManager</span></code>. As the CMC is computed differently from Market1501 for the 1367/100 split (see <aclass="reference external"href="http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html">here</a>), you need to enable <codeclass="docutils literal notranslate"><spanclass="pre">use_metric_cuhk03</span></code> in <codeclass="docutils literal notranslate"><spanclass="pre">ImageDataManager</span></code> to activate the <em>single-gallery-shot</em> metric for fair comparison with some methods that adopt the old splits (<em>do not need to report mAP</em>). In addition, we support both <em>labeled</em> and <em>detected</em> modes. The default mode loads <em>detected</em> images. Enable <codeclass="docutils literal notranslate"><spanclass="pre">cuhk03_labeled</span></code> in <codeclass="docutils literal notranslate"><spanclass="pre">ImageDataManager</span></code> if you wanna train and test on <em>labeled</em> images.</li>
</ul>
<divclass="admonition note">
<pclass="first admonition-title">Note</p>
<pclass="last">The code will extract images in “cuhk-03.mat” and save them under “cuhk03/images_detected” and “cuhk03/images_labeled”. Also, four json files will be automatically generated, i.e. “splits_classic_detected.json”, “splits_classic_labeled.json”, “splits_new_detected.json” and “splits_new_labeled.json”. If the parent path of <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code> is changed, these json files should be manually deleted. The code can automatically generate new json files to match the new path.</p>
<h3><aclass="toc-backref"href="#id4">DukeMTMC-reID <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">dukemtmcreid</span></code>)</a><aclass="headerlink"href="#dukemtmc-reid-dagger-dukemtmcreid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a directory called “dukemtmc-reid” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download “DukeMTMC-reID” from <aclass="reference external"href="http://vision.cs.duke.edu/DukeMTMC/">http://vision.cs.duke.edu/DukeMTMC/</a> and extract it under “dukemtmc-reid”.</li>
<h3><aclass="toc-backref"href="#id5">MSMT17 (<codeclass="docutils literal notranslate"><spanclass="pre">msmt17</span></code>)</a><aclass="headerlink"href="#msmt17-msmt17"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a directory called “msmt17” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="http://www.pkuvmc.com/publications/msmt17.html">http://www.pkuvmc.com/publications/msmt17.html</a> to “msmt17” and extract the files.</li>
<h3><aclass="toc-backref"href="#id6">VIPeR <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">viper</span></code>)</a><aclass="headerlink"href="#viper-dagger-viper"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>The download link is <aclass="reference external"href="http://users.soe.ucsc.edu/~manduchi/VIPeR.v1.0.zip">http://users.soe.ucsc.edu/~manduchi/VIPeR.v1.0.zip</a>.</li>
<li>Organize the dataset in a folder named “viper” as follows</li>
<h3><aclass="toc-backref"href="#id7">GRID <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">grid</span></code>)</a><aclass="headerlink"href="#grid-dagger-grid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>The download link is <aclass="reference external"href="http://personal.ie.cuhk.edu.hk/~ccloy/files/datasets/underground_reid.zip">http://personal.ie.cuhk.edu.hk/~ccloy/files/datasets/underground_reid.zip</a>.</li>
<li>Organize the dataset in a folder named “grid” as follows</li>
<h3><aclass="toc-backref"href="#id8">CUHK01 (<codeclass="docutils literal notranslate"><spanclass="pre">cuhk01</span></code>)</a><aclass="headerlink"href="#cuhk01-cuhk01"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a folder named “cuhk01” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download “CUHK01.zip” from <aclass="reference external"href="http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html">http://www.ee.cuhk.edu.hk/~xgwang/CUHK_identification.html</a> and place it under “cuhk01/”.</li>
<li>The code can automatically extract the files, or you can do it yourself.</li>
<h3><aclass="toc-backref"href="#id9">PRID450S <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">prid450s</span></code>)</a><aclass="headerlink"href="#prid450s-dagger-prid450s"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a folder named “prid450s” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/prid450s">https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/prid450s</a> and organize the data structure as follows</li>
<h3><aclass="toc-backref"href="#id10">SenseReID (<codeclass="docutils literal notranslate"><spanclass="pre">sensereid</span></code>)</a><aclass="headerlink"href="#sensereid-sensereid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create “sensereid” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from this <aclass="reference external"href="https://drive.google.com/file/d/0B56OfSrVI8hubVJLTzkwV2VaOWM/view">link</a> and extract it to “sensereid”.</li>
<h3><aclass="toc-backref"href="#id11">QMUL-iLIDS <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">ilids</span></code>)</a><aclass="headerlink"href="#qmul-ilids-dagger-ilids"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a folder named “ilids” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="http://personal.ie.cuhk.edu.hk/~ccloy/files/datasets/underground_reid.zip">http://personal.ie.cuhk.edu.hk/~ccloy/files/datasets/underground_reid.zip</a> and organize it to look like</li>
<h3><aclass="toc-backref"href="#id12">PRID (<codeclass="docutils literal notranslate"><spanclass="pre">prid</span></code>)</a><aclass="headerlink"href="#prid-prid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a directory named “prid2011” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/">https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/</a> and extract it under “prid2011”.</li>
<h2><aclass="toc-backref"href="#id13">Video Datasets</a><aclass="headerlink"href="#video-datasets"title="Permalink to this headline">¶</a></h2>
<divclass="section"id="mars-mars">
<h3><aclass="toc-backref"href="#id14">MARS (<codeclass="docutils literal notranslate"><spanclass="pre">mars</span></code>)</a><aclass="headerlink"href="#mars-mars"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create “mars/” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="http://www.liangzheng.com.cn/Project/project_mars.html">http://www.liangzheng.com.cn/Project/project_mars.html</a> and place it in “mars/”.</li>
<li>Extract “bbox_train.zip” and “bbox_test.zip”.</li>
<li>Download the split metadata from <aclass="reference external"href="https://github.com/liangzheng06/MARS-evaluation/tree/master/info">https://github.com/liangzheng06/MARS-evaluation/tree/master/info</a> and put “info/” in “mars/”.</li>
<h3><aclass="toc-backref"href="#id15">iLIDS-VID <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">ilidsvid</span></code>)</a><aclass="headerlink"href="#ilids-vid-dagger-ilidsvid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create “ilids-vid” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html">http://www.eecs.qmul.ac.uk/~xiatian/downloads_qmul_iLIDS-VID_ReID_dataset.html</a> to “ilids-vid”.</li>
<h3><aclass="toc-backref"href="#id16">PRID2011 (<codeclass="docutils literal notranslate"><spanclass="pre">prid2011</span></code>)</a><aclass="headerlink"href="#prid2011-prid2011"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create a directory named “prid2011” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download the dataset from <aclass="reference external"href="https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/">https://www.tugraz.at/institute/icg/research/team-bischof/lrs/downloads/PRID11/</a> and extract it under “prid2011”.</li>
<li>Download the split created by <em>iLIDS-VID</em> from <cite>here <http://www.eecs.qmul.ac.uk/~kz303/deep-person-reid/datasets/prid2011/splits_prid2011.json></cite> and put it under “prid2011/”. Following the standard protocol, only 178 persons whose sequences are more than a threshold are used.</li>
<h3><aclass="toc-backref"href="#id17">DukeMTMC-VideoReID <spanclass="math notranslate nohighlight">\(^\dagger\)</span> (<codeclass="docutils literal notranslate"><spanclass="pre">dukemtmcvidreid</span></code>)</a><aclass="headerlink"href="#dukemtmc-videoreid-dagger-dukemtmcvidreid"title="Permalink to this headline">¶</a></h3>
<ulclass="simple">
<li>Create “dukemtmc-vidreid” under <codeclass="docutils literal notranslate"><spanclass="pre">$REID</span></code>.</li>
<li>Download “DukeMTMC-VideoReID” from <aclass="reference external"href="http://vision.cs.duke.edu/DukeMTMC/">http://vision.cs.duke.edu/DukeMTMC/</a> and unzip the file to “dukemtmc-vidreid/”.</li>
Built with <ahref="http://sphinx-doc.org/">Sphinx</a> using a <ahref="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <ahref="https://readthedocs.org">Read the Docs</a>.