From ecc23705933563957d9c523d76e10c7de8e2aab8 Mon Sep 17 00:00:00 2001 From: YifanXu Date: Fri, 20 Oct 2023 22:52:39 +0800 Subject: [PATCH] Add instruction on customized pretraining --- CUSTOMIZED_PRETRAIN.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 CUSTOMIZED_PRETRAIN.md diff --git a/CUSTOMIZED_PRETRAIN.md b/CUSTOMIZED_PRETRAIN.md new file mode 100644 index 0000000..70d1a01 --- /dev/null +++ b/CUSTOMIZED_PRETRAIN.md @@ -0,0 +1,40 @@ +MQ-Det supports modulated training on any datasets in COCO format. Let's take COCO for example. + +To conduct customized modulating, you can follow these steps. + + +**1. Add customized dataset infomation in two places of the [code](maskrcnn_benchmark/config/paths_catalog.py)** + +1. L120: [Add DatasetCatalog](https://github.com/YifanXu74/MQ-Det/blob/bbacce45f8223d136ceb2be13dd18208cdc9b3db/maskrcnn_benchmark/config/paths_catalog.py#L120) + +2. L394: [Add to factory](https://github.com/YifanXu74/MQ-Det/blob/bbacce45f8223d136ceb2be13dd18208cdc9b3db/maskrcnn_benchmark/config/paths_catalog.py#L394) + +Here we add a new dataset ``coco_grounding_train_for_obj365``. + + +**2. Acquire customized config files** + +You can modified upon the [official pretraining config file](configs/pretrain/mq-glip-t.yaml) to get a customized config file. Here we provide an [example](configs/pretrain/mq-glip-t_coco.yaml). You customize your own needs following the "``NOTE``" in the file. + +Make sure to use correct ``DATASETS.TRAIN`` and ``VISION_QUERY.QUERY_BANK_PATH``. + +Here we use a new config file ``configs/pretrain/mq-glip-t_coco.yaml``. + + +**3. Extract vision queries** +``` +python tools/train_net.py \ +--config-file configs/pretrain/mq-glip-t_coco.yaml \ +--extract_query \ +VISION_QUERY.QUERY_BANK_SAVE_PATH MODEL/coco_query_5000_sel_tiny.pth +``` + +Here we can get a new query bank ``MODEL/coco_query_5000_sel_tiny.pth``. Make sure the ``VISION_QUERY.QUERY_BANK_PATH`` in the config file to be this query bank path. + +You can specify ``VISION_QUERY.MAX_QUERY_NUMBER`` (number of queries for each category in the bank) to any number to control the bank size. + + +**4. Conduct modulated pretraining** +``` +python -m torch.distributed.launch --nproc_per_node=8 tools/train_net.py --config-file configs/pretrain/mq-glip-t_coco.yaml --use-tensorboard OUTPUT_DIR 'OUTPUT/MQ-GLIP-TINY-COCO/' +``` \ No newline at end of file