mmselfsup/docs/MODEL_ZOO.md

Model Zoo

Pre-trained model download links and speed test.

Note

• The testing GPUs are NVIDIA Tesla V100.
• Experiments with the same batch size are directly comparable in speed.

Method	Config	Remarks	Download link	Batch size	Epochs	Time per epoch
ImageNet	-	torchvision	imagenet_r50-21352794.pth	-	-	-
Random	-	kaiming	random_r50-5d0fa71b.pth	-	-	-
Relative-Loc	selfsup/relative_loc/r50.py	default	relative_loc_r50-342c9097.pth	512	70	21min17s
Rotation-Pred	selfsup/rotation_pred/r50.py	default	rotation_r50-cfab8ebb.pth	128	70	49min58s
DeepCluster	selfsup/deepcluster/r50.py	default	deepcluster_r50-bb8681e2.pth	512	200	41min57s
NPID	selfsup/npid/r50.py	default	npid_r50-dec3df0c.pth	256	200	20min5s
	selfsup/npid/r50_ensure_neg.py	ensure_neg=True	npid_r50_ensure_neg-ce09b7ae.pth
ODC	selfsup/odc/r50_v1.py	default	odc_r50_v1-5af5dd0c.pth	512	440	28min22s
MoCo	selfsup/moco/r50_v1.py	default	moco_r50_v1-4ad89b5c.pth	256	200	22min58s
MoCo v2	selfsup/moco/r50_v2.py	default	moco_r50_v2-e3b0c442.pth	256	200	55min43s
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	simclr_r50_bs256_ep200-4577e9a6.pth	256	200	1h1min7s
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	simclr_r50_bs256_ep200_mocov2_neck-0d6e5ff2.pth
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	byol_r50-e3b0c442.pth	4096	200	14min40s

Benchmarks

VOC07 SVM & SVM Low-shot

Method	Config	Remarks	Best layer	VOC07 SVM	VOC07 SVM Low-shot
					1	2	4	8	16	32	64	96
ImageNet	-	torchvision	feat5	87.17	52.99	63.55	73.7	78.79	81.76	83.75	85.18	85.97
Random	-	kaiming	feat2	30.54	9.15	9.39	11.09	12.3	14.3	17.41	21.32	23.77
Relative-Loc	selfsup/relative_loc/r50.py	default	feat4	64.78	18.17	22.08	29.37	35.58	41.8	48.73	55.55	58.33
Rotation-Pred	selfsup/rotation_pred/r50.py	default	feat4	67.38	18.91	23.33	30.57	38.22	45.83	52.23	58.08	61.11
DeepCluster	selfsup/deepcluster/r50.py	default	feat5	74.26	29.73	37.66	45.85	55.57	62.48	66.15	70.0	71.37
NPID	selfsup/npid/r50.py	default	feat5	74.50	24.19	31.24	39.69	50.99	59.03	64.4	68.69	70.84
	selfsup/npid/r50_ensure_neg.py	ensure_neg=True	feat5	75.70
ODC	selfsup/odc/r50_v1.py	default	feat5	78.42	32.42	40.27	49.95	59.96	65.71	69.99	73.64	75.13
MoCo	selfsup/moco/r50_v1.py	default	feat5	79.18	30.03	37.73	47.64	58.78	66.0	70.6	74.6	76.07
MoCo v2	selfsup/moco/r50_v2.py	default	feat5	84.26	-	-	-	-	-	-	-	-
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	feat5	78.95	32.45	40.76	50.4	59.01	65.45	70.13	73.58	75.35
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	feat5	77.65
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	feat5	85.10	44.48	52.09	62.88	70.87	76.18	79.45	81.88	83.08

ImageNet Linear Classification

Note

• Config: configs/benchmarks/linear_classification/imagenet/r50_multihead.py for ImageNet (Multi) and configs/benchmarks/linear_classification/imagenet/r50_last.py for ImageNet (Last).
• For DeepCluster, use the corresponding one with _sobel.
• ImageNet (Multi) evaluates features in around 9k dimensions from different layers. Top-1 result of the last epoch is reported.
• ImageNet (Last) evaluates the last feature after global average pooling, e.g., 2048 dimensions for resnet50. The best top-1 result among all epochs is reported.

Method	Config	Remarks	ImageNet (Multi)					ImageNet (Last)
			feat1	feat2	feat3	feat4	feat5	avgpool
ImageNet	-	torchvision	15.18	33.96	47.86	67.56	76.17	74.12
Random	-	kaiming	11.37	16.21	13.47	9.07	6.54	4.35
Relative-Loc	selfsup/relative_loc/r50.py	default	14.76	31.29	45.77	49.31	40.20	38.83
Rotation-Pred	selfsup/rotation_pred/r50.py	default	12.89	34.30	44.91	54.99	49.09	47.01
DeepCluster	selfsup/deepcluster/r50.py	default	12.78	30.81	43.88	57.71	51.68	46.92
NPID	selfsup/npid/r50.py	default	14.28	31.20	40.68	54.46	56.61	56.60
ODC	selfsup/odc/r50_v1.py	default	14.76	31.82	42.44	55.76	57.70	53.42
MoCo	selfsup/moco/r50_v1.py	default	15.32	33.08	44.68	57.27	60.60	61.02
MoCo v2	selfsup/moco/r50_v2.py	default	14.74	32.81	44.95	61.61	66.73	67.69
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	17.09	31.37	41.38	54.35	61.57	60.06
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	16.97	31.88	41.73	54.33	59.94	58.00
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	16.70	34.22	46.61	60.78	69.14	67.10

Places205 Linear Classification

Note

• Config: configs/benchmarks/linear_classification/places205/r50_multihead.py.
• For DeepCluster, use the corresponding one with _sobel.
• Places205 evaluates features in around 9k dimensions from different layers. Top-1 result of the last epoch is reported.

Method	Config	Remarks	Places205
			feat1	feat2	feat3	feat4	feat5
ImageNet	-	torchvision	21.27	36.10	43.03	51.38	53.05
Random	-	kaiming	17.19	21.70	19.23	14.59	11.73
Relative-Loc	selfsup/relative_loc/r50.py	default	21.07	34.86	42.84	45.71	41.45
Rotation-Pred	selfsup/rotation_pred/r50.py	default	18.65	35.71	42.28	45.98	43.72
DeepCluster	selfsup/deepcluster/r50.py	default	18.80	33.93	41.44	47.22	42.61
NPID	selfsup/npid/r50.py	default	20.53	34.03	40.48	47.13	47.73
ODC	selfsup/odc/r50_v1.py	default	20.94	34.78	41.19	47.45	49.18
MoCo	selfsup/moco/r50_v1.py	default	21.13	35.19	42.40	48.78	50.70
MoCo v2	selfsup/moco/r50_v2.py	default	20.86	35.63	42.57	49.93	52.05
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	22.55	34.14	40.35	47.15	51.64
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	22.28	35.95	43.03	49.79	52.75

ImageNet Semi-Supervised Classification

Note

• In this benchmark, the necks or heads are removed and only the backbone CNN is evaluated by appending a linear classification head. All parameters are fine-tuned.
• Config: under configs/benchmarks/semi_classification/imagenet_1percent/ for 1% data, and configs/benchmarks/semi_classification/imagenet_10percent/ for 10% data.
• When training with 1% ImageNet, we find hyper-parameters especially the learning rate greatly influence the performance. Hence, we prepare a list of settings with the base learning rate from {0.001, 0.01, 0.1} and the learning rate multiplier for the head from {1, 10, 100}. We choose the best performing setting for each method.
• Please use --deterministic in this benchmark.

Method	Config	Remarks	Optimal setting for ImageNet 1%	ImageNet 1%
				top-1	top-5
ImageNet	-	torchvision	r50_lr0_001_head100.py	68.68	88.87
Random	-	kaiming	r50_lr0_01_head1.py	1.56	4.99
Relative-Loc	selfsup/relative_loc/r50.py	default	r50_lr0_01_head100.py	16.48	40.37
Rotation-Pred	selfsup/rotation_pred/r50.py	default	r50_lr0_01_head100.py	18.98	44.05
DeepCluster	selfsup/deepcluster/r50.py	default	r50_lr0_01_head1_sobel.py	33.44	58.62
NPID	selfsup/npid/r50.py	default	r50_lr0_01_head100.py	27.95	54.37
ODC	selfsup/odc/r50_v1.py	default	r50_lr0_1_head100.py	32.39	61.02
MoCo	selfsup/moco/r50_v1.py	default	r50_lr0_01_head100.py	33.15	61.30
MoCo v2	selfsup/moco/r50_v2.py	default	r50_lr0_01_head100.py	38.71	67.90
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	r50_lr0_01_head100.py	36.09	64.50
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	r50_lr0_01_head100.py	36.31	64.68
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	r50_lr0_01_head10.py	49.37	76.75

Method	Config	Remarks	Optimal setting for ImageNet 10%	ImageNet 10%
				top-1	top-5
ImageNet	-	torchvision	r50_lr0_001_head10.py	74.53	92.19
Random	-	kaiming	r50_lr0_01_head1.py	21.78	44.24
Relative-Loc	selfsup/relative_loc/r50.py	default	r50_lr0_01_head100.py	53.86	79.62
Rotation-Pred	selfsup/rotation_pred/r50.py	default	r50_lr0_01_head100.py	54.75	80.21
DeepCluster	selfsup/deepcluster/r50.py	default	r50_lr0_01_head1_sobel.py	52.94	77.96
NPID	selfsup/npid/r50.py	default	r50_lr0_01_head100.py	57.22	81.39
ODC	selfsup/odc/r50_v1.py	default	r50_lr0_1_head10.py	58.15	82.55
MoCo	selfsup/moco/r50_v1.py	default	r50_lr0_01_head100.py	60.08	84.02
MoCo v2	selfsup/moco/r50_v2.py	default	r50_lr0_01_head100.py	61.64	84.90
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	r50_lr0_01_head100.py	58.46	82.60
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck		58.38	82.53
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	r50_lr0_01_head100.py	65.94	87.81

PASCAL VOC07+12 Object Detection

Note

• This benchmark follows the evluation protocols set up by MoCo.
• Config: benchmarks/detection/configs/pascal_voc_R_50_C4_24k_moco.yaml.
• Please follow here to run the evaluation.

Method	Config	Remarks	VOC07+12
			AP50	AP	AP75
ImageNet	-	torchvision	81.58	54.19	59.80
Random	-	kaiming	59.02	32.83	31.60
Relative-Loc	selfsup/relative_loc/r50.py	default	80.36	55.13	61.18
Rotation-Pred	selfsup/rotation_pred/r50.py	default	80.91	55.52	61.39
NPID	selfsup/npid/r50.py	default	80.03	54.11	59.50
MoCo	selfsup/moco/r50_v1.py	default	81.38	55.95	62.23
MoCo v2	selfsup/moco/r50_v2.py	default	81.96	56.63	62.90
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	79.41	51.54	55.63
BYOL	selfsup/byol/r50_bs4096_ep200.py	default	80.95	51.87	56.53

COCO2017 Object Detection

Note

• This benchmark follows the evluation protocols set up by MoCo.
• Config: benchmarks/detection/configs/coco_R_50_C4_2x_moco.yaml.
• Please follow here to run the evaluation.

Method	Config	Remarks	COCO2017
			AP50(Box)	AP(Box)	AP75(Box)	AP50(Mask)	AP(Mask)	AP75(Mask)
ImageNet	-	torchvision	59.9	40.0	43.1	56.5	34.7	36.9
Random	-	kaiming	54.6	35.6	38.2	51.5	31.4	33.5
Relative-Loc	selfsup/relative_loc/r50.py	default	59.6	40.0	43.5	56.5	35.0	37.3
Rotation-Pred	selfsup/rotation_pred/r50.py	default	59.3	40.0	43.6	56.0	34.9	37.4
NPID	selfsup/npid/r50.py	default	59.0	39.4	42.8	55.9	34.5	36.6
MoCo	selfsup/moco/r50_v1.py	default	60.5	40.9	44.2	57.1	35.5	37.7
MoCo v2	selfsup/moco/r50_v2.py	default	60.7	40.9	44.2	57.2	35.5	37.9
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	59.1	39.6	42.9	55.9	34.6	37.1
BYOL	selfsup/byol/r50.py	default	60.5	40.3	43.9	56.8	35.1	37.3