Model Zoo

Pre-trained model download links

Method	Config	Remarks	Download link
ImageNet	-	torchvision	imagenet_r50-21352794.pth
Random	-	kaiming	random_r50-5d0fa71b.pth
Relative-Loc	selfsup/relative_loc/r50.py	default
Rotation-Pred	selfsup/rotation_pred/r50.py	default	rotation_r50-cfab8ebb.pth
DeepCluster	selfsup/deepcluster/r50.py	default	deepcluster_r50-bb8681e2.pth
NPID	selfsup/npid/r50.py	default	npid_r50-dec3df0c.pth
ODC	selfsup/odc/r50_v1.py	default	odc_r50_v1-5af5dd0c.pth
MoCo	selfsup/moco/r50_v1.py	default	moco_r50_v1-4ad89b5c.pth
MoCo v2	selfsup/moco/r50_v2.py	default	moco_r50_v2-58f10cfe.pth
	selfsup/moco/r50_v2.py	-> SimCLR neck	moco_r50_v2_simclr_neck-70379356.pth
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	simclr_r50_bs256_ep200-4577e9a6.pth
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	simclr_r50_bs256_ep200_mocov2_neck-0d6e5ff2.pth
BYOL	selfsup/byol/r50.py	default

Method	Config	Remarks	Best layer	VOC07 SVM	ImageNet (Multi)
Method	Config	Remarks	Best layer	VOC07 SVM	1	2	4	8	16	32	64	96
ImageNet	-	torchvision	feat5	87.17	52.99	63.55	73.7	78.79	81.76	83.75	85.18	85.97
Random	-	kaiming	feat2	30.22
Relative-Loc			feat5
Rotation-Pred	selfsup/rotation_pred/r50.py	default	feat4	67.38
DeepCluster	selfsup/deepcluster/r50.py	default	feat5	74.26
NPID	selfsup/npid/r50.py	default	feat5	74.50
ODC	selfsup/odc/r50_v1.py	default	feat5	78.42
MoCo	selfsup/moco/r50_v1.py	default	feat5	79.18
MoCo v2	selfsup/moco/r50_v2.py	default	feat5	84.05
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck	feat5	84.00
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	feat5	78.95
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	feat5	77.65
BYOL	selfsup/byol/r50.py	default

Method	Config	Remarks	ImageNet (Multi)					ImageNet (Last)
Method	Config	Remarks	feat1	feat2	feat3	feat4	feat5	avgpool
ImageNet	-	torchvision	15.18	33.96	47.86	67.56	76.17	74.12
Random	-	kaiming						4.35
Relative-Loc	selfsup/relative_loc/r50.py	default
Rotation-Pred	selfsup/rotation_pred/r50.py	default	12.89	34.30	44.91	54.99	49.09
DeepCluster	selfsup/deepcluster/r50.py	default						46.92
NPID	selfsup/npid/r50.py	default	14.28	31.20	40.68	54.46	56.61
ODC	selfsup/odc/r50_v1.py	default	14.75	31.55	42.49	55.72	57.57
MoCo	selfsup/moco/r50_v1.py	default	15.32	33.08	44.68	57.27	60.60	61.02
MoCo v2	selfsup/moco/r50_v2.py	default	15.35	34.57	45.81	60.96	66.72	67.02
	selfsup/moco/r50_v2.py	-> SimCLR neck
SimCLR	selfsup/simclr/r50_bs256.py	default
BYOL	selfsup/byol/r50.py	default

Note

In this benchmark, the necks or heads are removed and only the backbone CNN is evaluated by appending a linear classification head. All parameters are fine-tuned.
When training with 1% ImageNet, we find hyper-parameters especially the learning rate greatly influence the performance. Hence, we prepare a list of settings with the base learning rate from {0.01, 0.1} and the learning rate multiplier for the head from {1, 10, 100}. We choose the best performing setting for each method.
Please use --deterministic in this benchmark.

Method	Config	Remarks	Optimal setting for ImageNet 1%	ImageNet 1%
Method	Config	Remarks	Optimal setting for ImageNet 1%	top-1	top-5
ImageNet	-	torchvision	r50_lr0_01_head1.py	63.10	85.73
Random	-	kaiming	r50_lr0_01_head1.py	1.56	4.99
Relative-Loc	selfsup/relative_loc/r50.py	default
Rotation-Pred	selfsup/rotation_pred/r50.py	default	r50_lr0_01_head100.py	18.98	44.05
DeepCluster	selfsup/deepcluster/r50.py	default	r50_lr0_01_head1_sobel.py	33.44	58.62
NPID	selfsup/npid/r50.py	default	r50_lr0_01_head100.py	27.95	54.37
ODC	selfsup/odc/r50_v1.py	default	r50_lr0_1_head100.py	32.39	61.02
MoCo	selfsup/moco/r50_v1.py	default	r50_lr0_01_head100.py	33.15	61.30
MoCo v2	selfsup/moco/r50_v2.py	default	r50_lr0_01_head100.py	38.71	67.90
	selfsup/moco/r50_v2.py	-> SimCLR neck
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	r50_lr0_01_head100.py	36.09	64.50
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck
BYOL	selfsup/byol/r50.py	default