{
    "cells": [
        {
            "cell_type": "markdown",
            "id": "856bf300",
            "metadata": {
                "id": "856bf300"
            },
            "source": [
                "<a href=\"https://colab.research.google.com/github/open-mmlab/mmselfsup/blob/master/demo/mmselfsup_colab_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a2505b44",
            "metadata": {
                "id": "a2505b44"
            },
            "source": [
                "# MMSelfSup Tutorial\n",
                "In this tutorial, we will introduce the following content:\n",
                "\n",
                "- How to install MMSelfSup\n",
                "- How to train algorithms in MMSelfSup\n",
                "- How to train downstream tasks\n",
                "\n",
                "If you have any other questions, welcome to report issues."
            ]
        },
        {
            "cell_type": "markdown",
            "id": "2a78b9a6",
            "metadata": {
                "id": "2a78b9a6"
            },
            "source": [
                "## How to install MMSelfSup\n",
                "\n",
                "Before using MMSelfSup, we need to prepare the environment with the following steps:\n",
                "\n",
                "1. Install Python, CUDA, C/C++ compiler and git\n",
                "2. Install PyTorch (CUDA version)\n",
                "3. Install dependent codebase (mmengine, mmcv, mmcls)\n",
                "4. Clone mmselfsup source code from GitHub and install it\n",
                "\n",
                "Because this tutorial is on Google Colab and all necessary packages have been installed, we can skip the first two steps."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "id": "4edc9682",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 18,
                    "status": "ok",
                    "timestamp": 1662048513424,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "4edc9682",
                "outputId": "b9f96ef5-96dc-4a26-850c-b42d80586e38"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "nvcc: NVIDIA (R) Cuda compiler driver\n",
                        "Copyright (c) 2005-2020 NVIDIA Corporation\n",
                        "Built on Mon_Oct_12_20:09:46_PDT_2020\n",
                        "Cuda compilation tools, release 11.1, V11.1.105\n",
                        "Build cuda_11.1.TC455_06.29190527_0\n"
                    ]
                }
            ],
            "source": [
                "# Check nvcc version\n",
                "!nvcc -V"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 2,
            "id": "f6c86477",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 11,
                    "status": "ok",
                    "timestamp": 1662048513425,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "f6c86477",
                "outputId": "4e73eded-7146-44a9-c90b-b4fb4f7cb055"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\n",
                        "Copyright (C) 2017 Free Software Foundation, Inc.\n",
                        "This is free software; see the source for copying conditions.  There is NO\n",
                        "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "# Check GCC version\n",
                "!gcc --version"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "id": "4d45e19e",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 2046,
                    "status": "ok",
                    "timestamp": 1662048515465,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "4d45e19e",
                "outputId": "049a158c-9471-4631-a43a-f24180bf55da"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "1.12.1+cu113\n",
                        "True\n"
                    ]
                }
            ],
            "source": [
                "# Check PyTorch installation\n",
                "import torch\n",
                "print(torch.__version__)\n",
                "print(torch.cuda.is_available())"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "d8b2afc9",
            "metadata": {
                "id": "d8b2afc9"
            },
            "source": [
                "### Install MMEngine and MMCV\n",
                "\n",
                "MMCV is the basic package of all OpenMMLab packages. We have pre-built wheels on Linux, so we can download and install them directly.\n",
                "\n",
                "Please pay attention to PyTorch and CUDA versions to match the wheel.\n",
                "\n",
                "In the above steps, we have checked the version of PyTorch and CUDA, and they are 1.10.2 and 11.3 respectively, so we need to choose the corresponding wheel.\n",
                "\n",
                "In addition, we can also install the full version of mmcv (mmcv-full). It includes full features and various CUDA ops out of the box, but needs a longer time to build."
            ]
        },
        {
            "cell_type": "markdown",
            "id": "12c97bbd",
            "metadata": {
                "id": "12c97bbd"
            },
            "source": [
                "MIM is recommended: https://github.com/open-mmlab/mim"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "id": "ac1462fd",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 26271,
                    "status": "ok",
                    "timestamp": 1662048541730,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "ac1462fd",
                "outputId": "9da88069-d3ac-4d0a-959b-9ffe9d60d3e8"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
                        "Collecting openmim\n",
                        "  Downloading openmim-0.3.0-py2.py3-none-any.whl (49 kB)\n",
                        "\u001b[K     |████████████████████████████████| 49 kB 3.1 MB/s \n",
                        "\u001b[?25hRequirement already satisfied: Click in /usr/local/lib/python3.7/dist-packages (from openmim) (7.1.2)\n",
                        "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from openmim) (2.23.0)\n",
                        "Collecting rich\n",
                        "  Downloading rich-12.5.1-py3-none-any.whl (235 kB)\n",
                        "\u001b[K     |████████████████████████████████| 235 kB 9.7 MB/s \n",
                        "\u001b[?25hCollecting model-index\n",
                        "  Downloading model_index-0.1.11-py3-none-any.whl (34 kB)\n",
                        "Requirement already satisfied: pip>=19.3 in /usr/local/lib/python3.7/dist-packages (from openmim) (21.1.3)\n",
                        "Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from openmim) (0.8.10)\n",
                        "Collecting colorama\n",
                        "  Downloading colorama-0.4.5-py2.py3-none-any.whl (16 kB)\n",
                        "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from openmim) (1.3.5)\n",
                        "Collecting ordered-set\n",
                        "  Downloading ordered_set-4.1.0-py3-none-any.whl (7.6 kB)\n",
                        "Requirement already satisfied: markdown in /usr/local/lib/python3.7/dist-packages (from model-index->openmim) (3.4.1)\n",
                        "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from model-index->openmim) (6.0)\n",
                        "Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown->model-index->openmim) (4.12.0)\n",
                        "Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown->model-index->openmim) (4.1.1)\n",
                        "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown->model-index->openmim) (3.8.1)\n",
                        "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (2022.2.1)\n",
                        "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (2.8.2)\n",
                        "Requirement already satisfied: numpy>=1.17.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (1.21.6)\n",
                        "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->openmim) (1.15.0)\n",
                        "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (1.24.3)\n",
                        "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (2022.6.15)\n",
                        "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (2.10)\n",
                        "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (3.0.4)\n",
                        "Collecting commonmark<0.10.0,>=0.9.0\n",
                        "  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)\n",
                        "\u001b[K     |████████████████████████████████| 51 kB 8.0 MB/s \n",
                        "\u001b[?25hRequirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich->openmim) (2.6.1)\n",
                        "Installing collected packages: ordered-set, commonmark, rich, model-index, colorama, openmim\n",
                        "Successfully installed colorama-0.4.5 commonmark-0.9.1 model-index-0.1.11 openmim-0.3.0 ordered-set-4.1.0 rich-12.5.1\n",
                        "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
                        "Requirement already satisfied: openmim in /usr/local/lib/python3.7/dist-packages (0.3.0)\n",
                        "Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from openmim) (1.3.5)\n",
                        "Requirement already satisfied: pip>=19.3 in /usr/local/lib/python3.7/dist-packages (from openmim) (21.1.3)\n",
                        "Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from openmim) (2.23.0)\n",
                        "Requirement already satisfied: colorama in /usr/local/lib/python3.7/dist-packages (from openmim) (0.4.5)\n",
                        "Requirement already satisfied: rich in /usr/local/lib/python3.7/dist-packages (from openmim) (12.5.1)\n",
                        "Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from openmim) (0.8.10)\n",
                        "Requirement already satisfied: Click in /usr/local/lib/python3.7/dist-packages (from openmim) (7.1.2)\n",
                        "Requirement already satisfied: model-index in /usr/local/lib/python3.7/dist-packages (from openmim) (0.1.11)\n",
                        "Requirement already satisfied: ordered-set in /usr/local/lib/python3.7/dist-packages (from model-index->openmim) (4.1.0)\n",
                        "Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from model-index->openmim) (6.0)\n",
                        "Requirement already satisfied: markdown in /usr/local/lib/python3.7/dist-packages (from model-index->openmim) (3.4.1)\n",
                        "Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown->model-index->openmim) (4.12.0)\n",
                        "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown->model-index->openmim) (3.8.1)\n",
                        "Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown->model-index->openmim) (4.1.1)\n",
                        "Requirement already satisfied: numpy>=1.17.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (1.21.6)\n",
                        "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (2022.2.1)\n",
                        "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->openmim) (2.8.2)\n",
                        "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->openmim) (1.15.0)\n",
                        "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (2.10)\n",
                        "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (2022.6.15)\n",
                        "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (3.0.4)\n",
                        "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->openmim) (1.24.3)\n",
                        "Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich->openmim) (2.6.1)\n",
                        "Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from rich->openmim) (0.9.1)\n",
                        "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
                        "Looking in links: https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html\n",
                        "Collecting mmengine==0.1.0\n",
                        "  Downloading mmengine-0.1.0-py3-none-any.whl (280 kB)\n",
                        "\u001b[K     |████████████████████████████████| 280 kB 5.2 MB/s \n",
                        "\u001b[?25hCollecting mmcv>=2.0.0rc1\n",
                        "  Downloading https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/mmcv-2.0.0rc1-cp37-cp37m-manylinux1_x86_64.whl (40.4 MB)\n",
                        "\u001b[K     |████████████████████████████████| 40.4 MB 12.3 MB/s \n",
                        "\u001b[?25hRequirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from mmengine==0.1.0) (6.0)\n",
                        "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from mmengine==0.1.0) (1.21.6)\n",
                        "Collecting yapf\n",
                        "  Downloading yapf-0.32.0-py2.py3-none-any.whl (190 kB)\n",
                        "\u001b[K     |████████████████████████████████| 190 kB 64.2 MB/s \n",
                        "\u001b[?25hCollecting addict\n",
                        "  Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)\n",
                        "Requirement already satisfied: opencv-python>=3 in /usr/local/lib/python3.7/dist-packages (from mmengine==0.1.0) (4.6.0.66)\n",
                        "Requirement already satisfied: termcolor in /usr/local/lib/python3.7/dist-packages (from mmengine==0.1.0) (1.1.0)\n",
                        "Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from mmengine==0.1.0) (3.2.2)\n",
                        "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from mmcv>=2.0.0rc1) (21.3)\n",
                        "Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from mmcv>=2.0.0rc1) (7.1.2)\n",
                        "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmengine==0.1.0) (2.8.2)\n",
                        "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmengine==0.1.0) (0.11.0)\n",
                        "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmengine==0.1.0) (1.4.4)\n",
                        "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmengine==0.1.0) (3.0.9)\n",
                        "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->mmengine==0.1.0) (4.1.1)\n",
                        "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->mmengine==0.1.0) (1.15.0)\n",
                        "Installing collected packages: yapf, addict, mmengine, mmcv\n",
                        "Successfully installed addict-2.4.0 mmcv-2.0.0rc1 mmengine-0.1.0 yapf-0.32.0\n"
                    ]
                }
            ],
            "source": [
                "!pip3 install openmim\n",
                "!pip install -U openmim\n",
                "!mim install 'mmengine' 'mmcv>=2.0.0rc4'"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "id": "VevRvwZdl8U-",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 2455,
                    "status": "ok",
                    "timestamp": 1662048544177,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "VevRvwZdl8U-",
                "outputId": "11e8964d-83b5-4392-84f4-504d59293fe5"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "OrderedDict([('sys.platform', 'linux'), ('Python', '3.7.13 (default, Apr 24 2022, 01:04:09) [GCC 7.5.0]'), ('CUDA available', True), ('numpy_random_seed', 2147483648), ('GPU 0', 'Tesla T4'), ('CUDA_HOME', '/usr/local/cuda'), ('NVCC', 'Cuda compilation tools, release 11.1, V11.1.105'), ('GCC', 'x86_64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0'), ('PyTorch', '1.12.1+cu113'), ('PyTorch compiling details', 'PyTorch built with:\\n  - GCC 9.3\\n  - C++ Version: 201402\\n  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications\\n  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)\\n  - OpenMP 201511 (a.k.a. OpenMP 4.5)\\n  - LAPACK is enabled (usually provided by MKL)\\n  - NNPACK is enabled\\n  - CPU capability usage: AVX2\\n  - CUDA Runtime 11.3\\n  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\\n  - CuDNN 8.3.2  (built against CUDA 11.5)\\n  - Magma 2.5.2\\n  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \\n'), ('TorchVision', '0.13.1+cu113'), ('OpenCV', '4.6.0'), ('MMEngine', '0.1.0')])\n",
                        "2.0.0rc1\n"
                    ]
                }
            ],
            "source": [
                "# check mmengine install\n",
                "!python -c 'from mmengine.utils.dl_utils import collect_env;print(collect_env())'\n",
                "\n",
                "# check mmcv install\n",
                "import mmcv\n",
                "print(mmcv.__version__)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a0fb23e1",
            "metadata": {
                "id": "a0fb23e1"
            },
            "source": [
                "Besides, you can also use pip to install the packages, but you are supposed to check the pytorch and cuda version manually. The example command is provided below, but you need to modify it according to your PyTorch and CUDA version."
            ]
        },
        {
            "cell_type": "markdown",
            "id": "de19e9ee",
            "metadata": {
                "id": "de19e9ee"
            },
            "source": [
                "### Clone and install mmselfsup"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "id": "ee54ef1a",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 6770,
                    "status": "ok",
                    "timestamp": 1662048550930,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "ee54ef1a",
                "outputId": "b91f3fea-6821-4017-e030-5745c2e1ceb4"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "/content\n",
                        "Cloning into 'mmselfsup'...\n",
                        "remote: Enumerating objects: 6421, done.\u001b[K\n",
                        "remote: Counting objects: 100% (279/279), done.\u001b[K\n",
                        "remote: Compressing objects: 100% (192/192), done.\u001b[K\n",
                        "remote: Total 6421 (delta 123), reused 194 (delta 86), pack-reused 6142\u001b[K\n",
                        "Receiving objects: 100% (6421/6421), 2.75 MiB | 12.05 MiB/s, done.\n",
                        "Resolving deltas: 100% (4028/4028), done.\n",
                        "/content/mmselfsup\n",
                        "Branch 'dev-1.x' set up to track remote branch 'dev-1.x' from 'origin'.\n",
                        "Switched to a new branch 'dev-1.x'\n",
                        "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
                        "Obtaining file:///content/mmselfsup\n",
                        "Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (22.1.0)\n",
                        "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (0.16.0)\n",
                        "Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (3.2.2)\n",
                        "Collecting mmcls>=1.0.0rc0\n",
                        "  Downloading mmcls-1.0.0rc0-py2.py3-none-any.whl (557 kB)\n",
                        "\u001b[K     |████████████████████████████████| 557 kB 5.1 MB/s \n",
                        "\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (1.21.6)\n",
                        "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (21.3)\n",
                        "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (1.0.2)\n",
                        "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (1.7.3)\n",
                        "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (1.15.0)\n",
                        "Requirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (2.8.0)\n",
                        "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from mmselfsup==1.0.0rc1) (4.64.0)\n",
                        "Requirement already satisfied: rich in /usr/local/lib/python3.7/dist-packages (from mmcls>=1.0.0rc0->mmselfsup==1.0.0rc1) (12.5.1)\n",
                        "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmselfsup==1.0.0rc1) (0.11.0)\n",
                        "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmselfsup==1.0.0rc1) (1.4.4)\n",
                        "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmselfsup==1.0.0rc1) (3.0.9)\n",
                        "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->mmselfsup==1.0.0rc1) (2.8.2)\n",
                        "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib->mmselfsup==1.0.0rc1) (4.1.1)\n",
                        "Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich->mmcls>=1.0.0rc0->mmselfsup==1.0.0rc1) (2.6.1)\n",
                        "Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from rich->mmcls>=1.0.0rc0->mmselfsup==1.0.0rc1) (0.9.1)\n",
                        "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->mmselfsup==1.0.0rc1) (3.1.0)\n",
                        "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->mmselfsup==1.0.0rc1) (1.1.0)\n",
                        "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (57.4.0)\n",
                        "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (0.37.1)\n",
                        "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (0.6.1)\n",
                        "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (2.23.0)\n",
                        "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (1.0.1)\n",
                        "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (3.4.1)\n",
                        "Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (1.47.0)\n",
                        "Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (3.17.3)\n",
                        "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (1.35.0)\n",
                        "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (1.8.1)\n",
                        "Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (1.2.0)\n",
                        "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard->mmselfsup==1.0.0rc1) (0.4.6)\n",
                        "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->mmselfsup==1.0.0rc1) (4.9)\n",
                        "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->mmselfsup==1.0.0rc1) (0.2.8)\n",
                        "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->mmselfsup==1.0.0rc1) (4.2.4)\n",
                        "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->mmselfsup==1.0.0rc1) (1.3.1)\n",
                        "Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard->mmselfsup==1.0.0rc1) (4.12.0)\n",
                        "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard->mmselfsup==1.0.0rc1) (3.8.1)\n",
                        "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->mmselfsup==1.0.0rc1) (0.4.8)\n",
                        "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->mmselfsup==1.0.0rc1) (2.10)\n",
                        "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->mmselfsup==1.0.0rc1) (3.0.4)\n",
                        "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->mmselfsup==1.0.0rc1) (1.24.3)\n",
                        "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->mmselfsup==1.0.0rc1) (2022.6.15)\n",
                        "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->mmselfsup==1.0.0rc1) (3.2.0)\n",
                        "Installing collected packages: mmcls, mmselfsup\n",
                        "  Running setup.py develop for mmselfsup\n",
                        "Successfully installed mmcls-1.0.0rc0 mmselfsup-1.0.0rc1\n"
                    ]
                }
            ],
            "source": [
                "%cd /content\n",
                "# Clone MMSelfSup repository\n",
                "!git clone https://github.com/open-mmlab/mmselfsup.git\n",
                "%cd mmselfsup/\n",
                "\n",
                "# Install MMSelfSup from source\n",
                "!git checkout dev-1.x\n",
                "!pip install -e . "
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "id": "artAFjMutvBt",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 12,
                    "status": "ok",
                    "timestamp": 1662048550931,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "artAFjMutvBt",
                "outputId": "92d895d2-0270-4baa-e9d1-fae82748e827"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "1.0.0rc0\n"
                    ]
                }
            ],
            "source": [
                "# Check MMClassification installation\n",
                "import mmcls\n",
                "print(mmcls.__version__)"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 8,
            "id": "OpuyBEm9tgyR",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 8,
                    "status": "ok",
                    "timestamp": 1662048550931,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "OpuyBEm9tgyR",
                "outputId": "bd1786fb-1908-456a-efbc-c0732024bb02"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "1.0.0rc1\n"
                    ]
                }
            ],
            "source": [
                "# Check MMSelfSup installation\n",
                "import mmselfsup\n",
                "print(mmselfsup.__version__)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "8cc33efb",
            "metadata": {
                "id": "8cc33efb"
            },
            "source": [
                "## Example to start a self-supervised task\n",
                "\n",
                "Before you start training, you need to prepare your dataset, please check [prepare_data.md](https://github.com/open-mmlab/mmselfsup/blob/master/docs/en/prepare_data.md) file carefully.\n",
                "\n",
                "**Note**: As we follow the original algorithms to implement our codes, so many algorithms are supposed to run on distributed mode, they are not supported on 1 GPU training officially. You can check it [here](https://github.com/open-mmlab/mmselfsup/blob/master/tools/train.py#L120).\n"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "cece4760",
            "metadata": {
                "id": "cece4760"
            },
            "source": [
                "Here we provide a example and download a small dataset to display the demo."
            ]
        },
        {
            "cell_type": "markdown",
            "id": "AVJ7zKLyahBn",
            "metadata": {
                "id": "AVJ7zKLyahBn"
            },
            "source": [
                "### Prerapre data"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "id": "541169d6",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 19423,
                    "status": "ok",
                    "timestamp": 1662048570348,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "541169d6",
                "outputId": "3aad6344-4a8a-4a19-9858-cc1283608486"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "--2022-09-01 16:09:11--  https://download.openmmlab.com/mmselfsup/data/imagenet_examples.zip\n",
                        "Resolving download.openmmlab.com (download.openmmlab.com)... 47.89.140.71\n",
                        "Connecting to download.openmmlab.com (download.openmmlab.com)|47.89.140.71|:443... connected.\n",
                        "HTTP request sent, awaiting response... 200 OK\n",
                        "Length: 155496559 (148M) [application/zip]\n",
                        "Saving to: ‘imagenet_examples.zip’\n",
                        "\n",
                        "imagenet_examples.z 100%[===================>] 148.29M  8.65MB/s    in 17s     \n",
                        "\n",
                        "2022-09-01 16:09:29 (8.63 MB/s) - ‘imagenet_examples.zip’ saved [155496559/155496559]\n",
                        "\n"
                    ]
                }
            ],
            "source": [
                "!mkdir data\n",
                "!wget https://download.openmmlab.com/mmselfsup/data/imagenet_examples.zip\n",
                "!unzip -q imagenet_examples.zip -d ./data/"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "id": "2fd014a0",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 6244,
                    "status": "ok",
                    "timestamp": 1662048576572,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "2fd014a0",
                "outputId": "1d175f00-1fda-49a2-ea41-84df3d8b60d0"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Reading package lists... Done\n",
                        "Building dependency tree       \n",
                        "Reading state information... Done\n",
                        "The following package was automatically installed and is no longer required:\n",
                        "  libnvidia-common-460\n",
                        "Use 'apt autoremove' to remove it.\n",
                        "The following NEW packages will be installed:\n",
                        "  tree\n",
                        "0 upgraded, 1 newly installed, 0 to remove and 20 not upgraded.\n",
                        "Need to get 40.7 kB of archives.\n",
                        "After this operation, 105 kB of additional disk space will be used.\n",
                        "Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 tree amd64 1.7.0-5 [40.7 kB]\n",
                        "Fetched 40.7 kB in 0s (113 kB/s)\n",
                        "Selecting previously unselected package tree.\n",
                        "(Reading database ... 155676 files and directories currently installed.)\n",
                        "Preparing to unpack .../tree_1.7.0-5_amd64.deb ...\n",
                        "Unpacking tree (1.7.0-5) ...\n",
                        "Setting up tree (1.7.0-5) ...\n",
                        "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...\n",
                        "./data\n",
                        "└── imagenet\n",
                        "    ├── meta\n",
                        "    └── train\n",
                        "        └── n01440764\n",
                        "\n",
                        "4 directories\n"
                    ]
                }
            ],
            "source": [
                "# Check data directory\n",
                "!apt-get install tree\n",
                "!tree -d ./data"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "8cfa1b7b",
            "metadata": {
                "id": "8cfa1b7b"
            },
            "source": [
                "### Create a new config file\n",
                "To reuse the common parts of different config files, we support inheriting multiple base config files. For example, to train `relative_loc` algorithm, the new config file can create the model's basic structure by inheriting `configs/_base_/models/relative-loc.py`."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "id": "d2bbc2de",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 9,
                    "status": "ok",
                    "timestamp": 1662048576573,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "d2bbc2de",
                "outputId": "c035fe91-8d56-4aba-9b4e-02daabac6f21"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "Writing configs/selfsup/relative_loc/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab.py\n"
                    ]
                }
            ],
            "source": [
                "%%writefile configs/selfsup/relative_loc/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab.py\n",
                "_base_ = [\n",
                "    '../_base_/models/relative-loc.py',\n",
                "    '../_base_/datasets/imagenet_relative-loc.py',\n",
                "    '../_base_/schedules/sgd_steplr-200e_in1k.py',\n",
                "    '../_base_/default_runtime.py',\n",
                "]\n",
                "\n",
                "default_hooks = dict(logger=dict(type='LoggerHook', interval=10))\n",
                "\n",
                "# optimizer wrapper\n",
                "optimizer = dict(type='SGD', lr=0.2, momentum=0.9, weight_decay=1e-4)\n",
                "optim_wrapper = dict(\n",
                "    type='OptimWrapper',\n",
                "    optimizer=optimizer,\n",
                "    paramwise_cfg=dict(custom_keys={\n",
                "        'neck': dict(decay_mult=5.0),\n",
                "        'head': dict(decay_mult=5.0)\n",
                "    }))\n",
                "\n",
                "# learning rate scheduler\n",
                "param_scheduler = [dict(type='MultiStepLR', by_epoch=True, milestones=[1, 2])]\n",
                "\n",
                "# runtime settings\n",
                "# pre-train for 70 epochs\n",
                "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=70)\n",
                "# the max_keep_ckpts controls the max number of ckpt file in your work_dirs\n",
                "# if it is 3, when CheckpointHook (in mmcv) saves the 4th ckpt\n",
                "# it will remove the oldest one to keep the number of total ckpts as 3\n",
                "default_hooks = dict(\n",
                "    checkpoint=dict(type='CheckpointHook', interval=1, max_keep_ckpts=3))\n"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "4bc7018d",
            "metadata": {
                "id": "4bc7018d"
            },
            "source": [
                "### Read the config file and modify config\n",
                "\n",
                "We can modify the loaded config file."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "id": "b37379bc",
            "metadata": {
                "executionInfo": {
                    "elapsed": 6,
                    "status": "ok",
                    "timestamp": 1662048576573,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "b37379bc"
            },
            "outputs": [],
            "source": [
                "# Load the basic config file\n",
                "from mmengine.config import Config\n",
                "cfg = Config.fromfile('configs/selfsup/relative_loc/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab.py')\n",
                "\n",
                "# Specify the data settings\n",
                "cfg.train_dataloader.batch_size = 8\n",
                "cfg.train_dataloader.num_workers = 2\n",
                "\n",
                "# Specify the optimizer\n",
                "cfg.optimizer = dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001)\n",
                "cfg.optim_wrapper.clip_grad = None\n",
                "\n",
                "# Specify the learning rate scheduler\n",
                "cfg.param_scheduler = [dict(type='MultiStepLR', by_epoch=True, milestones=[1, 2])]\n",
                "\n",
                "# Modify runtime setting\n",
                "cfg.train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2)\n",
                "\n",
                "# Specify the work directory\n",
                "cfg.work_dir = './work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab'\n",
                "\n",
                "# Output logs for every 10 iterations\n",
                "cfg.default_hooks.logger.interval = 10\n",
                "# Set the random seed and enable the deterministic option of cuDNN\n",
                "# to keep the results' reproducible.\n",
                "cfg.randomness = dict(seed=0, deterministic=True)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "b8be1bfb",
            "metadata": {
                "id": "b8be1bfb"
            },
            "source": [
                "### Start self-supervised pre-train task"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "id": "ff82997c",
            "metadata": {
                "executionInfo": {
                    "elapsed": 7,
                    "status": "ok",
                    "timestamp": 1662048576574,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "ff82997c"
            },
            "outputs": [],
            "source": [
                "import os\n",
                "import torch\n",
                "\n",
                "if torch.cuda.is_available():\n",
                "    os.environ['CUBLAS_WORKSPACE_CONFIG'] = ':16:8'"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "id": "369c3734",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 92165,
                    "status": "ok",
                    "timestamp": 1662048668732,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "369c3734",
                "outputId": "d8d1c353-977f-4558-add2-087ebc972bc0"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "09/01 16:09:40 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "------------------------------------------------------------\n",
                        "System environment:\n",
                        "    sys.platform: linux\n",
                        "    Python: 3.7.13 (default, Apr 24 2022, 01:04:09) [GCC 7.5.0]\n",
                        "    CUDA available: True\n",
                        "    numpy_random_seed: 0\n",
                        "    GPU 0: Tesla T4\n",
                        "    CUDA_HOME: /usr/local/cuda\n",
                        "    NVCC: Cuda compilation tools, release 11.1, V11.1.105\n",
                        "    GCC: x86_64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\n",
                        "    PyTorch: 1.12.1+cu113\n",
                        "    PyTorch compiling details: PyTorch built with:\n",
                        "  - GCC 9.3\n",
                        "  - C++ Version: 201402\n",
                        "  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications\n",
                        "  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)\n",
                        "  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n",
                        "  - LAPACK is enabled (usually provided by MKL)\n",
                        "  - NNPACK is enabled\n",
                        "  - CPU capability usage: AVX2\n",
                        "  - CUDA Runtime 11.3\n",
                        "  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n",
                        "  - CuDNN 8.3.2  (built against CUDA 11.5)\n",
                        "  - Magma 2.5.2\n",
                        "  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n",
                        "\n",
                        "    TorchVision: 0.13.1+cu113\n",
                        "    OpenCV: 4.6.0\n",
                        "    MMEngine: 0.1.0\n",
                        "\n",
                        "Runtime environment:\n",
                        "    cudnn_benchmark: False\n",
                        "    mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n",
                        "    dist_cfg: {'backend': 'nccl'}\n",
                        "    seed: 0\n",
                        "    deterministic: True\n",
                        "    Distributed launcher: none\n",
                        "    Distributed training: False\n",
                        "    GPU number: 1\n",
                        "------------------------------------------------------------\n",
                        "\n",
                        "09/01 16:09:40 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Config:\n",
                        "model = dict(\n",
                        "    type='RelativeLoc',\n",
                        "    data_preprocessor=dict(\n",
                        "        type='mmselfsup.RelativeLocDataPreprocessor',\n",
                        "        mean=[124, 117, 104],\n",
                        "        std=[59, 58, 58],\n",
                        "        bgr_to_rgb=True),\n",
                        "    backbone=dict(\n",
                        "        type='ResNet',\n",
                        "        depth=50,\n",
                        "        in_channels=3,\n",
                        "        out_indices=[4],\n",
                        "        norm_cfg=dict(type='BN')),\n",
                        "    neck=dict(\n",
                        "        type='RelativeLocNeck',\n",
                        "        in_channels=2048,\n",
                        "        out_channels=4096,\n",
                        "        with_avg_pool=True),\n",
                        "    head=dict(\n",
                        "        type='ClsHead',\n",
                        "        loss=dict(type='mmcls.CrossEntropyLoss'),\n",
                        "        with_avg_pool=False,\n",
                        "        in_channels=4096,\n",
                        "        num_classes=8,\n",
                        "        init_cfg=[\n",
                        "            dict(type='Normal', std=0.005, layer='Linear'),\n",
                        "            dict(type='Constant', val=1, layer=['_BatchNorm', 'GroupNorm'])\n",
                        "        ]))\n",
                        "dataset_type = 'mmcls.ImageNet'\n",
                        "data_root = 'data/imagenet/'\n",
                        "file_client_args = dict(backend='disk')\n",
                        "train_pipeline = [\n",
                        "    dict(type='LoadImageFromFile', file_client_args=dict(backend='disk')),\n",
                        "    dict(type='Resize', scale=292),\n",
                        "    dict(type='RandomCrop', size=255),\n",
                        "    dict(type='RandomGrayscale', prob=0.66, keep_channels=True),\n",
                        "    dict(type='RandomPatchWithLabels'),\n",
                        "    dict(\n",
                        "        type='PackSelfSupInputs',\n",
                        "        pseudo_label_keys=['patch_box', 'patch_label', 'unpatched_img'],\n",
                        "        meta_keys=['img_path'])\n",
                        "]\n",
                        "train_dataloader = dict(\n",
                        "    batch_size=8,\n",
                        "    num_workers=2,\n",
                        "    persistent_workers=True,\n",
                        "    sampler=dict(type='DefaultSampler', shuffle=True),\n",
                        "    collate_fn=dict(type='default_collate'),\n",
                        "    dataset=dict(\n",
                        "        type='mmcls.ImageNet',\n",
                        "        data_root='data/imagenet/',\n",
                        "        ann_file='meta/train.txt',\n",
                        "        data_prefix=dict(img_path='train/'),\n",
                        "        pipeline=[\n",
                        "            dict(\n",
                        "                type='LoadImageFromFile',\n",
                        "                file_client_args=dict(backend='disk')),\n",
                        "            dict(type='Resize', scale=292),\n",
                        "            dict(type='RandomCrop', size=255),\n",
                        "            dict(type='RandomGrayscale', prob=0.66, keep_channels=True),\n",
                        "            dict(type='RandomPatchWithLabels'),\n",
                        "            dict(\n",
                        "                type='PackSelfSupInputs',\n",
                        "                pseudo_label_keys=[\n",
                        "                    'patch_box', 'patch_label', 'unpatched_img'\n",
                        "                ],\n",
                        "                meta_keys=['img_path'])\n",
                        "        ]))\n",
                        "optimizer = dict(type='SGD', lr=0.005, momentum=0.9, weight_decay=0.0001)\n",
                        "optim_wrapper = dict(\n",
                        "    type='OptimWrapper',\n",
                        "    optimizer=dict(type='SGD', lr=0.2, weight_decay=0.0001, momentum=0.9),\n",
                        "    paramwise_cfg=dict(\n",
                        "        custom_keys=dict(neck=dict(decay_mult=5.0), head=dict(\n",
                        "            decay_mult=5.0))),\n",
                        "    clip_grad=None)\n",
                        "param_scheduler = [dict(type='MultiStepLR', by_epoch=True, milestones=[1, 2])]\n",
                        "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2)\n",
                        "default_scope = 'mmselfsup'\n",
                        "default_hooks = dict(\n",
                        "    runtime_info=dict(type='RuntimeInfoHook'),\n",
                        "    timer=dict(type='IterTimerHook'),\n",
                        "    logger=dict(type='LoggerHook', interval=10),\n",
                        "    param_scheduler=dict(type='ParamSchedulerHook'),\n",
                        "    checkpoint=dict(type='CheckpointHook', interval=1, max_keep_ckpts=3),\n",
                        "    sampler_seed=dict(type='DistSamplerSeedHook'))\n",
                        "env_cfg = dict(\n",
                        "    cudnn_benchmark=False,\n",
                        "    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n",
                        "    dist_cfg=dict(backend='nccl'))\n",
                        "log_processor = dict(\n",
                        "    window_size=10,\n",
                        "    custom_cfg=[dict(data_src='', method='mean', windows_size='global')])\n",
                        "vis_backends = [dict(type='LocalVisBackend')]\n",
                        "visualizer = dict(\n",
                        "    type='SelfSupVisualizer',\n",
                        "    vis_backends=[dict(type='LocalVisBackend')],\n",
                        "    name='visualizer')\n",
                        "log_level = 'INFO'\n",
                        "load_from = None\n",
                        "resume = False\n",
                        "work_dir = './work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab'\n",
                        "randomness = dict(seed=0, deterministic=True)\n",
                        "\n",
                        "Result has been saved to /content/mmselfsup/work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/modules_statistic_results.json\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.downsample.1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.0.downsample.1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.1.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer1.2.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.downsample.1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.0.downsample.1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.1.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.2.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer2.3.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.downsample.1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.0.downsample.1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.1.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.2.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.3.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.4.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer3.5.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.downsample.1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.0.downsample.1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.1.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn1.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn1.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn2.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn2.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn3.weight:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- backbone.layer4.2.bn3.bias:weight_decay=0.0001\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.weight:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.weight:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.weight:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.bias:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.bias:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.fc.bias:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.weight:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.weight:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.weight:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.bias:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.bias:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- neck.bn.bias:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.weight:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.weight:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.weight:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.bias:lr=0.2\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.bias:weight_decay=0.0005\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - paramwise_options -- head.fc_cls.bias:decay_mult=5.0\n",
                        "09/01 16:09:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Checkpoints will be saved to /content/mmselfsup/work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab by HardDiskBackend.\n",
                        "09/01 16:09:59 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][10/163]  lr: 2.0000e-01  eta: 0:06:02  time: 1.1465  data_time: 0.0440  memory: 1392  loss: 27.9830\n",
                        "09/01 16:10:01 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][20/163]  lr: 2.0000e-01  eta: 0:03:28  time: 0.2144  data_time: 0.0218  memory: 1392  loss: 25.6786\n",
                        "09/01 16:10:03 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][30/163]  lr: 2.0000e-01  eta: 0:02:34  time: 0.2041  data_time: 0.0192  memory: 1392  loss: 14.5791\n",
                        "09/01 16:10:05 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][40/163]  lr: 2.0000e-01  eta: 0:02:06  time: 0.2023  data_time: 0.0192  memory: 1392  loss: 14.2424\n",
                        "09/01 16:10:07 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][50/163]  lr: 2.0000e-01  eta: 0:01:48  time: 0.2026  data_time: 0.0202  memory: 1392  loss: 17.9769\n",
                        "09/01 16:10:09 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][60/163]  lr: 2.0000e-01  eta: 0:01:36  time: 0.2026  data_time: 0.0196  memory: 1392  loss: 18.9486\n",
                        "09/01 16:10:11 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][70/163]  lr: 2.0000e-01  eta: 0:01:26  time: 0.2025  data_time: 0.0200  memory: 1392  loss: 28.0319\n",
                        "09/01 16:10:13 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][80/163]  lr: 2.0000e-01  eta: 0:01:19  time: 0.2033  data_time: 0.0198  memory: 1392  loss: 17.6793\n",
                        "09/01 16:10:15 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][90/163]  lr: 2.0000e-01  eta: 0:01:12  time: 0.2048  data_time: 0.0195  memory: 1392  loss: 15.4679\n",
                        "09/01 16:10:17 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][100/163]  lr: 2.0000e-01  eta: 0:01:07  time: 0.2058  data_time: 0.0206  memory: 1392  loss: 6.8410\n",
                        "09/01 16:10:19 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][110/163]  lr: 2.0000e-01  eta: 0:01:02  time: 0.2036  data_time: 0.0203  memory: 1392  loss: 6.3352\n",
                        "09/01 16:10:21 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][120/163]  lr: 2.0000e-01  eta: 0:00:58  time: 0.2045  data_time: 0.0200  memory: 1392  loss: 6.0879\n",
                        "09/01 16:10:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][130/163]  lr: 2.0000e-01  eta: 0:00:54  time: 0.2206  data_time: 0.0241  memory: 1392  loss: 4.7499\n",
                        "09/01 16:10:26 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][140/163]  lr: 2.0000e-01  eta: 0:00:50  time: 0.2143  data_time: 0.0199  memory: 1392  loss: 3.4295\n",
                        "09/01 16:10:28 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][150/163]  lr: 2.0000e-01  eta: 0:00:47  time: 0.2055  data_time: 0.0199  memory: 1392  loss: 3.2668\n",
                        "09/01 16:10:30 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][160/163]  lr: 2.0000e-01  eta: 0:00:43  time: 0.2033  data_time: 0.0188  memory: 1392  loss: 2.7335\n",
                        "09/01 16:10:30 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: relative-loc_resnet50_8xb64-steplr-70e_in1k_colab_20220901_160940\n",
                        "09/01 16:10:30 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 1 epochs\n",
                        "09/01 16:10:35 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][10/163]  lr: 2.0000e-02  eta: 0:00:39  time: 0.2302  data_time: 0.0299  memory: 1392  loss: 2.3706\n",
                        "09/01 16:10:37 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][20/163]  lr: 2.0000e-02  eta: 0:00:36  time: 0.2050  data_time: 0.0191  memory: 1392  loss: 2.2516\n",
                        "09/01 16:10:39 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][30/163]  lr: 2.0000e-02  eta: 0:00:33  time: 0.2100  data_time: 0.0202  memory: 1392  loss: 2.2116\n",
                        "09/01 16:10:41 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][40/163]  lr: 2.0000e-02  eta: 0:00:30  time: 0.2073  data_time: 0.0213  memory: 1392  loss: 2.1653\n",
                        "09/01 16:10:43 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][50/163]  lr: 2.0000e-02  eta: 0:00:28  time: 0.2103  data_time: 0.0209  memory: 1392  loss: 2.1445\n",
                        "09/01 16:10:45 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][60/163]  lr: 2.0000e-02  eta: 0:00:25  time: 0.2064  data_time: 0.0190  memory: 1392  loss: 2.1613\n",
                        "09/01 16:10:47 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][70/163]  lr: 2.0000e-02  eta: 0:00:22  time: 0.2084  data_time: 0.0215  memory: 1392  loss: 2.1216\n",
                        "09/01 16:10:49 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][80/163]  lr: 2.0000e-02  eta: 0:00:20  time: 0.2060  data_time: 0.0206  memory: 1392  loss: 2.1333\n",
                        "09/01 16:10:51 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][90/163]  lr: 2.0000e-02  eta: 0:00:17  time: 0.2072  data_time: 0.0196  memory: 1392  loss: 2.1104\n",
                        "09/01 16:10:54 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][100/163]  lr: 2.0000e-02  eta: 0:00:15  time: 0.2073  data_time: 0.0198  memory: 1392  loss: 2.1128\n",
                        "09/01 16:10:56 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][110/163]  lr: 2.0000e-02  eta: 0:00:12  time: 0.2056  data_time: 0.0195  memory: 1392  loss: 2.1260\n",
                        "09/01 16:10:58 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][120/163]  lr: 2.0000e-02  eta: 0:00:10  time: 0.2072  data_time: 0.0195  memory: 1392  loss: 2.1056\n",
                        "09/01 16:11:00 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][130/163]  lr: 2.0000e-02  eta: 0:00:07  time: 0.2100  data_time: 0.0196  memory: 1392  loss: 2.0948\n",
                        "09/01 16:11:02 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][140/163]  lr: 2.0000e-02  eta: 0:00:05  time: 0.2067  data_time: 0.0199  memory: 1392  loss: 2.0966\n",
                        "09/01 16:11:04 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][150/163]  lr: 2.0000e-02  eta: 0:00:03  time: 0.2082  data_time: 0.0196  memory: 1392  loss: 2.0897\n",
                        "09/01 16:11:06 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][160/163]  lr: 2.0000e-02  eta: 0:00:00  time: 0.2043  data_time: 0.0190  memory: 1392  loss: 2.0927\n",
                        "09/01 16:11:06 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: relative-loc_resnet50_8xb64-steplr-70e_in1k_colab_20220901_160940\n",
                        "09/01 16:11:06 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 2 epochs\n"
                    ]
                },
                {
                    "data": {
                        "text/plain": [
                            "RelativeLoc(\n",
                            "  (data_preprocessor): RelativeLocDataPreprocessor()\n",
                            "  (backbone): ResNet(\n",
                            "    (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
                            "    (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "    (relu): ReLU(inplace=True)\n",
                            "    (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
                            "    (layer1): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer2): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (3): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer3): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (3): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (4): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (5): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer4): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "  )\n",
                            "  init_cfg=[{'type': 'Kaiming', 'layer': ['Conv2d']}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}]\n",
                            "  (neck): RelativeLocNeck(\n",
                            "    (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))\n",
                            "    (fc): Linear(in_features=4096, out_features=4096, bias=True)\n",
                            "    (bn): BatchNorm1d(4096, eps=1e-05, momentum=0.003, affine=True, track_running_stats=True)\n",
                            "    (relu): ReLU(inplace=True)\n",
                            "    (dropout): Dropout(p=0.5, inplace=False)\n",
                            "  )\n",
                            "  init_cfg=[{'type': 'Normal', 'std': 0.01, 'layer': 'Linear'}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}]\n",
                            "  (head): ClsHead(\n",
                            "    (loss): CrossEntropyLoss()\n",
                            "    (fc_cls): Linear(in_features=4096, out_features=8, bias=True)\n",
                            "  )\n",
                            "  init_cfg=[{'type': 'Normal', 'std': 0.005, 'layer': 'Linear'}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}]\n",
                            ")"
                        ]
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "from mmengine.runner import Runner\n",
                "\n",
                "# build the runner from config\n",
                "runner = Runner.from_cfg(cfg)\n",
                "\n",
                "# start training\n",
                "runner.train()"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "a562c2dd",
            "metadata": {
                "id": "a562c2dd"
            },
            "source": [
                "## Example to start a downstream task\n"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "96ea98b2",
            "metadata": {
                "id": "96ea98b2"
            },
            "source": [
                "### Extract backbone weights from pre-train model"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "id": "9fa74770",
            "metadata": {
                "executionInfo": {
                    "elapsed": 2019,
                    "status": "ok",
                    "timestamp": 1662048670738,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "9fa74770"
            },
            "outputs": [],
            "source": [
                "!python tools/model_converters/extract_backbone_weights.py \\\n",
                "  work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/epoch_2.pth \\\n",
                "  work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "0f137b0e",
            "metadata": {
                "id": "0f137b0e"
            },
            "source": [
                "### Prepare config file\n",
                "\n",
                "Here we create a new config file for demo dataset, actually we provided various config files in directory `configs/benchmarks`."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "id": "65764022",
            "metadata": {
                "executionInfo": {
                    "elapsed": 7,
                    "status": "ok",
                    "timestamp": 1662048670739,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "65764022"
            },
            "outputs": [],
            "source": [
                "# Load the basic config file\n",
                "from mmengine.config import Config\n",
                "benchmark_cfg = Config.fromfile('configs/benchmarks/classification/imagenet/resnet50_linear-8xb32-steplr-100e_in1k.py')\n",
                "\n",
                "# Modify the model\n",
                "checkpoint_file = 'work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth'\n",
                "# Or directly using pre-train model provided by us\n",
                "# checkpoint_file = 'https://download.openmmlab.com/mmselfsup/moco/mocov2_resnet50_8xb32-coslr-200e_in1k_20220225-89e03af4.pth'\n",
                "\n",
                "benchmark_cfg.model.backbone.frozen_stages=4\n",
                "benchmark_cfg.model.backbone.init_cfg = dict(type='Pretrained', checkpoint=checkpoint_file)\n",
                "\n",
                "# As the imagenet_examples dataset folder doesn't have val dataset\n",
                "# Modify the path and meta files of validation dataset\n",
                "benchmark_cfg.val_dataloader.dataset.data_prefix = 'train'\n",
                "benchmark_cfg.val_dataloader.dataset.ann_file = 'meta/train.txt'\n",
                "\n",
                "# Specify the learning rate scheduler\n",
                "benchmark_cfg.param_scheduler = [dict(type='MultiStepLR', by_epoch=True, milestones=[1, 2])]\n",
                "\n",
                "# Output logs for every 10 iterations\n",
                "benchmark_cfg.default_hooks.logger.interval = 10\n",
                "\n",
                "# Modify runtime settings for demo\n",
                "benchmark_cfg.train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2)\n",
                "\n",
                "\n",
                "# Specify the work directory\n",
                "benchmark_cfg.work_dir = './work_dirs/benchmarks/classification/imagenet/resnet50_8xb32-steplr-100e_in1k_colab'\n",
                "\n",
                "# Set the random seed and enable the deterministic option of cuDNN\n",
                "# to keep the results' reproducible.\n",
                "benchmark_cfg.randomness = dict(seed=0, deterministic=True)"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "636e8865",
            "metadata": {
                "id": "636e8865"
            },
            "source": [
                "### Load extracted backbone weights to start a downstream task"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "id": "f9c51d5c",
            "metadata": {
                "colab": {
                    "base_uri": "https://localhost:8080/"
                },
                "executionInfo": {
                    "elapsed": 62032,
                    "status": "ok",
                    "timestamp": 1662048732765,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "f9c51d5c",
                "outputId": "7c83b7ac-20f3-4944-bedf-53fd2bf83890"
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "09/01 16:11:10 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "------------------------------------------------------------\n",
                        "System environment:\n",
                        "    sys.platform: linux\n",
                        "    Python: 3.7.13 (default, Apr 24 2022, 01:04:09) [GCC 7.5.0]\n",
                        "    CUDA available: True\n",
                        "    numpy_random_seed: 0\n",
                        "    GPU 0: Tesla T4\n",
                        "    CUDA_HOME: /usr/local/cuda\n",
                        "    NVCC: Cuda compilation tools, release 11.1, V11.1.105\n",
                        "    GCC: x86_64-linux-gnu-gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\n",
                        "    PyTorch: 1.12.1+cu113\n",
                        "    PyTorch compiling details: PyTorch built with:\n",
                        "  - GCC 9.3\n",
                        "  - C++ Version: 201402\n",
                        "  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications\n",
                        "  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)\n",
                        "  - OpenMP 201511 (a.k.a. OpenMP 4.5)\n",
                        "  - LAPACK is enabled (usually provided by MKL)\n",
                        "  - NNPACK is enabled\n",
                        "  - CPU capability usage: AVX2\n",
                        "  - CUDA Runtime 11.3\n",
                        "  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86\n",
                        "  - CuDNN 8.3.2  (built against CUDA 11.5)\n",
                        "  - Magma 2.5.2\n",
                        "  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, \n",
                        "\n",
                        "    TorchVision: 0.13.1+cu113\n",
                        "    OpenCV: 4.6.0\n",
                        "    MMEngine: 0.1.0\n",
                        "\n",
                        "Runtime environment:\n",
                        "    cudnn_benchmark: False\n",
                        "    mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0}\n",
                        "    dist_cfg: {'backend': 'nccl'}\n",
                        "    seed: 0\n",
                        "    deterministic: True\n",
                        "    Distributed launcher: none\n",
                        "    Distributed training: False\n",
                        "    GPU number: 1\n",
                        "------------------------------------------------------------\n",
                        "\n",
                        "09/01 16:11:11 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Config:\n",
                        "model = dict(\n",
                        "    type='ImageClassifier',\n",
                        "    data_preprocessor=dict(\n",
                        "        mean=[123.675, 116.28, 103.53],\n",
                        "        std=[58.395, 57.12, 57.375],\n",
                        "        to_rgb=True),\n",
                        "    backbone=dict(\n",
                        "        type='ResNet',\n",
                        "        depth=50,\n",
                        "        in_channels=3,\n",
                        "        num_stages=4,\n",
                        "        norm_cfg=dict(type='BN'),\n",
                        "        frozen_stages=4,\n",
                        "        init_cfg=dict(\n",
                        "            type='Pretrained',\n",
                        "            checkpoint=\n",
                        "            'work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth'\n",
                        "        )),\n",
                        "    neck=dict(type='GlobalAveragePooling'),\n",
                        "    head=dict(\n",
                        "        type='LinearClsHead',\n",
                        "        num_classes=1000,\n",
                        "        in_channels=2048,\n",
                        "        loss=dict(type='CrossEntropyLoss', loss_weight=1.0),\n",
                        "        topk=(1, 5)))\n",
                        "dataset_type = 'ImageNet'\n",
                        "data_root = 'data/imagenet/'\n",
                        "file_client_args = dict(backend='disk')\n",
                        "train_pipeline = [\n",
                        "    dict(type='LoadImageFromFile', file_client_args=dict(backend='disk')),\n",
                        "    dict(type='RandomResizedCrop', scale=224, backend='pillow'),\n",
                        "    dict(type='RandomFlip', prob=0.5, direction='horizontal'),\n",
                        "    dict(type='PackClsInputs')\n",
                        "]\n",
                        "test_pipeline = [\n",
                        "    dict(type='LoadImageFromFile', file_client_args=dict(backend='disk')),\n",
                        "    dict(type='ResizeEdge', scale=256, edge='short', backend='pillow'),\n",
                        "    dict(type='CenterCrop', crop_size=224),\n",
                        "    dict(type='PackClsInputs')\n",
                        "]\n",
                        "train_dataloader = dict(\n",
                        "    batch_size=32,\n",
                        "    num_workers=4,\n",
                        "    dataset=dict(\n",
                        "        type='ImageNet',\n",
                        "        data_root='data/imagenet',\n",
                        "        ann_file='meta/train.txt',\n",
                        "        data_prefix='train',\n",
                        "        pipeline=[\n",
                        "            dict(\n",
                        "                type='LoadImageFromFile',\n",
                        "                file_client_args=dict(backend='disk')),\n",
                        "            dict(type='RandomResizedCrop', scale=224, backend='pillow'),\n",
                        "            dict(type='RandomFlip', prob=0.5, direction='horizontal'),\n",
                        "            dict(type='PackClsInputs')\n",
                        "        ]),\n",
                        "    sampler=dict(type='DefaultSampler', shuffle=True),\n",
                        "    persistent_workers=True)\n",
                        "val_dataloader = dict(\n",
                        "    batch_size=32,\n",
                        "    num_workers=4,\n",
                        "    dataset=dict(\n",
                        "        type='ImageNet',\n",
                        "        data_root='data/imagenet',\n",
                        "        ann_file='meta/train.txt',\n",
                        "        data_prefix='train',\n",
                        "        pipeline=[\n",
                        "            dict(\n",
                        "                type='LoadImageFromFile',\n",
                        "                file_client_args=dict(backend='disk')),\n",
                        "            dict(type='ResizeEdge', scale=256, edge='short', backend='pillow'),\n",
                        "            dict(type='CenterCrop', crop_size=224),\n",
                        "            dict(type='PackClsInputs')\n",
                        "        ]),\n",
                        "    sampler=dict(type='DefaultSampler', shuffle=False),\n",
                        "    persistent_workers=True)\n",
                        "val_evaluator = dict(type='mmcls.Accuracy', topk=(1, 5))\n",
                        "test_dataloader = dict(\n",
                        "    batch_size=32,\n",
                        "    num_workers=4,\n",
                        "    dataset=dict(\n",
                        "        type='ImageNet',\n",
                        "        data_root='data/imagenet',\n",
                        "        ann_file='meta/val.txt',\n",
                        "        data_prefix='val',\n",
                        "        pipeline=[\n",
                        "            dict(\n",
                        "                type='LoadImageFromFile',\n",
                        "                file_client_args=dict(backend='disk')),\n",
                        "            dict(type='ResizeEdge', scale=256, edge='short', backend='pillow'),\n",
                        "            dict(type='CenterCrop', crop_size=224),\n",
                        "            dict(type='PackClsInputs')\n",
                        "        ]),\n",
                        "    sampler=dict(type='DefaultSampler', shuffle=False),\n",
                        "    persistent_workers=True)\n",
                        "test_evaluator = dict(type='mmcls.Accuracy', topk=(1, 5))\n",
                        "optimizer = dict(type='SGD', lr=30.0, momentum=0.9, weight_decay=0.0)\n",
                        "optim_wrapper = dict(\n",
                        "    type='OptimWrapper',\n",
                        "    optimizer=dict(type='SGD', lr=30.0, momentum=0.9, weight_decay=0.0))\n",
                        "param_scheduler = [dict(type='MultiStepLR', by_epoch=True, milestones=[1, 2])]\n",
                        "train_cfg = dict(type='EpochBasedTrainLoop', max_epochs=2)\n",
                        "val_cfg = dict()\n",
                        "test_cfg = dict()\n",
                        "default_scope = 'mmcls'\n",
                        "custom_imports = dict(\n",
                        "    imports=['mmselfsup.models', 'mmselfsup.engine'],\n",
                        "    allow_failed_imports=False)\n",
                        "default_hooks = dict(\n",
                        "    runtime_info=dict(type='RuntimeInfoHook'),\n",
                        "    timer=dict(type='IterTimerHook'),\n",
                        "    logger=dict(type='LoggerHook', interval=10),\n",
                        "    param_scheduler=dict(type='ParamSchedulerHook'),\n",
                        "    checkpoint=dict(type='CheckpointHook', interval=10, max_keep_ckpts=3),\n",
                        "    sampler_seed=dict(type='DistSamplerSeedHook'))\n",
                        "env_cfg = dict(\n",
                        "    cudnn_benchmark=False,\n",
                        "    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),\n",
                        "    dist_cfg=dict(backend='nccl'))\n",
                        "log_processor = dict(\n",
                        "    window_size=10,\n",
                        "    custom_cfg=[dict(data_src='', method='mean', windows_size='global')])\n",
                        "vis_backends = [dict(type='LocalVisBackend')]\n",
                        "visualizer = dict(\n",
                        "    type='ClsVisualizer',\n",
                        "    vis_backends=[dict(type='LocalVisBackend')],\n",
                        "    name='visualizer')\n",
                        "log_level = 'INFO'\n",
                        "load_from = None\n",
                        "resume = False\n",
                        "work_dir = './work_dirs/benchmarks/classification/imagenet/resnet50_8xb32-steplr-100e_in1k_colab'\n",
                        "randomness = dict(seed=0, deterministic=True)\n",
                        "\n",
                        "Result has been saved to /content/mmselfsup/work_dirs/benchmarks/classification/imagenet/resnet50_8xb32-steplr-100e_in1k_colab/modules_statistic_results.json\n",
                        "09/01 16:11:11 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used.\n"
                    ]
                },
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:566: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n",
                        "  cpuset_checked))\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - load model from: work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth\n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - local loads checkpoint from path: work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth\n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.conv1.weight - torch.Size([64, 3, 7, 7]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.bn1.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.bn1.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn1.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn1.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn2.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn2.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn3.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.bn3.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.downsample.1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.0.downsample.1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn1.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn1.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn2.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn2.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn3.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.1.bn3.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn1.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn1.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn2.weight - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn2.bias - torch.Size([64]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn3.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer1.2.bn3.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn1.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn1.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn2.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn2.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn3.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.bn3.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.downsample.1.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.0.downsample.1.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn1.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn1.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn2.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn2.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn3.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.1.bn3.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn1.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn1.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn2.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn2.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn3.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.2.bn3.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn1.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn1.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn2.weight - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn2.bias - torch.Size([128]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn3.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer2.3.bn3.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.downsample.1.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.0.downsample.1.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.1.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.2.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:24 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.3.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.4.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn1.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn1.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn2.weight - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn2.bias - torch.Size([256]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn3.weight - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer3.5.bn3.bias - torch.Size([1024]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn1.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn1.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn2.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn2.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn3.weight - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.bn3.bias - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.downsample.1.weight - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.0.downsample.1.bias - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn1.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn1.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn2.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn2.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn3.weight - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.1.bn3.bias - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn1.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn1.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn2.weight - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn2.bias - torch.Size([512]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn3.weight - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "backbone.layer4.2.bn3.bias - torch.Size([2048]): \n",
                        "PretrainedInit: load from work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "head.fc.weight - torch.Size([1000, 2048]): \n",
                        "NormalInit: mean=0, std=0.01, bias=0 \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - \n",
                        "head.fc.bias - torch.Size([1000]): \n",
                        "NormalInit: mean=0, std=0.01, bias=0 \n",
                        " \n",
                        "09/01 16:11:25 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Checkpoints will be saved to /content/mmselfsup/work_dirs/benchmarks/classification/imagenet/resnet50_8xb32-steplr-100e_in1k_colab by HardDiskBackend.\n"
                    ]
                },
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": [
                        "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:566: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n",
                        "  cpuset_checked))\n"
                    ]
                },
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": [
                        "09/01 16:11:30 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][10/41]  lr: 3.0000e+01  eta: 0:00:35  time: 0.4955  data_time: 0.3703  memory: 1392  loss: 1.0352\n",
                        "09/01 16:11:32 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][20/41]  lr: 3.0000e+01  eta: 0:00:23  time: 0.2497  data_time: 0.1329  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:35 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][30/41]  lr: 3.0000e+01  eta: 0:00:17  time: 0.2528  data_time: 0.1333  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:37 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [1][40/41]  lr: 3.0000e+01  eta: 0:00:12  time: 0.2088  data_time: 0.0967  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:37 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: resnet50_linear-8xb32-steplr-100e_in1k_20220901_161110\n",
                        "09/01 16:11:41 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [1][10/41]    eta: 0:00:12  time: 0.4174  data_time: 0.2966  memory: 762  \n",
                        "09/01 16:11:43 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [1][20/41]    eta: 0:00:03  time: 0.1886  data_time: 0.0625  memory: 762  \n",
                        "09/01 16:11:45 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [1][30/41]    eta: 0:00:02  time: 0.2412  data_time: 0.1149  memory: 762  \n",
                        "09/01 16:11:48 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [1][40/41]    eta: 0:00:00  time: 0.2665  data_time: 0.1533  memory: 762  \n",
                        "09/01 16:11:48 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [1][41/41]  accuracy/top1: 100.0000  accuracy/top5: 100.0000\n",
                        "09/01 16:11:52 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][10/41]  lr: 3.0000e+00  eta: 0:00:09  time: 0.3464  data_time: 0.2278  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:54 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][20/41]  lr: 3.0000e+00  eta: 0:00:06  time: 0.2781  data_time: 0.1648  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:57 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][30/41]  lr: 3.0000e+00  eta: 0:00:03  time: 0.2383  data_time: 0.1167  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:59 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(train) [2][40/41]  lr: 3.0000e+00  eta: 0:00:00  time: 0.2536  data_time: 0.1397  memory: 762  loss: 0.0000\n",
                        "09/01 16:11:59 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Exp name: resnet50_linear-8xb32-steplr-100e_in1k_20220901_161110\n",
                        "09/01 16:11:59 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Saving checkpoint at 2 epochs\n",
                        "09/01 16:12:05 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [2][10/41]    eta: 0:00:11  time: 0.3788  data_time: 0.2459  memory: 762  \n",
                        "09/01 16:12:07 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [2][20/41]    eta: 0:00:04  time: 0.2033  data_time: 0.0714  memory: 762  \n",
                        "09/01 16:12:09 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [2][30/41]    eta: 0:00:02  time: 0.2373  data_time: 0.1092  memory: 762  \n",
                        "09/01 16:12:12 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [2][40/41]    eta: 0:00:00  time: 0.2671  data_time: 0.1587  memory: 762  \n",
                        "09/01 16:12:12 - mmengine - \u001b[4m\u001b[37mINFO\u001b[0m - Epoch(val) [2][41/41]  accuracy/top1: 100.0000  accuracy/top5: 100.0000\n"
                    ]
                },
                {
                    "data": {
                        "text/plain": [
                            "ImageClassifier(\n",
                            "  (data_preprocessor): ClsDataPreprocessor()\n",
                            "  (backbone): ResNet(\n",
                            "    (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
                            "    (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "    (relu): ReLU(inplace=True)\n",
                            "    (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
                            "    (layer1): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer2): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (3): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer3): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (3): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (4): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (5): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "    (layer4): ResLayer(\n",
                            "      (0): Bottleneck(\n",
                            "        (conv1): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (downsample): Sequential(\n",
                            "          (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
                            "          (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        )\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (1): Bottleneck(\n",
                            "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "      (2): Bottleneck(\n",
                            "        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
                            "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
                            "        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
                            "        (relu): ReLU(inplace=True)\n",
                            "        (drop_path): Identity()\n",
                            "      )\n",
                            "    )\n",
                            "  )\n",
                            "  init_cfg={'type': 'Pretrained', 'checkpoint': 'work_dirs/selfsup/relative-loc_resnet50_8xb64-steplr-70e_in1k_colab/relative-loc_backbone-weights.pth'}\n",
                            "  (neck): GlobalAveragePooling(\n",
                            "    (gap): AdaptiveAvgPool2d(output_size=(1, 1))\n",
                            "  )\n",
                            "  (head): LinearClsHead(\n",
                            "    (loss_module): CrossEntropyLoss()\n",
                            "    (fc): Linear(in_features=2048, out_features=1000, bias=True)\n",
                            "  )\n",
                            "  init_cfg={'type': 'Normal', 'layer': 'Linear', 'std': 0.01}\n",
                            ")"
                        ]
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": [
                "from mmengine.runner import Runner\n",
                "\n",
                "# build the runner from config\n",
                "runner = Runner.from_cfg(benchmark_cfg)\n",
                "\n",
                "# start training\n",
                "runner.train()"
            ]
        },
        {
            "cell_type": "markdown",
            "id": "e1b5b983",
            "metadata": {
                "id": "e1b5b983"
            },
            "source": [
                "**Note: As the demo only has one class in dataset, the model collapsed and the results of loss and acc should be ignored.**"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "id": "4A0WOMeeeZ9E",
            "metadata": {
                "executionInfo": {
                    "elapsed": 33,
                    "status": "ok",
                    "timestamp": 1662048732766,
                    "user": {
                        "displayName": "qin ren",
                        "userId": "07205769677379266243"
                    },
                    "user_tz": -480
                },
                "id": "4A0WOMeeeZ9E"
            },
            "outputs": [],
            "source": []
        }
    ],
    "metadata": {
        "accelerator": "GPU",
        "colab": {
            "collapsed_sections": [],
            "provenance": [],
            "toc_visible": true
        },
        "gpuClass": "standard",
        "kernelspec": {
            "display_name": "openmmlab",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.7.0 (default, Oct  9 2018, 10:31:47) \n[GCC 7.3.0]"
        },
        "vscode": {
            "interpreter": {
                "hash": "5909b3386efe3692f76356628babf720cfd47771f5d858315790cc041eb41361"
            }
        }
    },
    "nbformat": 4,
    "nbformat_minor": 5
}
