2022-04-15 20:19:20 +08:00
|
|
|
.. role:: hidden
|
|
|
|
|
:class: hidden-section
|
|
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models
|
2022-08-30 18:45:58 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
mmpretrain.models
|
2022-04-15 20:19:20 +08:00
|
|
|
===================================
|
|
|
|
|
|
|
|
|
|
The ``models`` package contains several sub-packages for addressing the different components of a model.
|
|
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
- :mod:`~mmpretrain.models.classifiers`: The top-level module which defines the whole process of a classification model.
|
2023-03-06 16:53:15 +08:00
|
|
|
- :mod:`~mmpretrain.models.selfsup`: The top-level module which defines the whole process of a self-supervised learning model.
|
2023-03-20 15:56:09 +08:00
|
|
|
- :mod:`~mmpretrain.models.retrievers`: The top-level module which defines the whole process of a retrieval model.
|
2023-03-02 13:29:07 +08:00
|
|
|
- :mod:`~mmpretrain.models.backbones`: Usually a feature extraction network, e.g., ResNet, MobileNet.
|
|
|
|
|
- :mod:`~mmpretrain.models.necks`: The component between backbones and heads, e.g., GlobalAveragePooling.
|
2023-04-06 20:58:52 +08:00
|
|
|
- :mod:`~mmpretrain.models.heads`: The component for specific tasks.
|
2023-03-02 13:29:07 +08:00
|
|
|
- :mod:`~mmpretrain.models.losses`: Loss functions.
|
|
|
|
|
- :mod:`~mmpretrain.models.utils`: Some helper functions and common components used in various networks.
|
2022-08-29 15:35:10 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
- :mod:`~mmpretrain.models.utils.data_preprocessor`: The component before model to preprocess the inputs, e.g., ClsDataPreprocessor.
|
2022-08-29 15:35:10 +08:00
|
|
|
- :ref:`components`: Common components used in various networks.
|
|
|
|
|
- :ref:`helpers`: Helper functions.
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2022-08-29 15:35:10 +08:00
|
|
|
Build Functions
|
|
|
|
|
---------------
|
|
|
|
|
|
2022-04-15 20:19:20 +08:00
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
build_classifier
|
|
|
|
|
build_backbone
|
|
|
|
|
build_neck
|
|
|
|
|
build_head
|
|
|
|
|
build_loss
|
|
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.classifiers
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2022-08-30 18:45:58 +08:00
|
|
|
Classifiers
|
2022-04-15 20:19:20 +08:00
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
2022-08-29 15:35:10 +08:00
|
|
|
BaseClassifier
|
|
|
|
|
ImageClassifier
|
2022-11-10 14:56:19 +08:00
|
|
|
TimmClassifier
|
|
|
|
|
HuggingFaceClassifier
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2023-03-06 16:53:15 +08:00
|
|
|
.. module:: mmpretrain.models.selfsup
|
|
|
|
|
|
|
|
|
|
Self-supervised Algorithms
|
|
|
|
|
--------------------------
|
|
|
|
|
|
|
|
|
|
.. _selfsup_algorithms:
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
BaseSelfSupervisor
|
|
|
|
|
BEiT
|
|
|
|
|
BYOL
|
|
|
|
|
BarlowTwins
|
|
|
|
|
CAE
|
|
|
|
|
DenseCL
|
|
|
|
|
EVA
|
2023-05-26 12:08:34 +08:00
|
|
|
iTPN
|
2023-03-06 16:53:15 +08:00
|
|
|
MAE
|
|
|
|
|
MILAN
|
|
|
|
|
MaskFeat
|
|
|
|
|
MixMIM
|
|
|
|
|
MoCo
|
|
|
|
|
MoCoV3
|
|
|
|
|
SimCLR
|
|
|
|
|
SimMIM
|
|
|
|
|
SimSiam
|
2023-06-19 11:27:50 +08:00
|
|
|
SparK
|
2023-03-06 16:53:15 +08:00
|
|
|
SwAV
|
|
|
|
|
|
|
|
|
|
.. _selfsup_backbones:
|
|
|
|
|
|
|
|
|
|
Some of above algorithms modified the backbone module to adapt the extra inputs
|
|
|
|
|
like ``mask``, and here is the a list of these **modified backbone** modules.
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
BEiTPretrainViT
|
|
|
|
|
CAEPretrainViT
|
2023-05-26 12:08:34 +08:00
|
|
|
iTPNHiViT
|
|
|
|
|
MAEHiViT
|
2023-03-06 16:53:15 +08:00
|
|
|
MAEViT
|
|
|
|
|
MILANViT
|
|
|
|
|
MaskFeatViT
|
|
|
|
|
MixMIMPretrainTransformer
|
|
|
|
|
MoCoV3ViT
|
|
|
|
|
SimMIMSwinTransformer
|
|
|
|
|
|
|
|
|
|
.. _target_generators:
|
|
|
|
|
|
|
|
|
|
Some self-supervise algorithms need an external **target generator** to
|
|
|
|
|
generate the optimization target. Here is a list of target generators.
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
VQKD
|
|
|
|
|
DALLEEncoder
|
|
|
|
|
HOGGenerator
|
|
|
|
|
CLIPGenerator
|
|
|
|
|
|
2023-03-20 15:56:09 +08:00
|
|
|
.. module:: mmpretrain.models.retrievers
|
|
|
|
|
|
|
|
|
|
Retrievers
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
BaseRetriever
|
|
|
|
|
ImageToImageRetriever
|
|
|
|
|
|
2023-05-19 16:50:04 +08:00
|
|
|
.. module:: mmpretrain.models.multimodal
|
|
|
|
|
|
|
|
|
|
Multi-Modality Algorithms
|
|
|
|
|
--------------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
Blip2Caption
|
|
|
|
|
Blip2Retrieval
|
|
|
|
|
Blip2VQA
|
|
|
|
|
BlipCaption
|
|
|
|
|
BlipGrounding
|
|
|
|
|
BlipNLVR
|
|
|
|
|
BlipRetrieval
|
|
|
|
|
BlipVQA
|
|
|
|
|
Flamingo
|
|
|
|
|
OFA
|
2023-06-16 22:50:34 +08:00
|
|
|
MiniGPT4
|
2023-06-17 16:05:52 +08:00
|
|
|
Llava
|
2023-06-17 16:03:21 +08:00
|
|
|
Otter
|
2023-05-19 16:50:04 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.backbones
|
2022-04-15 20:19:20 +08:00
|
|
|
|
|
|
|
|
Backbones
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
AlexNet
|
2023-03-02 13:29:07 +08:00
|
|
|
BEiTViT
|
2022-04-15 20:19:20 +08:00
|
|
|
CSPDarkNet
|
|
|
|
|
CSPNet
|
|
|
|
|
CSPResNeXt
|
|
|
|
|
CSPResNet
|
|
|
|
|
Conformer
|
|
|
|
|
ConvMixer
|
|
|
|
|
ConvNeXt
|
2022-11-16 17:23:55 +08:00
|
|
|
DaViT
|
2022-10-10 14:54:20 +08:00
|
|
|
DeiT3
|
2022-10-18 17:05:59 +08:00
|
|
|
DenseNet
|
2022-04-15 20:19:20 +08:00
|
|
|
DistilledVisionTransformer
|
2022-10-18 17:05:59 +08:00
|
|
|
EdgeNeXt
|
2022-09-20 14:56:45 +08:00
|
|
|
EfficientFormer
|
2022-04-15 20:19:20 +08:00
|
|
|
EfficientNet
|
2022-12-30 15:18:39 +08:00
|
|
|
EfficientNetV2
|
2023-05-26 12:08:34 +08:00
|
|
|
HiViT
|
2022-04-15 20:19:20 +08:00
|
|
|
HRNet
|
2022-12-14 11:46:39 +08:00
|
|
|
HorNet
|
2022-09-20 14:56:45 +08:00
|
|
|
InceptionV3
|
2022-04-15 20:19:20 +08:00
|
|
|
LeNet5
|
2023-01-17 17:43:42 +08:00
|
|
|
LeViT
|
2022-09-22 09:23:49 +08:00
|
|
|
MViT
|
2022-04-15 20:19:20 +08:00
|
|
|
MlpMixer
|
|
|
|
|
MobileNetV2
|
|
|
|
|
MobileNetV3
|
2022-09-20 14:56:45 +08:00
|
|
|
MobileOne
|
2022-10-18 17:05:59 +08:00
|
|
|
MobileViT
|
2022-04-15 20:19:20 +08:00
|
|
|
PCPVT
|
|
|
|
|
PoolFormer
|
2023-01-17 16:55:56 +08:00
|
|
|
PyramidVig
|
2022-04-15 20:19:20 +08:00
|
|
|
RegNet
|
2022-11-21 10:18:58 +08:00
|
|
|
RepLKNet
|
2022-04-15 20:19:20 +08:00
|
|
|
RepMLPNet
|
|
|
|
|
RepVGG
|
|
|
|
|
Res2Net
|
|
|
|
|
ResNeSt
|
|
|
|
|
ResNeXt
|
|
|
|
|
ResNet
|
|
|
|
|
ResNetV1c
|
|
|
|
|
ResNetV1d
|
|
|
|
|
ResNet_CIFAR
|
2022-12-14 11:46:39 +08:00
|
|
|
RevVisionTransformer
|
2022-04-15 20:19:20 +08:00
|
|
|
SEResNeXt
|
|
|
|
|
SEResNet
|
|
|
|
|
SVT
|
|
|
|
|
ShuffleNetV1
|
|
|
|
|
ShuffleNetV2
|
2023-06-19 11:27:50 +08:00
|
|
|
SparseResNet
|
|
|
|
|
SparseConvNeXt
|
2022-04-15 20:19:20 +08:00
|
|
|
SwinTransformer
|
2022-09-20 15:45:27 +08:00
|
|
|
SwinTransformerV2
|
2022-04-15 20:19:20 +08:00
|
|
|
T2T_ViT
|
|
|
|
|
TIMMBackbone
|
|
|
|
|
TNT
|
2022-04-28 09:35:17 +08:00
|
|
|
VAN
|
2022-04-15 20:19:20 +08:00
|
|
|
VGG
|
2023-01-17 16:55:56 +08:00
|
|
|
Vig
|
2022-04-15 20:19:20 +08:00
|
|
|
VisionTransformer
|
2023-04-13 17:03:28 +08:00
|
|
|
ViTSAM
|
2023-02-15 10:32:35 +08:00
|
|
|
XCiT
|
2023-05-06 19:28:31 +08:00
|
|
|
ViTEVA02
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.necks
|
2022-04-15 20:19:20 +08:00
|
|
|
|
|
|
|
|
Necks
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
2023-03-06 16:53:15 +08:00
|
|
|
BEiTV2Neck
|
|
|
|
|
CAENeck
|
|
|
|
|
ClsBatchNormNeck
|
|
|
|
|
DenseCLNeck
|
2022-04-15 20:19:20 +08:00
|
|
|
GeneralizedMeanPooling
|
2023-03-06 16:53:15 +08:00
|
|
|
GlobalAveragePooling
|
2022-04-15 20:19:20 +08:00
|
|
|
HRFuseScales
|
2023-03-06 16:53:15 +08:00
|
|
|
LinearNeck
|
|
|
|
|
MAEPretrainDecoder
|
|
|
|
|
MILANPretrainDecoder
|
|
|
|
|
MixMIMPretrainDecoder
|
|
|
|
|
MoCoV2Neck
|
|
|
|
|
NonLinearNeck
|
|
|
|
|
SimMIMLinearDecoder
|
|
|
|
|
SwAVNeck
|
2023-05-26 12:08:34 +08:00
|
|
|
iTPNPretrainDecoder
|
2023-06-19 11:27:50 +08:00
|
|
|
SparKLightDecoder
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.heads
|
2022-04-15 20:19:20 +08:00
|
|
|
|
|
|
|
|
Heads
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
2023-03-06 16:53:15 +08:00
|
|
|
ArcFaceClsHead
|
|
|
|
|
BEiTV1Head
|
|
|
|
|
BEiTV2Head
|
|
|
|
|
CAEHead
|
|
|
|
|
CSRAClsHead
|
2022-04-15 20:19:20 +08:00
|
|
|
ClsHead
|
|
|
|
|
ConformerHead
|
2023-03-06 16:53:15 +08:00
|
|
|
ContrastiveHead
|
|
|
|
|
DeiTClsHead
|
|
|
|
|
EfficientFormerClsHead
|
|
|
|
|
LatentCrossCorrelationHead
|
|
|
|
|
LatentPredictHead
|
|
|
|
|
LeViTClsHead
|
|
|
|
|
LinearClsHead
|
|
|
|
|
MAEPretrainHead
|
|
|
|
|
MIMHead
|
|
|
|
|
MixMIMPretrainHead
|
|
|
|
|
MoCoV3Head
|
2022-08-29 15:35:10 +08:00
|
|
|
MultiLabelClsHead
|
|
|
|
|
MultiLabelLinearClsHead
|
2023-03-06 16:53:15 +08:00
|
|
|
MultiTaskHead
|
|
|
|
|
SimMIMHead
|
|
|
|
|
StackedLinearClsHead
|
|
|
|
|
SwAVHead
|
|
|
|
|
VigClsHead
|
|
|
|
|
VisionTransformerClsHead
|
2023-05-26 12:08:34 +08:00
|
|
|
iTPNClipHead
|
2023-06-19 11:27:50 +08:00
|
|
|
SparKPretrainHead
|
2022-04-15 20:19:20 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.losses
|
2022-04-15 20:19:20 +08:00
|
|
|
|
|
|
|
|
Losses
|
|
|
|
|
------------------
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
2023-03-06 16:53:15 +08:00
|
|
|
AsymmetricLoss
|
|
|
|
|
CAELoss
|
|
|
|
|
CosineSimilarityLoss
|
|
|
|
|
CrossCorrelationLoss
|
2022-04-15 20:19:20 +08:00
|
|
|
CrossEntropyLoss
|
|
|
|
|
FocalLoss
|
2023-03-06 16:53:15 +08:00
|
|
|
LabelSmoothLoss
|
|
|
|
|
PixelReconstructionLoss
|
2022-04-15 20:19:20 +08:00
|
|
|
SeesawLoss
|
2023-03-06 16:53:15 +08:00
|
|
|
SwAVLoss
|
2022-08-29 15:35:10 +08:00
|
|
|
|
2023-03-02 13:29:07 +08:00
|
|
|
.. module:: mmpretrain.models.utils
|
2022-08-29 15:35:10 +08:00
|
|
|
|
|
|
|
|
models.utils
|
|
|
|
|
------------
|
|
|
|
|
|
|
|
|
|
This package includes some helper functions and common components used in various networks.
|
|
|
|
|
|
|
|
|
|
.. _components:
|
|
|
|
|
|
|
|
|
|
Common Components
|
|
|
|
|
^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
2023-03-06 16:53:15 +08:00
|
|
|
ConditionalPositionEncoding
|
|
|
|
|
CosineEMA
|
|
|
|
|
HybridEmbed
|
2022-08-29 15:35:10 +08:00
|
|
|
InvertedResidual
|
2023-03-06 16:53:15 +08:00
|
|
|
LayerScale
|
2022-08-29 15:35:10 +08:00
|
|
|
MultiheadAttention
|
|
|
|
|
PatchEmbed
|
|
|
|
|
PatchMerging
|
2023-03-06 16:53:15 +08:00
|
|
|
SELayer
|
|
|
|
|
ShiftWindowMSA
|
|
|
|
|
WindowMSA
|
|
|
|
|
WindowMSAV2
|
2022-08-29 15:35:10 +08:00
|
|
|
|
|
|
|
|
.. _helpers:
|
|
|
|
|
|
|
|
|
|
Helper Functions
|
|
|
|
|
^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
|
|
.. autosummary::
|
|
|
|
|
:toctree: generated
|
|
|
|
|
:nosignatures:
|
|
|
|
|
|
|
|
|
|
channel_shuffle
|
2023-03-06 16:53:15 +08:00
|
|
|
is_tracing
|
2022-08-29 15:35:10 +08:00
|
|
|
make_divisible
|
|
|
|
|
resize_pos_embed
|
|
|
|
|
resize_relative_position_bias_table
|
|
|
|
|
to_ntuple
|
