mmpretrain/configs/vision_transformer/vit-large-p16_pt-64xb64_in1k-224.py

_base_ = [
    '../_base_/models/vit-large-p16.py',
    '../_base_/datasets/imagenet_bs64_pil_resize_autoaug.py',
    '../_base_/schedules/imagenet_bs4096_AdamW.py',
    '../_base_/default_runtime.py'
]

# model setting
model = dict(
    head=dict(hidden_dim=3072),
    train_cfg=dict(
        augments=dict(
            type='BatchMixup',
            alpha=0.2,
            num_classes=1000,
            prob=1.,
        )))

# schedule setting
optim_wrapper = dict(clip_grad=dict(max_norm=1.0))
[Refactor] Refator ViT (Continue #295) (#395) * [Squash] Refator ViT (from #295) * Use base variable to simplify auto_aug setting * Use common PatchEmbed, remove HybridEmbed and refactor ViT init structure. * Add `output_cls_token` option and change the output format of ViT and input format of ViT head. * Update unit tests and add test for `output_cls_token`. * Support out_indices. * Standardize config files * Support resize position embedding. * Add readme file of vit * Rename config file * Improve docs about ViT. * Update docstring * Use local version `MultiheadAttention` instead of mmcv version. * Fix MultiheadAttention * Support `qk_scale` argument in `MultiheadAttention` * Improve docs and change `layer_cfg` to `layer_cfgs` and support sequence. * Use init_cfg to init Linear layer in VisionTransformerHead * update metafile * Update checkpoints and configs * Imporve docstring. * Update README * Revert GAP modification. 2021-10-18 16:07:00 +08:00			`_base_ = [`
			`'../_base_/models/vit-large-p16.py',`
			`'../_base_/datasets/imagenet_bs64_pil_resize_autoaug.py',`
			`'../_base_/schedules/imagenet_bs4096_AdamW.py',`
			`'../_base_/default_runtime.py'`
			`]`

[Refactor] Add `ResizeEdge` and refactor all dataset configs. 2022-06-01 06:11:53 +00:00			`# model setting`
[Refactor] Refator ViT (Continue #295) (#395) * [Squash] Refator ViT (from #295) * Use base variable to simplify auto_aug setting * Use common PatchEmbed, remove HybridEmbed and refactor ViT init structure. * Add `output_cls_token` option and change the output format of ViT and input format of ViT head. * Update unit tests and add test for `output_cls_token`. * Support out_indices. * Standardize config files * Support resize position embedding. * Add readme file of vit * Rename config file * Improve docs about ViT. * Update docstring * Use local version `MultiheadAttention` instead of mmcv version. * Fix MultiheadAttention * Support `qk_scale` argument in `MultiheadAttention` * Improve docs and change `layer_cfg` to `layer_cfgs` and support sequence. * Use init_cfg to init Linear layer in VisionTransformerHead * update metafile * Update checkpoints and configs * Imporve docstring. * Update README * Revert GAP modification. 2021-10-18 16:07:00 +08:00			`model = dict(`
			`head=dict(hidden_dim=3072),`
			`train_cfg=dict(`
[Refactor] Add `ResizeEdge` and refactor all dataset configs. 2022-06-01 06:11:53 +00:00			`augments=dict(`
			`type='BatchMixup',`
			`alpha=0.2,`
			`num_classes=1000,`
			`prob=1.,`
			`)))`

[Refactor] Update optimizer related registries and configs. 2022-06-02 09:11:09 +00:00			`# schedule setting`
			`optim_wrapper = dict(clip_grad=dict(max_norm=1.0))`