a1cfe888e2
[Feature] Support SparK. ( #1531 )
...
* add spark configs
* fix configs
* remove repeat aug
* add module codes
* support lr layer decay of resnet
* update
* fix lint
* add metafile and readme
* fix lint
* add models and logs
* refactor codes
* fix lint
* update model rst
* update name
* add docstring
* add ut
* fix lint
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
2023-06-19 11:27:50 +08:00
1e478462b8
[Feature] Support Chinese CLIP. ( #1576 )
...
* support cn-clip
* update README
* Update progress bar
* update order of category
* fix lint
* update
* update readme and metafile
* update
* update docstring
* refactor tokenizer
* fix lint
* Update README and progress bar
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-05-22 15:46:13 +08:00
6847d20d57
[Feature] Support multiple multi-modal algorithms and inferencers. ( #1561 )
...
* [Feat] Migrate blip caption to mmpretrain. (#50 )
* Migrate blip caption to mmpretrain
* minor fix
* support train
* [Feature] Support OFA caption task. (#51 )
* [Feature] Support OFA caption task.
* Remove duplicated files.
* [Feature] Support OFA vqa task. (#58 )
* [Feature] Support OFA vqa task.
* Fix lint.
* [Feat] Add BLIP retrieval to mmpretrain. (#55 )
* init
* minor fix for train
* fix according to comments
* refactor
* Update Blip retrieval. (#62 )
* [Feature] Support OFA visual grounding task. (#59 )
* [Feature] Support OFA visual grounding task.
* minor add TODO
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Add flamingos coco caption and vqa. (#60 )
* first init
* init flamingo coco
* add vqa
* minor fix
* remove unnecessary modules
* Update config
* Use `ApplyToList`.
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 coco retrieval (#53 )
* [Feature]: Add blip2 retriever
* [Feature]: Add blip2 all modules
* [Feature]: Refine model
* [Feature]: x1
* [Feature]: Runnable coco ret
* [Feature]: Runnable version
* [Feature]: Fix lint
* [Fix]: Fix lint
* [Feature]: Use 364 img size
* [Feature]: Refactor blip2
* [Fix]: Fix lint
* refactor files
* minor fix
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Remove
* fix blip caption inputs (#68 )
* [Feat] Add BLIP NLVR support. (#67 )
* first init
* init flamingo coco
* add vqa
* add nlvr
* refactor nlvr
* minor fix
* minor fix
* Update dataset
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 Caption (#70 )
* [Feature]: Add language model
* [Feature]: blip2 caption forward
* [Feature]: Reproduce the results
* [Feature]: Refactor caption
* refine config
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Migrate BLIP VQA to mmpretrain (#69 )
* reformat
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* refactor code
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Update RefCOCO dataset
* [Fix] fix lint
* [Feature] Implement inference APIs for multi-modal tasks. (#65 )
* [Feature] Implement inference APIs for multi-modal tasks.
* [Project] Add gradio demo.
* [Improve] Update requirements
* Update flamingo
* Update blip
* Add NLVR inferencer
* Update flamingo
* Update hugging face model register
* Update ofa vqa
* Update BLIP-vqa (#71 )
* Update blip-vqa docstring (#72 )
* Refine flamingo docstring (#73 )
* [Feature]: BLIP2 VQA (#61 )
* [Feature]: VQA forward
* [Feature]: Reproduce accuracy
* [Fix]: Fix lint
* [Fix]: Add blank line
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feature]: BLIP2 docstring (#74 )
* [Feature]: Add caption docstring
* [Feature]: Add docstring to blip2 vqa
* [Feature]: Add docstring to retrieval
* Update BLIP-2 metafile and README (#75 )
* [Feature]: Add readme and docstring
* Update blip2 results
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature] BLIP Visual Grounding on MMPretrain Branch (#66 )
* blip grounding merge with mmpretrain
* remove commit
* blip grounding test and inference api
* refcoco dataset
* refcoco dataset refine config
* rebasing
* gitignore
* rebasing
* minor edit
* minor edit
* Update blip-vqa docstring (#72 )
* rebasing
* Revert "minor edit"
This reverts commit 639cec757c215e654625ed0979319e60f0be9044.
* blip grounding final
* precommit
* refine config
* refine config
* Update blip visual grounding
---------
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: mzr1996 <mzr1996@163.com>
* Update visual grounding metric
* Update OFA docstring, README and metafiles. (#76 )
* [Docs] Update installation docs and gradio demo docs. (#77 )
* Update OFA name
* Update Visual Grounding Visualizer
* Integrate accelerate support
* Fix imports.
* Fix timm backbone
* Update imports
* Update README
* Update circle ci
* Update flamingo config
* Add gradio demo README
* [Feature]: Add scienceqa (#1571 )
* [Feature]: Add scienceqa
* [Feature]: Change param name
* Update docs
* Update video
---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
Co-authored-by: yingfhu <yingfhu@gmail.com>
Co-authored-by: Yuan Liu <30762564+YuanLiuuuuuu@users.noreply.github.com>
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: Rongjie Li <limo97@163.com>
2023-05-19 16:50:04 +08:00
034919d032
[Feature] add eva02 backbone ( #1450 )
...
* [CI] Add test mim CI. (#879 )
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* update readme and configs
* refactore eva02
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* refactore eva02
* update readme and metafile
* update readme and metafile
* update readme and metafile
* update
* rename eva02
* rename eva02
* fix uts
* rename configs
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-06 19:28:31 +08:00
d9e561a09d
[Feature] Support dinov2 backbone ( #1522 )
...
* support dinov2 backbone
* update metafile and readme
* compatible to use_layer_scale
* update SwiGLUFFN
* add deprecation warning
* update
2023-05-05 16:59:37 +08:00
1ee9bbe050
[Docs] Update links ( #1457 )
...
* update links
* update readtherdocs
* update
* update
* fix lint
* update
* update
* update
* update cov branch
* update
* update
* update
2023-04-06 20:58:52 +08:00
b017670e1b
[Improve] Use PyTorch official `scaled_dot_product_attention` to accelerate `MultiheadAttention`. ( #1434 )
...
* [Improve] Use PyTorch official `scaled_dot_product_attention` to accelerate `MultiheadAttention`.
* Support `--local-rank` and `--amp` option for new version PyTorch.
* Fix imports and UT.
2023-03-29 15:50:44 +08:00
c4ccae40db
[Docs] Update user guides docs and tools for MMPretrain. ( #1429 )
...
* [Docs] Update user guides docs and tools for MMPretrain.
* Fix UT
* Fix Chinese docs.
* Improve according to comments.
* Fix windows CI.
2023-03-27 14:32:26 +08:00
dbf3df21a3
[Refactor] Use `out_type` to specify ViT-like backbone output. ( #1408 )
...
* [Refactor] Use to specify ViT-like backbone output.
* Fix ClsBatchNormNeck
* Update mmpretrain/models/necks/mae_neck.py
---------
Co-authored-by: Yixiao Fang <36138628+fangyixiao18@users.noreply.github.com>
2023-03-09 11:02:58 +08:00
08dc8c75d3
[Refactor] Add selfsup algorithms. ( #1389 )
...
* remove basehead
* add moco series
* add byol simclr simsiam
* add ut
* update configs
* add simsiam hook
* add and refactor beit
* update ut
* add cae
* update extract_feat
* refactor cae
* add mae
* refactor data preprocessor
* update heads
* add maskfeat
* add milan
* add simmim
* add mixmim
* fix lint
* fix ut
* fix lint
* add eva
* add densecl
* add barlowtwins
* add swav
* fix lint
* update readtherdocs rst
* update docs
* update
* Decrease UT memory usage
* Fix docstring
* update DALLEEncoder
* Update model docs
* refactor dalle encoder
* update docstring
* fix ut
* fix config error
* add val_cfg and test_cfg
* refactor clip generator
* fix lint
* pass check
* fix ut
* add lars
* update type of BEiT in configs
* Use MMEngine style momentum in EMA.
* apply mmpretrain solarize
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-03-06 16:53:15 +08:00
c9670173aa
[Refactor] Move and refactor utils from mmselfsup. ( #1385 )
...
* add heads
* add losses
* fix
* remove mim head
* add modified backbones and target generators
* fix lint
* fix lint
* add heads
* add losses
* fix
* add data preprocessor from mmselfsup
* add ut for data prepocessor
* add GatherLayer
* add ema
* add batch shuffle
* add misc
* fix lint
* update
* update docstring
2023-02-28 17:04:40 +08:00
414ba80274
[Refactor] Refactor APIs, add `ImageRetrievalInferencer` and `FeatureExtractor`. ( #1380 )
...
* [Refactor] Refactor APIs, add `ImageRetrievalInferencer` and `FeatureExtractor'.
* Update image retrieval
* Update FeatureExtractor
* Fix UT
2023-02-28 16:31:42 +08:00
e453a45d31
[Refactor] Add self-supervised backbones and target generators. ( #1379 )
...
* add heads
* add losses
* fix
* remove mim head
* add modified backbones and target generators
* add unittest
* refactor caevit
* add window_size check
* fix lint
* apply new DataSample
* fix ut error
* update ut
* fix ut
* fix lint
* Update base modules.
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-02-28 15:59:17 +08:00
63d9f27fde
[Refactor] Add necks, heads and losses for the self-supervised task. ( #1376 )
...
* add necks
* refactor linear neck
* rename simmim neck
* add heads
* add losses
* fix
* add unittest
* update
* update cae
* remove mim head
* update config
2023-02-28 10:05:00 +08:00
36bea13fca
[Refactor] Refactor ClsDatasample to a union DataSample. ( #1371 )
...
* [Refactor] Refactor ClsDatasample to a union DataSample.
* Add method
* Fix docstring
* Update docstring.
2023-02-23 10:07:53 +08:00
0979e78573
Rename the package name to `mmpretrain`.
2023-02-17 15:20:55 +08:00
Yixiao Fang
Ma Zerun
zzc98