29d706248c
[Enhancement] Support training of BLIP2 ( #1700 )
...
* [Fix] Fix BEiT pre_norm
* [Enhancement] Support BLIP2 training
* [Fix] Fix quoted strings
* [Fix] Fix init_weights
* [Fix] Fix with_cls_token
* [Fix] Fix tokenizer
* [Fix] Fix quoted strings
* [Fix] Fix predict
* [Fix] Cancel changing BEiT
* [Fix] Add loading hook
* [Fix] Reformat with yapf
* [Fix] Fix prompt
* [Fix] Fix typo
2023-08-10 11:15:38 +08:00
fa53174fd9
[Feature]: Add MFF ( #1725 )
...
* [Feature]: Add MFF
* [Feature]: Add mff linear prob
* [Feature]: Add ft
* [Fix]: Update docstring
* [Feature]: Update out_indices
* [Feature]: Add prefix to ft
* [Feature]: Add README
* [Feature]: Update readme
* [Feature]: Update README
* [Feature]: Add metafile
* [Feature]: Update README
* [Fix]: Fix lint
* [Feature]: Add UT
* [Feature]: Update paper link
2023-08-08 16:01:07 +08:00
58a2243d99
Merge branch 'main' into dev
2023-07-28 15:35:55 +08:00
1f99279657
[Fix] Fix dict update in minigpt4. ( #1709 )
2023-07-28 15:30:30 +08:00
b1cd05caf2
[Enhance] Set 'is_init' in some multimodal methods ( #1718 )
...
* update is_init of multimodal
* Update minigpt4.py
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
2023-07-28 15:28:07 +08:00
e7fc25cf64
[Fix] Fix nested predict for multi-task prediction. ( #1716 )
...
* fix: multi task predict
* change the loop
---------
Co-authored-by: Pierre Colle <piercus@gmail.com>
2023-07-28 13:44:12 +08:00
4d1dbafaa2
[Enhance] Add GPU Acceleration Apple silicon mac ( #1699 )
...
* Add GPU Acceleration Apple silicon mac
* lint fix
* Update launch.py
* Use to refactor the device selection.
* Update launch.py
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-07-26 17:51:00 +08:00
64c446d507
[Feature] Support LoRA. ( #1687 )
...
* [Feature] Support LoRA
* [Feature] Support LoRA
* [Fix] Fix bugs
* [Refactor] Add copyright
* [Fix] Fix bugs
* [Enhancement] Add
* [Fix] Fix bugs
* [Fix] Fix bugs
* [Fix] Fix bugs
* [Fix] Fix bugs
* [Fix] Fix bugs
* [Docs] Update docstring
* [Docs] Update docstring
* [Refactor] Reformat with yapf
* [Docs] Update docstring
* [Refactor] Docformat
* [Refactor] Fix double-quote-string
* [Fix] fix pytorch version
* [Fix] isort
* [Fix] isort
* [Enhancement] Extend forward
* [Enhancement] Extend test
* [Fix] Fix targets
* [Enhancement] Extend LoRA to frozen models
* [Fix] Fix spelling
* [Fix] Override __getattr__
* [Fix] Add init_cfg
* [Enhancement] Add example config
* [Fix] Fix init_cfg
* [Enhancement] Add merging script
* [Fix] Remove init_cfg
* [Fix] Change lora key
* [Fix] Fix merge scripts
* [Fix] Fix merge scripts
* [Docs] Add docs
* [Fix] fix
2023-07-24 11:30:57 +08:00
60d780f99e
Fix docs
2023-07-20 10:21:15 +08:00
db395d35b1
fix_freeze_without_cls_token_vit ( #1693 )
2023-07-14 15:43:19 +08:00
465b6bdeec
[Refactor] Fix spelling ( #1689 )
2023-07-13 15:38:58 +08:00
5c43d3ef42
[Refactor] BEiT refactor ( #1705 )
...
* [Refactor] BEiT refactor
* [Fix] Fix arch zoo
* [Fix] Fix arch zoo
* [Fix] Fix freeze stages
* [Fix] Fix freeze ln2
* [Fix] Fix freezing vit ln2
2023-07-11 15:49:41 +08:00
7cbfb36c14
[Refactor] Fix spelling ( #1681 )
...
* [Refactor] Fix spelling
* [Refactor] Fix spelling
* [Refactor] Fix spelling
* [Refactor] Fix spelling
2023-07-05 11:07:43 +08:00
00030e3f7d
[Fix] refactor _prepare_pos_embed in ViT to fix bug in loading old checkpoint ( #1679 )
2023-07-03 11:36:44 +08:00
68758db7a8
[Fix] freeze pre norm in vision transformer. ( #1672 )
2023-06-28 17:00:27 +08:00
70ff2abbf7
[Refactor] Refactor _prepare_pos_embed in ViT ( #1656 )
...
* deal with cls_token
* Update implement
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-06-20 17:37:08 +08:00
7d850dfadd
[Improve] Update Otter and LLaVA docs and config. ( #1653 )
2023-06-19 20:16:13 +08:00
a1cfe888e2
[Feature] Support SparK. ( #1531 )
...
* add spark configs
* fix configs
* remove repeat aug
* add module codes
* support lr layer decay of resnet
* update
* fix lint
* add metafile and readme
* fix lint
* add models and logs
* refactor codes
* fix lint
* update model rst
* update name
* add docstring
* add ut
* fix lint
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
2023-06-19 11:27:50 +08:00
bfd49b0d52
[Feature] Support LLaVA ( #1652 )
2023-06-17 16:05:52 +08:00
e69bace03f
[Feature] Support otter ( #1651 )
...
* [Feature] Support Otter
* Update docs
2023-06-17 16:03:21 +08:00
9d3fc43073
[Feature] Support MiniGPT-4 ( #1642 )
...
* support inference of MiniGPT-4
* refine codes
* update metafile, readme and docs
* fix typo
* fix lint
* add ckpt load hook
2023-06-16 22:50:34 +08:00
53648baca5
[Fix] fix sam bug ( #1633 )
2023-06-15 10:10:51 +08:00
3eaf719a64
[Feature] Add InternImage Classification project ( #1569 )
...
* [Feature] add internimage project
* [Feature] add internimage project
* update license
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* update license
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* update internimage configs
* support internimage project
* support internimage project
* support internimage project
* internimage
2023-06-13 19:11:54 +08:00
e4c4a81b56
[Feature] Support iTPN and HiViT ( #1584 )
...
* hivit added
* Update hivit.py
* Update hivit.py
* Add files via upload
* Update __init__.py
* Add files via upload
* Update __init__.py
* Add files via upload
* Update hivit.py
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update itpn.py
* Add files via upload
* Update __init__.py
* Update mae_hivit-base-p16.py
* Delete mim_itpn-base-p16.py
* Add files via upload
* Update itpn_hivit-base-p16.py
* Update itpn.py
* Update hivit.py
* Update __init__.py
* Update mae.py
* Delete hivit.py
* Update __init__.py
* Delete configs/itpn directory
* Add files via upload
* Add files via upload
* Delete configs/hivit directory
* Add files via upload
* refactor and add metafile and readme
* update clip
* add ut
* update ut
* update
* update docstring
* update model.rst
---------
Co-authored-by: 田运杰 <48153283+sunsmarterjie@users.noreply.github.com>
2023-05-26 12:08:34 +08:00
023d6869bd
[Fix] Incorrect stage freeze on RIFormer Model ( #1573 )
...
* [Doc] RIFormer's README did not link to its paper properly
* Incorrect code for reproducing RIFormer
the default value of frozen stage is set to 0, and the doc says that this will lead to no stage be frozen. But the actual case is the patch_embed will be freezed.
This may cause incorrect training, thus influencing the result.
I suggest a careful review.
2023-05-22 16:01:32 +08:00
1e478462b8
[Feature] Support Chinese CLIP. ( #1576 )
...
* support cn-clip
* update README
* Update progress bar
* update order of category
* fix lint
* update
* update readme and metafile
* update
* update docstring
* refactor tokenizer
* fix lint
* Update README and progress bar
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-05-22 15:46:13 +08:00
6847d20d57
[Feature] Support multiple multi-modal algorithms and inferencers. ( #1561 )
...
* [Feat] Migrate blip caption to mmpretrain. (#50 )
* Migrate blip caption to mmpretrain
* minor fix
* support train
* [Feature] Support OFA caption task. (#51 )
* [Feature] Support OFA caption task.
* Remove duplicated files.
* [Feature] Support OFA vqa task. (#58 )
* [Feature] Support OFA vqa task.
* Fix lint.
* [Feat] Add BLIP retrieval to mmpretrain. (#55 )
* init
* minor fix for train
* fix according to comments
* refactor
* Update Blip retrieval. (#62 )
* [Feature] Support OFA visual grounding task. (#59 )
* [Feature] Support OFA visual grounding task.
* minor add TODO
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Add flamingos coco caption and vqa. (#60 )
* first init
* init flamingo coco
* add vqa
* minor fix
* remove unnecessary modules
* Update config
* Use `ApplyToList`.
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 coco retrieval (#53 )
* [Feature]: Add blip2 retriever
* [Feature]: Add blip2 all modules
* [Feature]: Refine model
* [Feature]: x1
* [Feature]: Runnable coco ret
* [Feature]: Runnable version
* [Feature]: Fix lint
* [Fix]: Fix lint
* [Feature]: Use 364 img size
* [Feature]: Refactor blip2
* [Fix]: Fix lint
* refactor files
* minor fix
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Remove
* fix blip caption inputs (#68 )
* [Feat] Add BLIP NLVR support. (#67 )
* first init
* init flamingo coco
* add vqa
* add nlvr
* refactor nlvr
* minor fix
* minor fix
* Update dataset
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 Caption (#70 )
* [Feature]: Add language model
* [Feature]: blip2 caption forward
* [Feature]: Reproduce the results
* [Feature]: Refactor caption
* refine config
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Migrate BLIP VQA to mmpretrain (#69 )
* reformat
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* refactor code
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Update RefCOCO dataset
* [Fix] fix lint
* [Feature] Implement inference APIs for multi-modal tasks. (#65 )
* [Feature] Implement inference APIs for multi-modal tasks.
* [Project] Add gradio demo.
* [Improve] Update requirements
* Update flamingo
* Update blip
* Add NLVR inferencer
* Update flamingo
* Update hugging face model register
* Update ofa vqa
* Update BLIP-vqa (#71 )
* Update blip-vqa docstring (#72 )
* Refine flamingo docstring (#73 )
* [Feature]: BLIP2 VQA (#61 )
* [Feature]: VQA forward
* [Feature]: Reproduce accuracy
* [Fix]: Fix lint
* [Fix]: Add blank line
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feature]: BLIP2 docstring (#74 )
* [Feature]: Add caption docstring
* [Feature]: Add docstring to blip2 vqa
* [Feature]: Add docstring to retrieval
* Update BLIP-2 metafile and README (#75 )
* [Feature]: Add readme and docstring
* Update blip2 results
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature] BLIP Visual Grounding on MMPretrain Branch (#66 )
* blip grounding merge with mmpretrain
* remove commit
* blip grounding test and inference api
* refcoco dataset
* refcoco dataset refine config
* rebasing
* gitignore
* rebasing
* minor edit
* minor edit
* Update blip-vqa docstring (#72 )
* rebasing
* Revert "minor edit"
This reverts commit 639cec757c215e654625ed0979319e60f0be9044.
* blip grounding final
* precommit
* refine config
* refine config
* Update blip visual grounding
---------
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: mzr1996 <mzr1996@163.com>
* Update visual grounding metric
* Update OFA docstring, README and metafiles. (#76 )
* [Docs] Update installation docs and gradio demo docs. (#77 )
* Update OFA name
* Update Visual Grounding Visualizer
* Integrate accelerate support
* Fix imports.
* Fix timm backbone
* Update imports
* Update README
* Update circle ci
* Update flamingo config
* Add gradio demo README
* [Feature]: Add scienceqa (#1571 )
* [Feature]: Add scienceqa
* [Feature]: Change param name
* Update docs
* Update video
---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
Co-authored-by: yingfhu <yingfhu@gmail.com>
Co-authored-by: Yuan Liu <30762564+YuanLiuuuuuu@users.noreply.github.com>
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: Rongjie Li <limo97@163.com>
2023-05-19 16:50:04 +08:00
770eb8e24a
[Fix] Fix ddp bugs caused by `out_type`. ( #1570 )
...
* set out_type to be 'raw'
* update test
2023-05-17 17:32:10 +08:00
034919d032
[Feature] add eva02 backbone ( #1450 )
...
* [CI] Add test mim CI. (#879 )
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* update readme and configs
* refactore eva02
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* refactore eva02
* update readme and metafile
* update readme and metafile
* update readme and metafile
* update
* rename eva02
* rename eva02
* fix uts
* rename configs
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-06 19:28:31 +08:00
7f4eccbecf
[Fix] Fix multi-task-head loss potential bug ( #1530 )
...
* fix bug
* add comments
2023-05-06 18:04:34 +08:00
afa60c73bb
[Fix] Support bce loss without batch augmentations ( #1525 )
...
* Support bce loss without batch augmentations
---------
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-05 17:19:42 +08:00
d9e561a09d
[Feature] Support dinov2 backbone ( #1522 )
...
* support dinov2 backbone
* update metafile and readme
* compatible to use_layer_scale
* update SwiGLUFFN
* add deprecation warning
* update
2023-05-05 16:59:37 +08:00
15cc2a5193
[Fix] Fix clip generator init bug ( #1518 )
2023-04-25 19:35:09 +08:00
e954cf0aaf
[Fix] Fix the bug in binary cross entropy loss ( #1499 )
...
* [Fix] Fix the bug in binary cross entropy loss
Fix the bug in binary cross entropy loss when using multi-label datasets e.g.VOC2007
* update ci
---------
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-04-19 13:53:31 +08:00
2c913020b9
[Refactor] Support to freeze channel reduction and add layer decay function ( #1490 )
...
* support to freeze channel reduction module
* add layer decay setting function
2023-04-17 13:36:47 +08:00
e93d124ad4
[Refactor] Support resizing pos_embed while loading ckpt and format output ( #1488 )
...
* support resize pos_embed while loading ckpt
* update
2023-04-14 19:08:35 +08:00
99e48116aa
[Feature] Register torchvision transforms into mmcls ( #1265 )
...
* [Enhance] Add stochastic depth decay rule in resnet. (#1363 )
* add stochastic depth decay rule to drop path rate
* add default value
* update
* pass ut
* update
* pass ut
* remove np
* rebase
* update ToPIL and ToNumpy
* rebase
* rebase
* rebase
* rebase
* add readme
* fix review suggestions
* rebase
* fix conflicts
* fix conflicts
* fix lint
* remove comments
* remove useless code
* update docstring
* update doc API
* update doc
---------
Co-authored-by: Yixiao Fang <36138628+fangyixiao18@users.noreply.github.com>
2023-04-13 18:05:57 +08:00
0826df8963
[Feature] Add ViT of SAM ( #1476 )
...
* add vit of sam
* update
* update
* add ut
* update ut
* remove num_classes
* support dynamic input
* add ut
* add comments
* update ut
2023-04-13 17:03:28 +08:00
05124dbb71
fix lint
2023-04-06 22:01:11 +08:00
b8cab5c9f7
update readme
2023-04-06 21:56:25 +08:00
3932ddec10
update ckpt path
2023-04-06 21:56:25 +08:00
a6c24d104e
update riformer mmpretrain
2023-04-06 21:56:25 +08:00
32c258ff19
update riformer mmpretrain
2023-04-06 21:56:25 +08:00
0b70c108b0
update riformer mmpretrain
2023-04-06 21:56:25 +08:00
1ee9bbe050
[Docs] Update links ( #1457 )
...
* update links
* update readtherdocs
* update
* update
* fix lint
* update
* update
* update
* update cov branch
* update
* update
* update
2023-04-06 20:58:52 +08:00
75dceaa78f
[Refactor] Add ln to vit avg_featmap output ( #1447 )
2023-04-06 11:59:39 +08:00
b017670e1b
[Improve] Use PyTorch official `scaled_dot_product_attention` to accelerate `MultiheadAttention`. ( #1434 )
...
* [Improve] Use PyTorch official `scaled_dot_product_attention` to accelerate `MultiheadAttention`.
* Support `--local-rank` and `--amp` option for new version PyTorch.
* Fix imports and UT.
2023-03-29 15:50:44 +08:00
c4ccae40db
[Docs] Update user guides docs and tools for MMPretrain. ( #1429 )
...
* [Docs] Update user guides docs and tools for MMPretrain.
* Fix UT
* Fix Chinese docs.
* Improve according to comments.
* Fix windows CI.
2023-03-27 14:32:26 +08:00
dbf3df21a3
[Refactor] Use `out_type` to specify ViT-like backbone output. ( #1408 )
...
* [Refactor] Use to specify ViT-like backbone output.
* Fix ClsBatchNormNeck
* Update mmpretrain/models/necks/mae_neck.py
---------
Co-authored-by: Yixiao Fang <36138628+fangyixiao18@users.noreply.github.com>
2023-03-09 11:02:58 +08:00
274a67223e
[Feature] Implement layer-wise learning rate decay optimizer constructor. ( #1399 )
...
* [Feature] Implement layer-wise learning rate decay optimizer constructor.
* Use num_layers instead of max_depth to avoid misleading
* Add UT
* Update docstring
* Update log info
* update LearningRateDecay configs
---------
Co-authored-by: fangyixiao18 <fangyx18@hotmail.com>
2023-03-07 17:30:39 +08:00
fanqiNO1
Yuan Liu
fangyixiao18
Yixiao Fang
marouane amzil
Nripesh Niketan
mzr1996
Fabien Merceron PRL
Peng Lu
Wangbo Zhao(黑色枷锁)
zzc98
ZhangYiqin
Ezra-Yu
Kei-Chi Tse
techmonsterwang