59c077746f
[Feat] Download dataset by using MIM&OpenDataLab ( #1630 )
...
* add dataset.index
* update preprocess shell
* update shell
* update docs
* update docs
2023-06-30 13:55:13 +08:00
8afad77a35
[Enhance] Update fsdp vit-huge and vit-large config ( #1675 )
...
* Update fsdp vit-huge and vit-large config
* Update fsdp vit-huge and vit-large config
* rename
2023-06-30 11:15:18 +08:00
658db80089
[Enhancement] Support deepspeed with flexible runner ( #1673 )
...
* [Feature] Support deepspeed with flexible runner
* [Fix] Reformat with yapf
* [Refacor] Rename configs
* [Fix] Reformat with yapf
* [Refactor] Remove unused keys
* [Refactor] Change the _base_ path
* [Refactor] Reformat
2023-06-29 10:16:27 +08:00
68758db7a8
[Fix] freeze pre norm in vision transformer. ( #1672 )
2023-06-28 17:00:27 +08:00
10685fc81c
[Refactor] Replace if '_base_' with read_base(). ( #1665 )
2023-06-28 16:57:18 +08:00
70ff2abbf7
[Refactor] Refactor _prepare_pos_embed in ViT ( #1656 )
...
* deal with cls_token
* Update implement
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-06-20 17:37:08 +08:00
d4a6dfa00a
Add benchmark options ( #1654 )
...
* update dev_scripts
* update metafile
* update multimodal floating range
* fix lint
* update
* update
* fix lint
* Update metric map
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-06-20 14:18:57 +08:00
7d850dfadd
[Improve] Update Otter and LLaVA docs and config. ( #1653 )
2023-06-19 20:16:13 +08:00
dbef2b41c6
[Fix] Align COCO dataset format.
2023-06-19 07:24:07 +00:00
d6056af2b8
[Fix][New_config] Fix demo bug ( #1647 )
...
* Fix demo
* Update implement
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-06-19 15:15:28 +08:00
6d7fe91a98
[Feature] Support Flickr30k Retrieval dataset ( #1625 )
...
* format
* remove abs path
* init add flickr30k caption
* remove abs dir
* update blip readme
* add convert sscripts
* minor
* minor
2023-06-19 15:15:03 +08:00
a1cfe888e2
[Feature] Support SparK. ( #1531 )
...
* add spark configs
* fix configs
* remove repeat aug
* add module codes
* support lr layer decay of resnet
* update
* fix lint
* add metafile and readme
* fix lint
* add models and logs
* refactor codes
* fix lint
* update model rst
* update name
* add docstring
* add ut
* fix lint
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
2023-06-19 11:27:50 +08:00
bfd49b0d52
[Feature] Support LLaVA ( #1652 )
2023-06-17 16:05:52 +08:00
e69bace03f
[Feature] Support otter ( #1651 )
...
* [Feature] Support Otter
* Update docs
2023-06-17 16:03:21 +08:00
9d3fc43073
[Feature] Support MiniGPT-4 ( #1642 )
...
* support inference of MiniGPT-4
* refine codes
* update metafile, readme and docs
* fix typo
* fix lint
* add ckpt load hook
2023-06-16 22:50:34 +08:00
a673b048a5
[Feature] Add support for VizWiz dataset. ( #1636 )
...
* add vizwiz
* update dataset
* [Fix] Build img_path in data_sample.
* Fix isort.
---------
Co-authored-by: ZhangYuanhan-AI <yuanhan002@ntu.edu.sg>
2023-06-16 17:16:17 +08:00
aac398a83f
[Feature] Support new configs. ( #1639 )
...
* [Feature] Support new configs (#1638 )
* add new config of mae and simclr
* update
* update setup.cfg
* update eva
* update
* update new config
* Add new config
* remove __init__.py
* 1. remove ; 2. remove mmpretrain/configs/_base_/models/convnext
* remove model_wrapper_cfg and add out type
* Add comment for setting default_scope to NOne
* update if '_base_' order
* update
* revert changes
---------
Co-authored-by: fangyixiao18 <fangyx18@hotmail.com>
* Add warn at the head of new config files
---------
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-06-16 16:54:45 +08:00
93e0f107c4
[Fix] Fix bug loading IN1k dataset. ( #1641 )
2023-06-16 15:35:27 +08:00
7581b76233
[Feature] Add support for vsr dataset ( #1634 )
...
* add VSR dataset
* [Fix] Modify example and load gt_answer as string.
---------
Co-authored-by: ZhangYuanhan-AI <yuanhan002@ntu.edu.sg>
2023-06-15 19:17:02 +08:00
53648baca5
[Fix] fix sam bug ( #1633 )
2023-06-15 10:10:51 +08:00
3eaf719a64
[Feature] Add InternImage Classification project ( #1569 )
...
* [Feature] add internimage project
* [Feature] add internimage project
* update license
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* update license
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* [Feature] add internimage project
* update internimage configs
* support internimage project
* support internimage project
* support internimage project
* internimage
2023-06-13 19:11:54 +08:00
8e9e880601
[Feat] Add download link for coco caption and retrieval annotations. ( #1607 )
...
* [Feat] Add download link for coco caption and retrieval annotations.
* minor fix
2023-06-13 10:29:54 +08:00
bb415b91be
[Feature] Support OCR-VQA dataset ( #1621 )
...
* support ocrvqa dataset
* minor
* remove abs path
* refine README
2023-06-13 10:28:45 +08:00
dbfb84ccbd
[Feature] Support OK-VQA dataset ( #1615 )
...
* add okvqa
* refine README
2023-06-08 16:57:18 +08:00
057d7c6d6a
[BUG] Fixed circular import error for new transform ( #1609 )
2023-06-08 14:00:41 +08:00
bddbc085fc
[Feature]: Add image_only param ( #1613 )
...
* [Feature]: Add image_only param
* [Feature]: Use image_only
2023-06-06 12:50:42 +08:00
3a277ee9e6
[Feature] support TextVQA dataset ( #1596 )
...
* [Support] Suport TextVQA dataset
* add folder structure
* fix readme
2023-06-02 11:50:38 +08:00
bc3c4a35ee
[Refactor] Support to use "split" to specify training set/validation set in the ImageNet dataset ( #1535 )
...
* [Feature]: Add caption
* [Feature]: Update scienceqa
* [CI] Add test mim CI. (#879 )
* refactor imagenet dataset
* refactor imagenet dataset
* refactor imagenet dataset
* update imagenet21k
* update configs
* update mnist
* update dataset_prepare.md
* fix sun397 url and update user_guides/dataset_prepare.md
* update dataset_prepare.md
* fix sun397 dataset
* fix sun397
* update chinese dataset_prepare.md
* update dataset_prepare.md
* [Refactor] update voc dataset
* [Refactor] update voc dataset
* refactor imagenet
* refactor imagenet
* use mmengine.fileio
---------
Co-authored-by: liuyuan <3463423099@qq.com>
Co-authored-by: Ma Zerun <mzr1996@163.com>
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-06-02 11:03:18 +08:00
795607cfeb
[Docs] Add t-SNE visualization doc ( #1555 )
...
* 2023-05-08 add t-sne docs
* 2023-05-08 add t-sne docs
* 2023-05-10 add t-sne docs CN
* 2023-05-25 rebase dev
* add docs for running t-sne on mae models, and fix a bug in vis_tsne.py
* rewrite t-sne docs to correct some mistakes
2023-06-01 10:04:06 +08:00
5bd088ef43
[Fix] Update torchvision transform wrapper ( #1595 )
...
* Update torchvision transform wrapper
* Update requirements
* fix unit tests
---------
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-26 17:56:09 +08:00
e4c4a81b56
[Feature] Support iTPN and HiViT ( #1584 )
...
* hivit added
* Update hivit.py
* Update hivit.py
* Add files via upload
* Update __init__.py
* Add files via upload
* Update __init__.py
* Add files via upload
* Update hivit.py
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update itpn.py
* Add files via upload
* Update __init__.py
* Update mae_hivit-base-p16.py
* Delete mim_itpn-base-p16.py
* Add files via upload
* Update itpn_hivit-base-p16.py
* Update itpn.py
* Update hivit.py
* Update __init__.py
* Update mae.py
* Delete hivit.py
* Update __init__.py
* Delete configs/itpn directory
* Add files via upload
* Add files via upload
* Delete configs/hivit directory
* Add files via upload
* refactor and add metafile and readme
* update clip
* add ut
* update ut
* update
* update docstring
* update model.rst
---------
Co-authored-by: 田运杰 <48153283+sunsmarterjie@users.noreply.github.com>
2023-05-26 12:08:34 +08:00
1f07c92ed1
[Feature] Add retrieval mAP metric. ( #1552 )
...
* rebase
* fefine
* fix lint
* update readme
* rebase
* fix lint
* update docstring
* update docstring
* rebase
* rename corespanding names
* rebase
2023-05-26 10:40:08 +08:00
9bb692e440
[Fix] Set default out_type in CAM visualization. ( #1586 )
2023-05-24 14:09:41 +08:00
a779c8c5a7
[Feature] Support NoCap dataset based on BLIP. ( #1582 )
...
* [Feature] Support nocaps dataset
* precommit
* Use official coco format
* add nocp readme
* fix readme
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-05-23 18:06:43 +08:00
46a523ef63
[Feature] Add GQA dataset. ( #1585 )
...
* [Feature]: Add GQA dataset
* [Feature]: Add GQA
* [Feature]: Add GQA UT
* [Fix]: Fix hint
* [Feature]: Add BLIP2 GQA
* [Fix]: Fix lint
* [Feature]: Update anno link
* [Fix]: Update docstring
* [Feature]: Update all links
2023-05-23 11:25:42 +08:00
4dd8a86145
Bump version to v1.0.0rc8 ( #1583 )
...
* Bump version to v1.0.0rc8
* Apply suggestions from code review
Co-authored-by: Yixiao Fang <36138628+fangyixiao18@users.noreply.github.com>
* Update README.md
---------
Co-authored-by: Yixiao Fang <36138628+fangyixiao18@users.noreply.github.com>
2023-05-23 11:22:51 +08:00
be389eb846
[Fix] Fix scienceqa ( #1581 )
2023-05-22 16:10:17 +08:00
023d6869bd
[Fix] Incorrect stage freeze on RIFormer Model ( #1573 )
...
* [Doc] RIFormer's README did not link to its paper properly
* Incorrect code for reproducing RIFormer
the default value of frozen stage is set to 0, and the doc says that this will lead to no stage be frozen. But the actual case is the patch_embed will be freezed.
This may cause incorrect training, thus influencing the result.
I suggest a careful review.
2023-05-22 16:01:32 +08:00
b058912c0c
[Docs] Fix example_project README ( #1575 )
2023-05-22 15:47:03 +08:00
1e478462b8
[Feature] Support Chinese CLIP. ( #1576 )
...
* support cn-clip
* update README
* Update progress bar
* update order of category
* fix lint
* update
* update readme and metafile
* update
* update docstring
* refactor tokenizer
* fix lint
* Update README and progress bar
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
2023-05-22 15:46:13 +08:00
d04ef8a29e
Merge pull request #1577 from YuanLiuuuuuu/scienceqa_metrics
...
[Feature]: Add ScienceQA Metrics
2023-05-22 13:08:06 +08:00
74f24658e7
[Fix]: Delete GQA
2023-05-22 11:57:18 +08:00
13e4d6c512
[Fix]: Fix UT
2023-05-22 11:55:08 +08:00
b0ad99afb9
[Fix]: Fix bug
2023-05-22 11:38:34 +08:00
1537d46596
[Feature]: Update scienceqa
2023-05-22 11:31:07 +08:00
87f849cbb6
[Feature]: Add scienceqa metric
2023-05-22 11:31:07 +08:00
1b8e86dca6
[Feature]: Add caption
2023-05-22 11:31:07 +08:00
6847d20d57
[Feature] Support multiple multi-modal algorithms and inferencers. ( #1561 )
...
* [Feat] Migrate blip caption to mmpretrain. (#50 )
* Migrate blip caption to mmpretrain
* minor fix
* support train
* [Feature] Support OFA caption task. (#51 )
* [Feature] Support OFA caption task.
* Remove duplicated files.
* [Feature] Support OFA vqa task. (#58 )
* [Feature] Support OFA vqa task.
* Fix lint.
* [Feat] Add BLIP retrieval to mmpretrain. (#55 )
* init
* minor fix for train
* fix according to comments
* refactor
* Update Blip retrieval. (#62 )
* [Feature] Support OFA visual grounding task. (#59 )
* [Feature] Support OFA visual grounding task.
* minor add TODO
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Add flamingos coco caption and vqa. (#60 )
* first init
* init flamingo coco
* add vqa
* minor fix
* remove unnecessary modules
* Update config
* Use `ApplyToList`.
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 coco retrieval (#53 )
* [Feature]: Add blip2 retriever
* [Feature]: Add blip2 all modules
* [Feature]: Refine model
* [Feature]: x1
* [Feature]: Runnable coco ret
* [Feature]: Runnable version
* [Feature]: Fix lint
* [Fix]: Fix lint
* [Feature]: Use 364 img size
* [Feature]: Refactor blip2
* [Fix]: Fix lint
* refactor files
* minor fix
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Remove
* fix blip caption inputs (#68 )
* [Feat] Add BLIP NLVR support. (#67 )
* first init
* init flamingo coco
* add vqa
* add nlvr
* refactor nlvr
* minor fix
* minor fix
* Update dataset
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature]: BLIP2 Caption (#70 )
* [Feature]: Add language model
* [Feature]: blip2 caption forward
* [Feature]: Reproduce the results
* [Feature]: Refactor caption
* refine config
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feat] Migrate BLIP VQA to mmpretrain (#69 )
* reformat
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* change
* refactor code
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* Update RefCOCO dataset
* [Fix] fix lint
* [Feature] Implement inference APIs for multi-modal tasks. (#65 )
* [Feature] Implement inference APIs for multi-modal tasks.
* [Project] Add gradio demo.
* [Improve] Update requirements
* Update flamingo
* Update blip
* Add NLVR inferencer
* Update flamingo
* Update hugging face model register
* Update ofa vqa
* Update BLIP-vqa (#71 )
* Update blip-vqa docstring (#72 )
* Refine flamingo docstring (#73 )
* [Feature]: BLIP2 VQA (#61 )
* [Feature]: VQA forward
* [Feature]: Reproduce accuracy
* [Fix]: Fix lint
* [Fix]: Add blank line
* minor fix
---------
Co-authored-by: yingfhu <yingfhu@gmail.com>
* [Feature]: BLIP2 docstring (#74 )
* [Feature]: Add caption docstring
* [Feature]: Add docstring to blip2 vqa
* [Feature]: Add docstring to retrieval
* Update BLIP-2 metafile and README (#75 )
* [Feature]: Add readme and docstring
* Update blip2 results
---------
Co-authored-by: mzr1996 <mzr1996@163.com>
* [Feature] BLIP Visual Grounding on MMPretrain Branch (#66 )
* blip grounding merge with mmpretrain
* remove commit
* blip grounding test and inference api
* refcoco dataset
* refcoco dataset refine config
* rebasing
* gitignore
* rebasing
* minor edit
* minor edit
* Update blip-vqa docstring (#72 )
* rebasing
* Revert "minor edit"
This reverts commit 639cec757c215e654625ed0979319e60f0be9044.
* blip grounding final
* precommit
* refine config
* refine config
* Update blip visual grounding
---------
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: mzr1996 <mzr1996@163.com>
* Update visual grounding metric
* Update OFA docstring, README and metafiles. (#76 )
* [Docs] Update installation docs and gradio demo docs. (#77 )
* Update OFA name
* Update Visual Grounding Visualizer
* Integrate accelerate support
* Fix imports.
* Fix timm backbone
* Update imports
* Update README
* Update circle ci
* Update flamingo config
* Add gradio demo README
* [Feature]: Add scienceqa (#1571 )
* [Feature]: Add scienceqa
* [Feature]: Change param name
* Update docs
* Update video
---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
Co-authored-by: yingfhu <yingfhu@gmail.com>
Co-authored-by: Yuan Liu <30762564+YuanLiuuuuuu@users.noreply.github.com>
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: Rongjie Li <limo97@163.com>
2023-05-19 16:50:04 +08:00
770eb8e24a
[Fix] Fix ddp bugs caused by `out_type`. ( #1570 )
...
* set out_type to be 'raw'
* update test
2023-05-17 17:32:10 +08:00
034919d032
[Feature] add eva02 backbone ( #1450 )
...
* [CI] Add test mim CI. (#879 )
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* update readme and configs
* refactore eva02
* [CI] Add test mim CI. (#879 )
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update ci
* rebase
* feat: add eva02 backbone
* feat: add eva02 backbone
* feat: add eva02 backbone
* update
* update readme and configs
* refactore eva02
* update readme and metafile
* update readme and metafile
* update readme and metafile
* update
* rename eva02
* rename eva02
* fix uts
* rename configs
---------
Co-authored-by: Ma Zerun <mzr1996@163.com>
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-06 19:28:31 +08:00
Ezra-Yu
Mashiro
fanqiNO1
Wangbo Zhao(黑色枷锁)
Yixiao Fang
Ma Zerun
Yiqin Wang 王逸钦
Yike Yuan
zzc98
Hubert
Mr.Li
Yuan Liu
Wang Xiang
ZhangYiqin
liuyuan