Commit Graph

15 Commits (17a886cb5825cd8c26df4e65f7112d404b99fe12)

Author SHA1 Message Date
hmtbgc c0766519b1
[Feature] Add minigpt4 gradio demo and training script. (#1758)
* Add minigpt4 gradio demo

* update minigpt4 demo

* update minigpt4 demo (inference with float16)

* update minigpt4 and some dependent files

* add minigpt4 dataset for training

* add training script for minigpt4

* restore files deleted by mistake

* fix an error

* remove useless modification

* provide command line arguments for minigpt4 gradio demo and update some comments

* update code

* Update minigpt-4 readme

---------

Co-authored-by: mzr1996 <mzr1996@163.com>
2023-10-12 10:36:17 +08:00
Yuan Liu fa53174fd9
[Feature]: Add MFF (#1725)
* [Feature]: Add MFF

* [Feature]: Add mff linear prob

* [Feature]: Add ft

* [Fix]: Update docstring

* [Feature]: Update out_indices

* [Feature]: Add prefix to ft

* [Feature]: Add README

* [Feature]: Update readme

* [Feature]: Update README

* [Feature]: Add metafile

* [Feature]: Update README

* [Fix]: Fix lint

* [Feature]: Add UT

* [Feature]: Update paper link
2023-08-08 16:01:07 +08:00
Yike Yuan 340d187765
Support Infographic VQA dataset and ANLS metric. (#1667) 2023-08-01 16:22:34 +08:00
Yike Yuan 4f2f3752d9
Support IconQA dataset. (#1670) 2023-08-01 16:14:40 +08:00
Yiqin Wang 王逸钦 6d7fe91a98
[Feature] Support Flickr30k Retrieval dataset (#1625)
* format

* remove abs path

* init add flickr30k caption

* remove abs dir

* update blip readme

* add convert sscripts

* minor

* minor
2023-06-19 15:15:03 +08:00
Yike Yuan a673b048a5
[Feature] Add support for VizWiz dataset. (#1636)
* add vizwiz

* update dataset

* [Fix] Build img_path in data_sample.

* Fix isort.

---------

Co-authored-by: ZhangYuanhan-AI <yuanhan002@ntu.edu.sg>
2023-06-16 17:16:17 +08:00
Yike Yuan 7581b76233
[Feature] Add support for vsr dataset (#1634)
* add VSR dataset

* [Fix] Modify example and load gt_answer as string.

---------

Co-authored-by: ZhangYuanhan-AI <yuanhan002@ntu.edu.sg>
2023-06-15 19:17:02 +08:00
Yiqin Wang 王逸钦 bb415b91be
[Feature] Support OCR-VQA dataset (#1621)
* support ocrvqa dataset

* minor

* remove abs path

* refine README
2023-06-13 10:28:45 +08:00
Wangbo Zhao(黑色枷锁) 3a277ee9e6
[Feature] support TextVQA dataset (#1596)
* [Support] Suport TextVQA dataset

* add folder structure

* fix readme
2023-06-02 11:50:38 +08:00
Wangbo Zhao(黑色枷锁) a779c8c5a7
[Feature] Support NoCap dataset based on BLIP. (#1582)
* [Feature] Support nocaps dataset

* precommit

* Use official coco format

* add nocp readme

* fix readme

---------

Co-authored-by: mzr1996 <mzr1996@163.com>
2023-05-23 18:06:43 +08:00
Yuan Liu 46a523ef63
[Feature] Add GQA dataset. (#1585)
* [Feature]: Add GQA dataset

* [Feature]: Add GQA

* [Feature]: Add GQA UT

* [Fix]: Fix hint

* [Feature]: Add BLIP2 GQA

* [Fix]: Fix lint

* [Feature]: Update anno link

* [Fix]: Update docstring

* [Feature]: Update all links
2023-05-23 11:25:42 +08:00
Ma Zerun 6847d20d57
[Feature] Support multiple multi-modal algorithms and inferencers. (#1561)
* [Feat] Migrate blip caption to mmpretrain. (#50)

* Migrate blip caption to mmpretrain

* minor fix

* support train

* [Feature] Support OFA caption task. (#51)

* [Feature] Support OFA caption task.

* Remove duplicated files.

* [Feature] Support OFA vqa task. (#58)

* [Feature] Support OFA vqa task.

* Fix lint.

* [Feat] Add BLIP retrieval to mmpretrain. (#55)

* init

* minor fix for train

* fix according to comments

* refactor

* Update Blip retrieval. (#62)

* [Feature] Support OFA visual grounding task. (#59)

* [Feature] Support OFA visual grounding task.

* minor add TODO

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Feat] Add flamingos coco caption and vqa. (#60)

* first init

* init flamingo coco

* add vqa

* minor fix

* remove unnecessary modules

* Update config

* Use `ApplyToList`.

---------

Co-authored-by: mzr1996 <mzr1996@163.com>

* [Feature]: BLIP2 coco retrieval  (#53)

* [Feature]: Add blip2 retriever

* [Feature]: Add blip2 all modules

* [Feature]: Refine model

* [Feature]: x1

* [Feature]: Runnable coco ret

* [Feature]: Runnable version

* [Feature]: Fix lint

* [Fix]: Fix lint

* [Feature]: Use 364 img size

* [Feature]: Refactor blip2

* [Fix]: Fix lint

* refactor files

* minor fix

* minor fix

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>

* Remove

* fix blip caption inputs (#68)

* [Feat] Add BLIP NLVR support. (#67)

* first init

* init flamingo coco

* add vqa

* add nlvr

* refactor nlvr

* minor fix

* minor fix

* Update dataset

---------

Co-authored-by: mzr1996 <mzr1996@163.com>

* [Feature]: BLIP2 Caption (#70)

* [Feature]: Add language model

* [Feature]: blip2 caption forward

* [Feature]: Reproduce the results

* [Feature]: Refactor caption

* refine config

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Feat] Migrate BLIP VQA to mmpretrain (#69)

* reformat

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* change

* refactor code

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>

* Update RefCOCO dataset

* [Fix] fix lint

* [Feature] Implement inference APIs for multi-modal tasks. (#65)

* [Feature] Implement inference APIs for multi-modal tasks.

* [Project] Add gradio demo.

* [Improve] Update requirements

* Update flamingo

* Update blip

* Add NLVR inferencer

* Update flamingo

* Update hugging face model register

* Update ofa vqa

* Update BLIP-vqa (#71)

* Update blip-vqa docstring (#72)

* Refine flamingo docstring (#73)

* [Feature]: BLIP2 VQA (#61)

* [Feature]: VQA forward

* [Feature]: Reproduce accuracy

* [Fix]: Fix lint

* [Fix]: Add blank line

* minor fix

---------

Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Feature]: BLIP2 docstring (#74)

* [Feature]: Add caption docstring

* [Feature]: Add docstring to blip2 vqa

* [Feature]: Add docstring to retrieval

* Update BLIP-2 metafile and README (#75)

* [Feature]: Add readme and docstring

* Update blip2 results

---------

Co-authored-by: mzr1996 <mzr1996@163.com>

* [Feature] BLIP Visual Grounding on MMPretrain Branch (#66)

* blip grounding merge with mmpretrain

* remove commit

* blip grounding test and inference api

* refcoco dataset

* refcoco dataset refine config

* rebasing

* gitignore

* rebasing

* minor edit

* minor edit

* Update blip-vqa docstring (#72)

* rebasing

* Revert "minor edit"

This reverts commit 639cec757c215e654625ed0979319e60f0be9044.

* blip grounding final

* precommit

* refine config

* refine config

* Update blip visual grounding

---------

Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: mzr1996 <mzr1996@163.com>

* Update visual grounding metric

* Update OFA docstring, README and metafiles. (#76)

* [Docs] Update installation docs and gradio demo docs. (#77)

* Update OFA name

* Update Visual Grounding Visualizer

* Integrate accelerate support

* Fix imports.

* Fix timm backbone

* Update imports

* Update README

* Update circle ci

* Update flamingo config

* Add gradio demo README

* [Feature]: Add scienceqa (#1571)

* [Feature]: Add scienceqa

* [Feature]: Change param name

* Update docs

* Update video

---------

Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>
Co-authored-by: yingfhu <yingfhu@gmail.com>
Co-authored-by: Yuan Liu <30762564+YuanLiuuuuuu@users.noreply.github.com>
Co-authored-by: Yiqin Wang 王逸钦 <wyq1217@outlook.com>
Co-authored-by: Rongjie Li <limo97@163.com>
2023-05-19 16:50:04 +08:00
zzc98 496e098b21
[Feature] Support some downstream classification datasets. (#1467)
* feat: support some downstream classification datasets

* update sun397

* sum

* update sun397

* [CI] Add test mim CI. (#879)

* feat: support some downstream classification datasets

* update sun397

* sum

* update sun397

* rebase

* feat: support some downstream classification datasets

* update sun397

* update sun397

* update sun397

* update sun397

* fix unittest

* update docstring

* rm

* update

* update

* refactor names of datasets

* refactor some implements of datasets

* refactor some implements of datasets

* fix datasets unittest

* refactor cub and stanford cars

* refactor cub and cifar

* refactor cub and cifar

* refactor cub and cifar

* update downstream datasets and docs

* update docstring

---------

Co-authored-by: Ma Zerun <mzr1996@163.com>
Co-authored-by: Ezra-Yu <18586273+Ezra-Yu@users.noreply.github.com>
2023-05-05 14:43:14 +08:00
Yixiao Fang 75c79311f4
[Refactor] Update datasets (#1375)
* add ut

* add places205

* support ann_file without labels

* temp test

* update custom

* update

* update ut

* Update CustomDataset.

* Update Places205.

---------

Co-authored-by: mzr1996 <mzr1996@163.com>
2023-02-27 15:42:22 +08:00
mzr1996 0979e78573 Rename the package name to `mmpretrain`. 2023-02-17 15:20:55 +08:00