Commit Graph

250 Commits (381c5f103c3c77b55ee8ecf233ce0916eac67b61)

Author SHA1 Message Date
Ma Zerun fe26c6559e
[Enhance] Support multiple `--cfg-options`. (#759) 2022-12-07 10:44:25 +08:00
Qian Zhao bd6791382f
[Fix] BaseModel & BaseDataPreprocessor `to` method to be consistent with torch.nn.Module (#783)
* fix BaseModel `to` method to be consistent with torch.nn.Module

* fix data_preprocessor as well

* fix docstring alignment

* fix docstring alignment
2022-12-05 18:04:47 +08:00
SUNYC 7e787d8ffa
[Enhance] Add a new parameter conf_path in PetrelBackend (#774)
* feat: add conf_path in PetrelBackend

* Update mmengine/fileio/backends/petrel_backend.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
2022-11-29 14:23:16 +08:00
liukuikun 5f58517069
[Enchance] remove redundant attributes in BaseDataElement (#690)
* remove redundant attribute in dump or print

* optimize efficiency

Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-11-24 21:36:19 +08:00
Mashiro ded73f3a56
[Enhance] Enhance compatibility of `revert_sync_batchnorm` (#695)
* [Enhance] Enhance revert_sync_batchnorm and convert_sync_batchnorm

* [Enhance] Enhance revert_sync_batchnorm and convert_sync_batchnorm

* Fix unit test

* Add coments

* Refine comments

* clean the code

* revert convert_sync_batchnorm

* revert convert_sync_batchnorm

* refine comment

* fix CI

* fix CI
2022-11-24 16:41:54 +08:00
cir7 0e6bb48b12
[Enhance] Support eta_min_ratio in CosineAnnealingParamScheduler (#725)
* [Enhance] support eta_min_ratio in CosineAnnealingParamScheduler

* [doc] fix docstring

* [Enhance] add ut for eta_min_ratio

* [doc] update docstring

* avoid bc-breaking of eta_min

* [doc] add docstring in CosineAnnealingParamScheduler and CosineAnnealingMomentum

* Apply suggestions from code review

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-22 20:19:16 +08:00
songyuc 6636f07cfe
[Feature] Add get_hooks_info() to print hooks messages (#672)
* Add test of get_hooks_info()

* Change to use original Runner for get_hook_info() test

* Change to test after_train_iter hooks for get_hook_info()

* Complement the stages list

* Add logging hooks information in Runner.__init__()

* Rearrange the stages list

* Restore the stages to tuple type

* Clean the unnecessary changes

* Replace  statement with TestCase's methods

* add test stages in method_stages_map

* change the hooks info into a f-string

* return list(trigger_stages) directly

* change keys of method_stages_map

* Fix previous changes to method_stages_map.keys
2022-11-22 20:02:29 +08:00
Mashiro b06234cfcd
[Enhance] Right align the log (#436)
* right allign the log

* fix as comment

* Add comments for magic number 3

* remove max_len_str

* Update mmengine/runner/log_processor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-21 11:55:18 +08:00
Mashiro c478bdca27
[Enhance] enhance runner test case (#631)
* Add runner test cast

* Fix unit test

* fix unit test

* pop None if key does not exist

* Fix is_model_wrapper and force register class in test_runner

* [Fix] Fix is_model_wrapper

* destroy group after ut

* register module in testcase

* fix as comment

* minor refine

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* fix lint

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-21 11:54:05 +08:00
RangiLyu a9a575866f [Fix] Fix offline_evaluate index error (#630)
* [Fix] Fix offline eval dataset index error.

* update

* update
2022-11-18 01:17:12 +08:00
RangiLyu f2b0540f58 [Enhance] Raise warning for abnormal momentum (#655) 2022-11-18 01:17:12 +08:00
Yanhong Zeng 618a063ddc
[Docs] Fix docstring and documentation used for hub.get_model (#659)
* fix docstring and documentation used for hub.get_model

* fix config name in test_hub

* fix docstring mentioned faster-rcnn in utils
2022-11-02 10:16:56 +08:00
BigDong 376251961d
[Enhance] Add `ignore_keys` in ConcatDataset (#556)
* [Fix] Fix ConcatDataset error in VOCDataset

* minor fix

* minor fix

* minor fix

* add UT

* minor fix

* minor fix

* minor fix

* Update mmengine/dataset/dataset_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* minor fix

* Update mmengine/dataset/dataset_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-11-01 17:06:02 +08:00
Z-Fran 090104df21
[Fix] Fix the calculation error of eta_min in CosineRestart (#639)
* [Fix] fix CosineRestart eta_min

* add ut case

* Enhance unit test

Enhance unit test

* remove unused code

Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-11-01 15:48:39 +08:00
Mashiro d1dd240796
[Fix] Fix `BaseDataPreprocessor.cast_data` cound not handle string data (#602)
* [Fix] Fix  cound not handle string data

* Minor refine

* Refine type hint

Refine type hint

* fix as comment

* Minor refine

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-28 15:17:27 +08:00
Hakjin Lee 0857f9fb40
[Feature] Support torch ZeroRedundancyOptimizer (#551)
* [Feature] Support torch ZeRORedundancyOptimizer

Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>

* lint

* Fix saving optimizer state_dict

* Fix handling import error

* Add test case

* fix UT

* Revert "fix UT"

This reverts commit dd64538960.

* fix handling import in UT

* Fix saving zero checkpoint and delete redundant master_only

* lint

* test unittest

* Fix handling impor error

* Fix UT condition

* Edit docstrings

* Fix typo

* Skip redundant procudure in checkpoint hook

* fix typo again

* Update mmengine/optim/optimizer/zero_optimizer.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Add api info

* lint

* Fix lint

* Handling AmpOptimWrapper case

* handling overlap_with_ddp

* Fix error

Signed-off-by: Junhwa Song <ethan9867@gmail.com>
Signed-off-by: Hakjin Lee <nijkah@gmail.com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-27 20:31:50 +08:00
Mashiro c1447734c2
[Enhance] Call `register_all_modules` in `Registry.get()` (#541)
* call register_all_modules  in Registry.get()

* Fix ci

* fix scope bug (scope_name -> scope), for temp sync

* Fix unit test

* Refine log information

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Fix typo

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-24 20:15:14 +08:00
Mashiro bda92b49e7
[Feature] Add a function to mark the deprecated function. (#609)
* [Enhancement] Add a deprecated function

* Refine and add ut

* tmp save

* minor refine

* Fix unit test

* Fix unit test

* fix docstring

* Update utils.rst
2022-10-24 14:49:48 +08:00
wangjiangben-hw 601db12d38
[Feature] Support engine with NPU backend. (#572)
* init npu

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/dist/dist.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* change to is_hccl_backend

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add comment with AmpOptimWrapper

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/runner/amp.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add npu fn in base_model

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* clean lint

* Update mmengine/optim/optimizer/amp_optimizer_wrapper.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/base_model.py

Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>

* add is_npu_available

* try to fix

* Add comments

* Refine grammar

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Mashiro <57566630+HAOCHENYE@users.noreply.github.com>
Co-authored-by: HAOCHENYE <21724054@zju.edu.cn>
2022-10-24 11:42:29 +08:00
shenmishajing d270516fe8
[Fix]: fix add_config func is not called bug (#613)
* fix add_config func is not called bug
fix wandb add_config bug
add wandb log_code feature

* move log_code_name param to the last one
add log_code_name param to docstring

* add config only when there is a cfg

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* add unit test for log_code_name param of WandbVisBackend

* Update mmengine/visualization/vis_backend.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* use log_code_kwargs instead of only log_code_name
add PATH_FILTERS registry

* use log_code_kwargs instead of only log_code_name
add PATH_FILTERS registry

* fix add config unit test

* roll back to log_code_name version

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-24 09:58:10 +08:00
Mashiro aaba1d8871
[Fix] Fix error format of log message (#508)
* Fix error format of log message

* Fix unit test

* remove unnecessary comment
2022-10-18 18:04:15 +08:00
Mashiro abe56651db
[Fix] Make autocast compatible with mps (#587)
* [Fix] Make autocast compatible with mps

* Enhance unit test

* fix unit test

* clean the code

* fix unit test
2022-10-18 18:03:17 +08:00
Mashiro 6073d9ebd8
[Enhance] add documents for `clip_grad` , and support clip grad by value. (#513)
* [Enhance] add documents for , and support clip grad by value

* refine docstring

* fix as comment

* Fix as comment

* minor refine

* minor refine

* remove error comment for clip grad

* refine docstring
2022-10-18 18:02:46 +08:00
shenmishajing c2982723c9
[Feats]: add non_blocking feature to BaseDataPreprocessor (#618)
* add non_blocking feature to BaseDataPreprocessor

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* move new parameters to the last to avoid bc issue

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

* Update mmengine/model/base_model/data_preprocessor.py

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>

Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
2022-10-18 17:58:51 +08:00
Mashiro 62456217f9
[Feature] Add test time augmentation base model. (#538)
* First commit

* add BaseTestTimeAugModel

* Add unit test

* move loop logic to test_step

* fix ddp test

* rename model to module

* optim __init__

* Fix as comment

* Fix as comment

* make val_step should not be called

* make tta do not inherit base model

* Fix unit test

* Enhance docstring

* Fix as comment

* Fix as comment

* minor refine

* minor refine

* minor refine

* fix unit test

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* minor refine

* fix unit test
2022-10-10 14:29:33 +08:00
Qian Zhao e73c4bf135
[Fix] some out-of-date unittests (#586)
* fix dist ut

* fix config & hub ut break due to mmdet faster-rcnn rename
2022-10-08 19:48:51 +08:00
Mashiro 89146a5b90
[Fix] Fix detect_anomalous_params (#588) 2022-10-08 19:48:35 +08:00
Zaida Zhou ed84dfd34d
[Refactor] Refactor fileio without breaking back compatibility (#533)
* [Refactor] Refactor fileio but without breaking bc

* handle compatibility

* fix format

* modify io functions

* fix ut

* fix ut

* rename method names

* refine

* refine docstring

* fix ut in windows

* update ut

* minor fix

* ensure client is not None when closing it

* add more examples for list_dir_or_file interface

* refine docstring

* refine deprecated info

* fix ut

* add a description for lmdb docstring
2022-09-26 14:30:40 +08:00
Qian Zhao c64243aa9e
[Fix] CheckpointHook behavior incorrect if given `filename_tmpl` argument (#518) 2022-09-22 12:47:45 +08:00
Mashiro 5d7527d75e
[Fix ] Fix unit test could may fail caused by `MultiProcessTestCase` (#535)
* fix merge ci

* make timeout larger

* tmp test merge

* debug

* sleep 1 in optimizer ut

* skip test clip grad

* skip test clip grad

* fix merge ci
2022-09-15 18:08:56 +08:00
Mashiro 8ee31dbc3b
[Feature] Support convert `BN` to `SyncBN` by config (#506)
* [Feature] Support convert BN to SyncBN by config

* make unit test compatible with cpu

* Fix as comment

* fix unit test

* change signature of convert_sync_batchnorm: rename sync_bn to implemention

* fix unit test

* fix unit test
2022-09-15 18:08:36 +08:00
Mashiro 6b1b8a3751
[Fix] Fix unit test in windows (#515) 2022-09-13 11:46:21 +08:00
Mashiro 0fb2b8ca8c
[Fix] Fix circle import in EMAHook (#523) 2022-09-09 19:33:27 +08:00
takuoko a6f5297727
[fix] EMAHook load state dict (#507)
* fix ema load_state_dict

* fix ema load_state_dict

* fix for test

* fix by review

* fix resume and keys
2022-09-09 11:41:12 +08:00
Zaida Zhou 19d4eba866
[Refactor] Update Github Action CI and CircleCI (#512)
* [CI] Update Github Action CI and CircleCI

* clear outdated commands in CI

* clear outdated commands in CI

* add '-r' when installing packages from txt

* update

* update circleci

* fix mismatch conflicts

* update

* install system dependencies

* update

* set env

* fix ut

* update cuda version on Windows

* update push action
2022-09-07 20:24:11 +08:00
Mashiro 2d3bff4418
[Fix] Fix unit test of `Config` will install `mmdet` and `mmcls`. (#492)
* fix unit test install mmdet and mmcls

* raise error when mmdet is not installed

* rename check_and_install to install_package

* split test case

Co-authored-by: C1rN09 <zhaoqian@pjlab.org.cn>
2022-08-31 21:18:05 +08:00
liukuikun 1fea82aad5
[Docs] update data element tutorials (#431)
* structure tutorials

* refine data element docs

* modify introduce

* fix comment

* fix comment

* fix comment
2022-08-31 20:39:08 +08:00
Qian Zhao 94412f72d8
[Fix] Exception while building registered function (#491) 2022-08-31 13:53:32 +08:00
Mashiro 33d7c81bc4
[Refactor]: Avoid creating instances with the same name (#483) 2022-08-30 16:28:18 +08:00
wxDai fba9a94f52
[Refactor] add testing utils (#475)
* add testing utils

* fix ut

* add blank line betweeen `Args` and `Returns`
2022-08-29 22:59:20 +08:00
Zaida Zhou 2f09342663
[Docs] Refine registry docs (#443)
* [Docs] Refine registry docs

* explain how to use _scope_

* refine
2022-08-29 14:50:48 +08:00
Mashiro 576e5c8f91
[Fix] Regist pytorch ddp and dp to `MODEL_WRAPPERS`, add unit test to `is_model_wrapper` (#474)
* regist pytorch ddp and dp, add unit test

* minor refine

* Support check custom wrapper

* enhance ut
2022-08-28 17:09:41 +08:00
RangiLyu 8d25dbdeda
[Fix] Fix EMAHook trigger train loop and AveragedModel sync buffer. (#467)
* [Fix] Fix EMAHook trigger train loop init during testing.

* fix sync buffer

* update ut

* fix sync buffer

* fix sync buffer
2022-08-26 14:21:56 +08:00
Mashiro 8770c6c7fc
[Refactor] Refactor data flow to make the interface more natural (#468)
* [Refactor]: modify interface of Visualizer.add_datasample (#365)

* [Refactor] Refactor data flow: refine `data_preprocessor`. (#359)

* refine data_preprocessor

* remove unused BATCH_DATA alias

* Fix type hints

* rename move_data to cast_data

* [Refactor] Refactor data flow: collate data in `collate_fn` of `DataLoader`  (#323)

* acollate data in dataloader

* fix docstring

* refine comment

* fix as comment

* refactor default collate and psedo collate

* foramt test file

* fix docstring

* fix as comment

* rename elem to data_item

* minor fix

* fix as comment

* [Refactor] Refactor data flow: `data_batch` argument of `Evaluator.process is a `dict` (#360)

* refine evaluator and metric

* compatible with new default collate

* replace default collate with pseudo

* Handle data_batch in metric

* fix unit test

* fix unit test

* fix unit test

* minor refine

* make data_batch optional

make data_batch optional

* rename outputs to predictions

* fix ut

* rename predictions to outputs

* fix docstring

* fix docstring

* fix unit test

* make outputs and data_batch to kwargs

* fix unit test

* keep signature of metric

* fix ut

* rename pred_sample arguments to data_sample(Visualizer)

* fix loop and ut

* [refactor]: Refactor model dataflow (#398)

* [Refactor] Refactor data flow: refine `data_preprocessor`. (#359)

* refine data_preprocessor

* remove unused BATCH_DATA alias

* Fix type hints

* rename move_data to cast_data

* refactor model data flow

tmp_commt

tmp commit

* make val_cfg and test_cfg optional

* roll back runner

* pass test mmdet

* fix as comment

fix as comment

fix ci in DataPreprocessor

* fix ut

* fix ut

* fix rebase main

* [Fix]: Fix test val ddp (#462)

* [Fix] Fix docstring and type hint of data flow (#463)

* Fix docstring of data flow

* change signature of hook

* fix unit test

* resolve conflicts

* fix lint
2022-08-24 22:04:55 +08:00
Zaida Zhou 7e1d7af2d9
[Refactor] Refactor code structure (#395)
* Rename data to structure

* adjust the way to import module

* adjust the way to import module

* rename Structure to Data Structures in docs api

* rename structure to structures

* support using some modules of mmengine without torch

* fix circleci config

* fix circleci config

* fix registry ut

* minor fix

* move init method from model/utils to model/weight_init.py

* move init method from model/utils to model/weight_init.py

* move sync_bn to model

* move functions depending on torch to dl_utils

* format import

* fix logging ut

* add weight init in model/__init__.py

* move get_config and get_model to mmengine/hub

* move log_processor.py to mmengine/runner

* fix ut

* Add TimeCounter in dl_utils/__init__.py
2022-08-24 19:14:07 +08:00
Zaida Zhou 486d8cda56
[Refactor] Refactor the import rule (#459)
* [Refactor] Refactor the import rule

* minor refinement

* add a comment
2022-08-23 18:58:36 +08:00
Zaida Zhou a9ad09bded
[Fix] Fix utils ut (#458) 2022-08-23 16:56:47 +08:00
liukuikun bb56cf42ab
[Visualizer] use FigureManager to manage figure to avoid affecting plt.show() outside Visualizer(#440)
* figure in Visualizer is not managed by plt

* encapsulate code and remove unused code
2022-08-22 17:16:14 +08:00
Zaida Zhou b75962a660
Migrate utils from mmcv (#447) 2022-08-22 11:51:56 +08:00
Mashiro e907931fb8
Fix unit tests (#449) 2022-08-21 14:54:24 +08:00