diff --git a/docs/en/tutorials/config.md b/docs/en/advanced_tutorials/config.md similarity index 100% rename from docs/en/tutorials/config.md rename to docs/en/advanced_tutorials/config.md diff --git a/docs/en/examples/cross_library.md b/docs/en/advanced_tutorials/cross_library.md similarity index 100% rename from docs/en/examples/cross_library.md rename to docs/en/advanced_tutorials/cross_library.md diff --git a/docs/en/tutorials/data_transform.md b/docs/en/advanced_tutorials/data_transform.md similarity index 100% rename from docs/en/tutorials/data_transform.md rename to docs/en/advanced_tutorials/data_transform.md diff --git a/docs/en/tutorials/registry.md b/docs/en/advanced_tutorials/registry.md similarity index 100% rename from docs/en/tutorials/registry.md rename to docs/en/advanced_tutorials/registry.md diff --git a/docs/en/index.rst b/docs/en/index.rst index 7f2de14c..a54b5eec 100644 --- a/docs/en/index.rst +++ b/docs/en/index.rst @@ -10,42 +10,43 @@ You can switch between Chinese and English documents in the lower-left corner of get_started/installation.md get_started/15_minutes.md +.. toctree:: + :maxdepth: 1 + :caption: Common Usage + + examples/resume_training.md + examples/speed_up_training.md + examples/save_gpu_memory.md + examples/train_a_gan.md + .. toctree:: :maxdepth: 3 :caption: Tutorials - tutorials/registry.md - tutorials/config.md tutorials/runner.md - tutorials/hook.md + tutorials/dataset.md tutorials/model.md tutorials/evaluation.md tutorials/optim_wrapper.md tutorials/param_scheduler.md - tutorials/data_transform.md + tutorials/hook.md .. toctree:: :maxdepth: 1 :caption: Advanced tutorials + advanced_tutorials/registry.md + advanced_tutorials/config.md advanced_tutorials/basedataset.md - advanced_tutorials/data_element.md - advanced_tutorials/visualization.md + advanced_tutorials/data_transform.md advanced_tutorials/initialize.md + advanced_tutorials/visualization.md + advanced_tutorials/data_element.md advanced_tutorials/distributed.md advanced_tutorials/logging.md advanced_tutorials/fileio.md advanced_tutorials/utils.md - -.. toctree:: - :maxdepth: 1 - :caption: Examples - - examples/resume_training.md - examples/speed_up_training.md - examples/save_gpu_memory.md - examples/cross_library.md - examples/train_a_gan.md + advanced_tutorials/cross_library.md .. toctree:: :maxdepth: 1 diff --git a/docs/en/tutorials/dataset.md b/docs/en/tutorials/dataset.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/zh_cn/tutorials/config.md b/docs/zh_cn/advanced_tutorials/config.md similarity index 100% rename from docs/zh_cn/tutorials/config.md rename to docs/zh_cn/advanced_tutorials/config.md diff --git a/docs/zh_cn/examples/cross_library.md b/docs/zh_cn/advanced_tutorials/cross_library.md similarity index 93% rename from docs/zh_cn/examples/cross_library.md rename to docs/zh_cn/advanced_tutorials/cross_library.md index aef9a8bd..4460d59e 100644 --- a/docs/zh_cn/examples/cross_library.md +++ b/docs/zh_cn/advanced_tutorials/cross_library.md @@ -1,6 +1,6 @@ # 跨库调用模块 -通过使用 MMEngine 的[注册器（Registry）](../tutorials/registry.md)和[配置文件（Config）](../tutorials/config.md)，用户可以实现跨软件包的模块构建。 +通过使用 MMEngine 的[注册器（Registry）](registry.md)和[配置文件（Config）](config.md)，用户可以实现跨软件包的模块构建。 例如，在 [MMDetection](https://github.com/open-mmlab/mmdetection) 中使用 [MMClassification](https://github.com/open-mmlab/mmclassification) 的 Backbone，或者在 [MMRotate](https://github.com/open-mmlab/mmrotate) 中使用 [MMDetection](https://github.com/open-mmlab/mmdetection) 的 Transform，或者在 [MMTracking](https://github.com/open-mmlab/mmtracking) 中使用 [MMDetection](https://github.com/open-mmlab/mmdetection) 的 Detector。 一般来说，同类模块都可以进行跨库调用，只需要在配置文件的模块类型前加上软件包名的前缀即可。下面举几个常见的例子： @@ -91,4 +91,4 @@ model = dict( 以上这两种写法互相等价。 -若希望了解更多关于注册器和配置文件的内容，请参考[配置文件教程](../tutorials/config.md)和[注册器教程](../tutorials/registry.md) +若希望了解更多关于注册器和配置文件的内容，请参考[配置文件教程](config.md)和[注册器教程](registry.md) diff --git a/docs/zh_cn/tutorials/data_transform.md b/docs/zh_cn/advanced_tutorials/data_transform.md similarity index 100% rename from docs/zh_cn/tutorials/data_transform.md rename to docs/zh_cn/advanced_tutorials/data_transform.md diff --git a/docs/zh_cn/advanced_tutorials/initialize.md b/docs/zh_cn/advanced_tutorials/initialize.md index eeac2a33..a75d024a 100644 --- a/docs/zh_cn/advanced_tutorials/initialize.md +++ b/docs/zh_cn/advanced_tutorials/initialize.md @@ -53,7 +53,7 @@ toy_net.init_weights() 08/19 16:50:24 - mmengine - INFO - local loads checkpoint from path: ./pretrained.pth ``` -当 `init_cfg` 是一个字典时，`type` 字段就表示一种初始化器，它需要被注册到 `WEIGHT_INITIALIZERS` [注册器](../tutorials/registry.md)。我们可以通过指定 `init_cfg=dict(type='Pretrained', checkpoint='path/to/ckpt')` 来加载预训练权重，其中 `Pretrained` 为 `PretrainedInit` 初始化器的缩写，这个映射名由 `WEIGHT_INITIALIZERS` 维护；`checkpoint` 是 `PretrainedInit` 的初始化参数，用于指定权重的加载路径，它可以是本地磁盘路径，也可以是 URL。 +当 `init_cfg` 是一个字典时，`type` 字段就表示一种初始化器，它需要被注册到 `WEIGHT_INITIALIZERS` [注册器](registry.md)。我们可以通过指定 `init_cfg=dict(type='Pretrained', checkpoint='path/to/ckpt')` 来加载预训练权重，其中 `Pretrained` 为 `PretrainedInit` 初始化器的缩写，这个映射名由 `WEIGHT_INITIALIZERS` 维护；`checkpoint` 是 `PretrainedInit` 的初始化参数，用于指定权重的加载路径，它可以是本地磁盘路径，也可以是 URL。 ### 常用的初始化方式 diff --git a/docs/zh_cn/tutorials/registry.md b/docs/zh_cn/advanced_tutorials/registry.md similarity index 100% rename from docs/zh_cn/tutorials/registry.md rename to docs/zh_cn/advanced_tutorials/registry.md diff --git a/docs/zh_cn/design/evaluation.md b/docs/zh_cn/design/evaluation.md index dc3e0393..2585aa7a 100644 --- a/docs/zh_cn/design/evaluation.md +++ b/docs/zh_cn/design/evaluation.md @@ -38,7 +38,7 @@ 我们以实现分类正确率（Classification Accuracy）评测指标为例，说明自定义评测指标的方法。 -首先，评测指标类应继承自 `BaseMetric`，并应加入注册器 `METRICS` (关于注册器的说明请参考[相关文档](../tutorials/registry.md))。 +首先，评测指标类应继承自 `BaseMetric`，并应加入注册器 `METRICS` (关于注册器的说明请参考[相关文档](../advanced_tutorials/registry.md))。 `process()` 方法有 2 个输入参数，分别是一个批次的测试数据样本 `data_batch` 和模型预测结果 `predictions`。我们从中分别取出样本类别标签和分类预测结果，并存放在 `self.results` 中。 diff --git a/docs/zh_cn/get_started/introduction.md b/docs/zh_cn/get_started/introduction.md index 6dc2f9cd..2e69af6c 100644 --- a/docs/zh_cn/get_started/introduction.md +++ b/docs/zh_cn/get_started/introduction.md @@ -51,8 +51,8 @@ MMEngine 将训练过程中涉及的组件和它们的关系进行了抽象， MMEngine 中还实现了各种算法模型执行过程中需要用到的公共基础模块，包括 -- [配置类（Config）](../tutorials/config.md)：在 OpenMMLab 算法库中，用户可以通过编写 config 来配置训练、测试过程以及相关的组件。 -- [注册器（Registry）](../tutorials/registry.md)：负责管理算法库中具有相同功能的模块。MMEngine 根据对算法库模块的抽象，定义了一套根注册器，算法库中的注册器可以继承自这套根注册器，实现模块的跨算法库调用。 +- [配置类（Config）](../advanced_tutorials/config.md)：在 OpenMMLab 算法库中，用户可以通过编写 config 来配置训练、测试过程以及相关的组件。 +- [注册器（Registry）](../advanced_tutorials/registry.md)：负责管理算法库中具有相同功能的模块。MMEngine 根据对算法库模块的抽象，定义了一套根注册器，算法库中的注册器可以继承自这套根注册器，实现模块的跨算法库调用。 - [文件读写（File I/O）](../tutorials/fileio.md)：为各个模块的文件读写提供了统一的接口，以统一的形式支持了多种文件读写后端和多种文件格式，并具备扩展性。 - [分布式通信原语（Distributed Communication Primitives）](../tutorials/distributed.md)：负责在程序分布式运行过程中不同进程间的通信。这套接口屏蔽了分布式和非分布式环境的区别，同时也自动处理了数据的设备和通信后端。 - [其他工具（Utils）](../tutorials/utils.md)：还有一些工具性的模块，如 ManagerMixin，它实现了一种全局变量的创建和获取方式，执行器内很多全局可见对象的基类就是 ManagerMixin。 diff --git a/docs/zh_cn/index.rst b/docs/zh_cn/index.rst index 2d2f4165..389a62ee 100644 --- a/docs/zh_cn/index.rst +++ b/docs/zh_cn/index.rst @@ -10,42 +10,43 @@ get_started/installation.md get_started/15_minutes.md +.. toctree:: + :maxdepth: 1 + :caption: 常用功能 + + examples/resume_training.md + examples/speed_up_training.md + examples/save_gpu_memory.md + examples/train_a_gan.md + .. toctree:: :maxdepth: 3 :caption: 入门教程 - tutorials/registry.md - tutorials/config.md tutorials/runner.md - tutorials/hook.md + tutorials/dataset.md tutorials/model.md tutorials/evaluation.md tutorials/optim_wrapper.md tutorials/param_scheduler.md - tutorials/data_transform.md + tutorials/hook.md .. toctree:: :maxdepth: 1 :caption: 进阶教程 + advanced_tutorials/registry.md + advanced_tutorials/config.md advanced_tutorials/basedataset.md - advanced_tutorials/data_element.md - advanced_tutorials/visualization.md + advanced_tutorials/data_transform.md advanced_tutorials/initialize.md + advanced_tutorials/visualization.md + advanced_tutorials/data_element.md advanced_tutorials/distributed.md advanced_tutorials/logging.md advanced_tutorials/fileio.md advanced_tutorials/utils.md - -.. toctree:: - :maxdepth: 1 - :caption: 示例 - - examples/resume_training.md - examples/speed_up_training.md - examples/save_gpu_memory.md - examples/cross_library.md - examples/train_a_gan.md + advanced_tutorials/cross_library.md .. toctree:: :maxdepth: 1 diff --git a/docs/zh_cn/migration/transform.md b/docs/zh_cn/migration/transform.md index e5ada4b1..5235348d 100644 --- a/docs/zh_cn/migration/transform.md +++ b/docs/zh_cn/migration/transform.md @@ -4,7 +4,7 @@ 在 TorchVision 的数据变换类接口约定中，数据变换类需要实现 `__call__` 方法，而在 OpenMMLab 1.0 的接口约定中，进一步要求 `__call__` 方法的输出应当是一个字典，在各种数据变换中对这个字典进行增删查改。在 OpenMMLab 2.0 中，为了提升后续的可扩展性，我们将原先的 `__call__` 方法迁移为 `transform` 方法，并要求数据变换类应当继承 -[`mmcv.transforms.BaseTransfrom`](https://mmcv.readthedocs.io/en/dev-2.x/api.html#TODO)。具体如何实现一个数据变换类，可以参见[文档](../tutorials/data_transform.md)。 +[`mmcv.transforms.BaseTransfrom`](https://mmcv.readthedocs.io/en/dev-2.x/api.html#TODO)。具体如何实现一个数据变换类，可以参见[文档](../advanced_tutorials/data_transform.md)。 由于在此次更新中，我们将部分共用的数据变换类统一迁移至 MMCV 中，因此本文的将会以 [MMClassification v0.23.2](https://github.com/open-mmlab/mmclassification/tree/v0.23.2)、[MMDetection v2.25.1](https://github.com/open-mmlab/mmdetection/tree/v2.25.1) 和 [MMCV v2.0.0rc0](https://github.com/open-mmlab/mmcv/tree/dev-2.x) 为例，对比这些数据变换类在新旧版本中功能、用法和实现上的差异。 diff --git a/docs/zh_cn/notes/changelog.md b/docs/zh_cn/notes/changelog.md new file mode 100644 index 00000000..20e476a3 --- /dev/null +++ b/docs/zh_cn/notes/changelog.md @@ -0,0 +1,57 @@ +# Changelog of v0.x + +## v0.2.0 (11/10/2022) + +### New Features & Enhancements + +- Add SMDDP backend and support running on AWS by @austinmw in https://github.com/open-mmlab/mmengine/pull/579 +- Refactor `FileIO` but without breaking bc by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/533 +- Add test time augmentation base model by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/538 +- Use `torch.lerp\_()` to speed up EMA by @RangiLyu in https://github.com/open-mmlab/mmengine/pull/519 +- Support converting `BN` to `SyncBN` by config by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/506 +- Support defining metric name in wandb backend by @okotaku in https://github.com/open-mmlab/mmengine/pull/509 +- Add dockerfile by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/347 + +### Docs + +- Fix API files of English documentation by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/525 +- Fix typo in `instance_data.py` by @Dai-Wenxun in https://github.com/open-mmlab/mmengine/pull/530 +- Fix the docstring of the model sub-package by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/573 +- Fix a spelling error in docs/zh_cn by @cxiang26 in https://github.com/open-mmlab/mmengine/pull/548 +- Fix typo in docstring by @MengzhangLI in https://github.com/open-mmlab/mmengine/pull/527 +- Update `config.md` by @Zhengfei-0311 in https://github.com/open-mmlab/mmengine/pull/562 + +### Bug Fixes + +- Fix `LogProcessor` does not smooth loss if the name of loss doesn't start with `loss` by @liuyanyi in + https://github.com/open-mmlab/mmengine/pull/539 +- Fix failed to enable `detect_anomalous_params` in `MMSeparateDistributedDataParallel` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/588 +- Fix CheckpointHook behavior unexpected if given `filename_tmpl` argument by @C1rN09 in https://github.com/open-mmlab/mmengine/pull/518 +- Fix error argument sequence in `FSDP` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/520 +- Fix uploading image in wandb backend @okotaku in https://github.com/open-mmlab/mmengine/pull/510 +- Fix loading state dictionary in `EMAHook` by @okotaku in https://github.com/open-mmlab/mmengine/pull/507 +- Fix circle import in `EMAHook` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/523 +- Fix unit test could fail caused by `MultiProcessTestCase` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/535 +- Remove unnecessary "if statement" in `Registry` by @MambaWong in https://github.com/open-mmlab/mmengine/pull/536 +- Fix `_save_to_state_dict` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/542 +- Support comparing NumPy array dataset meta in `Runner.resume` by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/511 +- Use `get` instead of `pop` to dump `runner_type` in `build_runner_from_cfg` by @nijkah in https://github.com/open-mmlab/mmengine/pull/549 +- Upgrade pre-commit hooks by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/576 +- Delete the error comment in `registry.md` by @vansin in https://github.com/open-mmlab/mmengine/pull/514 +- Fix Some out-of-date unit tests by @C1rN09 in https://github.com/open-mmlab/mmengine/pull/586 +- Fix typo in `MMFullyShardedDataParallel` by @yhna940 in https://github.com/open-mmlab/mmengine/pull/569 +- Update Github Action CI and CircleCI by @zhouzaida in https://github.com/open-mmlab/mmengine/pull/512 +- Fix unit test in windows by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/515 +- Fix merge ci & multiprocessing unit test by @HAOCHENYE in https://github.com/open-mmlab/mmengine/pull/529 + +### New Contributors + +- @okotaku made their first contribution in https://github.com/open-mmlab/mmengine/pull/510 +- @MengzhangLI made their first contribution in https://github.com/open-mmlab/mmengine/pull/527 +- @MambaWong made their first contribution in https://github.com/open-mmlab/mmengine/pull/536 +- @cxiang26 made their first contribution in https://github.com/open-mmlab/mmengine/pull/548 +- @nijkah made their first contribution in https://github.com/open-mmlab/mmengine/pull/549 +- @Zhengfei-0311 made their first contribution in https://github.com/open-mmlab/mmengine/pull/562 +- @austinmw made their first contribution in https://github.com/open-mmlab/mmengine/pull/579 +- @yhna940 made their first contribution in https://github.com/open-mmlab/mmengine/pull/569 +- @liuyanyi made their first contribution in https://github.com/open-mmlab/mmengine/pull/539 diff --git a/docs/zh_cn/tutorials/dataset.md b/docs/zh_cn/tutorials/dataset.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/zh_cn/tutorials/runner.md b/docs/zh_cn/tutorials/runner.md index d32bf065..4d981df1 100644 --- a/docs/zh_cn/tutorials/runner.md +++ b/docs/zh_cn/tutorials/runner.md @@ -2,7 +2,7 @@ 深度学习算法的训练、验证和测试通常都拥有相似的流程，因此 MMEngine 提供了执行器以帮助用户简化这些任务的实现流程。用户只需要准备好模型训练、验证、测试所需要的模块构建执行器，便能够通过简单调用执行器的接口来完成这些任务。用户如果需要使用这几项功能中的某一项，只需要准备好对应功能所依赖的模块即可。 -用户可以手动构建这些模块的实例，也可以通过编写[配置文件](./config.md)，由执行器自动从[注册器](./registry.md)中构建所需要的模块，我们推荐使用后一种方式。 +用户可以手动构建这些模块的实例，也可以通过编写[配置文件](../advanced_tutorials/config.md)，由执行器自动从[注册器](../advanced_tutorials/registry.md)中构建所需要的模块，我们推荐使用后一种方式。 ## 手动构建模块来使用执行器 @@ -236,7 +236,7 @@ resume = False 一个完整的配置文件主要由模型、数据、优化器、参数调度器、评测器等模块的配置，训练、验证、测试等流程的配置，还有执行流程过程中的各种钩子模块的配置，以及环境和日志等其他配置的字段组成。通过配置文件构建的执行器采用了懒初始化 (lazy initialization)，只有当调用到训练或测试等执行函数时，才会根据配置文件去完整初始化所需要的模块。 -关于配置文件的更详细的使用方式，请参考[配置文件教程](./config.md) +关于配置文件的更详细的使用方式，请参考[配置文件教程](../advanced_tutorials/config.md) ## 加载权重或恢复训练 diff --git a/mmengine/config/config.py b/mmengine/config/config.py index e29260a3..254b352a 100644 --- a/mmengine/config/config.py +++ b/mmengine/config/config.py @@ -263,7 +263,7 @@ class Config: cfg.work_dir # ". /work_dir/config_setting1" - For details, Please refer to docs/zh_cn/tutorials/config.md . + For details, Please refer to docs/zh_cn/advanced_tutorials/config.md . Args: filename (str): Filename of config.