20 Commits

Author SHA1 Message Date
Yang Gao
8d603d917e
[Feature] Add Dsnas Algorithm (#226)
* [tmp] Update Dsnas

* [tmp] refactor arch_loss & flops_loss

* Update Dsnas & MMRAZOR_EVALUATOR:
1. finalized compute_loss & handle_grads in algorithm;
2. add MMRAZOR_EVALUATOR;
3. fix bugs.

* Update lr scheduler & fix a bug:
1. update param_scheduler & lr_scheduler for dsnas;
2. fix a bug of switching to finetune stage.

* remove old evaluators

* remove old evaluators

* update param_scheduler config

* merge dev-1.x into gy/estimator

* add flops_loss in Dsnas using ResourcesEstimator

* get resources before mutator.prepare_from_supernet

* delete unness broadcast api from gml

* broadcast spec_modules_resources when estimating

* update early fix mechanism for Dsnas

* fix merge

* update units in estimator

* minor change

* fix data_preprocessor api

* add flops_loss_coef

* remove DsnasOptimWrapper

* fix bn eps and data_preprocessor

* fix bn weight decay bug

* add betas for mutator optimizer

* set diff_rank_seed=True for dsnas

* fix start_factor of lr when warm up

* remove .module in non-ddp mode

* add GlobalAveragePoolingWithDropout

* add UT for dsnas

* remove unness channel adjustment for shufflenetv2

* update supernet configs

* delete unness dropout

* delete unness part with minor change on dsnas

* minor change on the flag of search stage

* update README and subnet configs

* add UT for OneHotMutableOP
2022-09-29 16:48:47 +08:00
pppppM
179bd5287d
[Fix] Adapt latest mmcv (#253)
* Adapt to the latest mmcv and mmengine

* fixed ut_subnet_sampler_loop

* fix get_model

* fix lints

Co-authored-by: humu789 <humu@pjlab.org.cn>
2022-08-29 20:34:51 +08:00
P.Huang
5d9fcd8070
[feature] add A Comprehensive Overhaul of Feature Distillation (#244)
* init

* init

* linting

* add README

* add wrn docstrings

* add vanilla wrn configs

* fix UT

* OFD DONE

* update OFD readme

* update config path

* rename vanilla model files

* rename vanilla models config files

Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-08-29 14:06:58 +08:00
zhongyu zhang
876b2ac033
[Feature] Add Zero-shot Knowledge Transfer via Adversarial Belief Matching (#241)
* 1.Add ZSKT algorithm with zskt_generator, at_loss. 2.Add teacher_detach in kl_divergence.

* 1.Amend readme. 2.Revise UT bugs of test_graph and test_distill.

* 1.Amend docstring of zskt_generator

* 1.Add torch version judgment in test_distillation_loss.

* 1.Revise defaults of batch_size to 1 in generators. 2.Revise mmcls.data to mmcls.structures

* 1.Rename function "at" to "calc_attention_matrix".
2022-08-24 14:57:35 +08:00
wang shiguang
ba71abf357
[fix] fix mmcv mmengine (#242)
* align_with_mmcv_and_mmengine

* fix_mmcv.fileio
2022-08-24 09:58:11 +08:00
zhongyu zhang
57aec1f730
[Feature] Add DAFL Distillation (#235)
* 1.Add DAFL, including config, DAFLLoss and readme. 2.Add DataFreeDistillationtillation. 3.Add Generator, including base_generator and dafl_generator. 4.Add get_module_device and set_requires_grad functions in utils.

* 1.Amend the file that report error in mypy test under py37, including gather_tensors, datafree_distillation, base_generator. 2.Revise other linting error.

* 1.Revise some docstrings.

* 1.Add UT for datafreedistillation. 2.Add all typing.hints.

* 1.Add UT for generators and gather_tensors.

* 1.Add assert of batch_size in base_generator

* 1.Isort

Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-08-23 10:47:34 +08:00
zengyi
c6e8dcd209
[Feature] Add BYOT Distillation (#232)
* byot connector & distiller

* fix config

* fix connector

* tmpsave

* add byot & kdsoftce loss

* update dev-1.x

* fx wsld

* Update README.md

* Update README.md

* fix md

* add ut & REQUIRE REVIEW part

* fix md

* add SelfDistillValLoop UT

* fix comments

* fix comments v2

* fix comments v3

* add connector_idx=None to ConfigurableDistiller.get_record()

Co-authored-by: zengyi.vendor <zengyi.vendor@sensetime.com>
2022-08-22 14:08:02 +08:00
qiufeng
d190037a5e
[Feature] Add dynamic bricks (#228)
* add dynamic bricks

* add dynamic conv2d test

* add tests for dynamic linear and dynamic norm

* add docstring for dynamic conv2d

* add docstring for dynamic linear

* add docstring for dynamic batchnorm

* Refactor the dynamic op ( put more logic into the mixin )

* fix UT

* Fix UT ( fileio was moved to mmengine)

* derived mutable adds choices property

* Unify the register interface of mutable in dynamic op

* Unified getter interface of mutable in dynamic op

Co-authored-by: gaojianfei <gaojianfei@sensetime.com>
Co-authored-by: pppppM <gjf_mail@126.com>
2022-08-19 15:15:38 +08:00
PJDong
696191e0c0
[Refactor] Move build_arch_param from DiffMutableModule to DiffModuleMutator (#221)
* move build_arch_param from mutable to mutator

* fix UT of diff mutable and mutator

* modify based on shiguang's comments

* remove mutator from the unittest of mutable
2022-08-10 10:05:32 +08:00
qiufeng
6987511e6b
[Fix] Fix mmcls import error (#206)
* fix mmcls import error

* fix __init__.py
2022-07-26 15:33:11 +08:00
pppppM
ae205ac0c6
Refactor darts (#204)
* add separate optim wrapper

* refactor darts related modules

* refactor darts algorithm

* fix some bugs

* update darts related modules

* update unittest

* update darts configs
2022-07-25 09:52:39 +08:00
qiufeng
5ddfed5040 [Feature] Add slimmable algorithm 2022-07-15 23:05:07 +08:00
pppppM
a4c76ae47e [Refactor] Add distiller 2022-07-15 23:04:38 +08:00
pppppM
cb238e36e3 [Feature] Add Recorder to improve Distiller 2022-07-15 23:04:38 +08:00
PJDong
0939aecdb6 refactor name of mutable module 2022-07-15 23:04:38 +08:00
pppppM
577a3a2a94 [Refactor] Base Algorithm 2022-07-15 23:04:13 +08:00
pppppM
baa8c8614e [Deprecated] Clean up code that will be deprecated in OpenMMLab 2.0 2022-07-15 23:02:37 +08:00
pppppM
49f1bee45b
Bump version to v0.3.0 (#135)
* [Feature] Add function to meet mmdeploy support (#102)

* add init_model function for mmdeploy

* fix lint

* add unittest for init_xxx_model

* fix lint

* mv test_inference.py to test_apis directory

* [Feature] Add function to meet mmdeploy support (#102)

* add init_model function for mmdeploy

* fix lint

* add unittest for init_xxx_model

* fix lint

* mv test_inference.py to test_apis directory

* [Refactor] Delete redundant `set_random_seed` function (#104)

* refactor set_random_seed

* add unittests

* fix unittests error

* fix lint

* avoid bc breaking

* [Feature] Add diff seeds to diff ranks and set torch seed in worker_init_fn (#113)

* add init_random_seed

* Set diff seed to diff workers

* [Feature] Add multi machine dist_train (#114)

* support multi nodes

* update training doc

* fix lints

* remove fixed seed

* fix ddp wrapper registry (#128)

* [Docs] Add brief installation steps in README(_zh-CN).md (#121)

* Add brief installation

* add brief installtion ref to mmediting pr#816

Co-authored-by: caoweihan <caoweihan@sensetime.com>

* [BUG]Fix bugs in pruner (#126)

* fix bugs in pruner when pruning models with shared modules

* pruner can trace models with dilation conv2d

* fix deploy_subnet

* fix add_pruning_attrs

* fix bugs in modify_forward

* fix lint

* fix StructurePruner

* test tracing models with shared modules

Co-authored-by: caoweihan <caoweihan@sensetime.com>

* [Docs]Add some more details to docs (#133)

* add docs for dataset

* add cfg-options for distillation

* fix docs

Co-authored-by: caoweihan <caoweihan@sensetime.com>

* reset norm running status after prepare_from_supernet (#81)

* [Improvement]Sync train api (#115)

Co-authored-by: caoweihan <caoweihan@sensetime.com>

* [Feature]Support Relational Knowledge Distillation (#127)

* add rkd

* add rkd pytest

* add rkd configs

* fix readme

* fix rkd

* split rkd loss to distance-wise and angle-wise losses

* rename rkd losses

* add rkd metaflie

* add rkd related links

* rename rkd metafile and add to model index

* delete cifar100

Co-authored-by: caoweihan <caoweihan@sensetime.com>
Co-authored-by: pppppM <gjf_mail@126.com>

Co-authored-by: qiufeng <44188071+wutongshenqiu@users.noreply.github.com>
Co-authored-by: wutongshenqiu <690364065@qq.com>
Co-authored-by: whcao <41630003+HIT-cwh@users.noreply.github.com>
Co-authored-by: caoweihan <caoweihan@sensetime.com>
2022-04-02 19:30:50 +08:00
pppppM
ef1637e866
[FEATURE] Mobilenet Series Search Space (#82)
* add mbv2 block and identity

* add mbv2 block and identity unittests

* expand_factor -> expand_ratio

* add searchable mobilenet v2

* fix lints

* add spos subnet retraining config

* fix mmcls slurm search

* add proxyless_gpu setting for mbv2

* use bn default

* add angelnas spos config

* update spos readme

* fix SELayer's useage

* add docstring

* rename mbv2 to mb

* add some unittest of mb

* rename mb to mobilenet

* add some rename-mb in configs

* update README of spos

* add rename-mb in unittest

* update test_mmcls

Co-authored-by: wutongshenqiu <690364065@qq.com>
Co-authored-by: humu789 <humu@pjlab.org.cn>
2022-03-07 23:15:18 +08:00
whcao
f7fbe770a6
add pytest (#18)
Co-authored-by: caoweihan <caoweihan@sensetime.com>
2021-12-23 11:29:12 +08:00