14 Commits

Author SHA1 Message Date
whcao
86c61539b1
[Feature] PyTorch version of PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient. (#304)
* add pkd

* add pytest for pkd

* fix cfg

* WIP: support fcos3d

* WIP: support fcos3d pkd

* support mmdet3d

* fix cfgs

* change eps to 1e-6 and add some comments

* fix docstring

* fix cfg

* add assert

* add type hint

* WIP: add readme and metafile

* fix readme

* update metafiles and readme

* fix metafile

* fix pipeline figure
2022-10-26 19:18:20 +08:00
LKJacky
f98ac3416b
fix ci (#284)
* fix ci for circle ci

* fix bug in test_metafiles

* add  pr_stage_test for github ci

* add multiple version

* fix ut

* fix lint

* Temporarily skip dataset UT

* update github ci

* add github lint ci

* install wheel

* remove timm from requirements

* install wheel when test on windows

* fix error

* fix bug

* remove github windows ci

* fix device error of arch_params when DsnasDDP

* fix CRD dataset ut

* fix scope error

* rm test_cuda in workflows of github

* [Doc] fix typos in en/usr_guides

Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
2022-10-10 10:06:57 +08:00
P.Huang
eb25bb7577
[feature] CONTRASTIVE REPRESENTATION DISTILLATION with dataset wrapper (#281)
* init

* TD: CRDLoss

* complete UT

* fix docstrings

* fix ci

* update

* fix CI

* DONE

* maintain CRD dataset unique funcs as a mixin

* maintain CRD dataset unique funcs as a mixin

* maintain CRD dataset unique funcs as a mixin

* add UT: CRD_ClsDataset

* init

* TODO: UT test formatting.

* init

* crd dataset wrapper

* update docstring

Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-09-13 20:53:43 +08:00
zhongyu zhang
f69aeabc69
[Docs] Add metafiles for KD algos (ABLoss, DAFL, DFAD, FBKD, FitNets, ZSKT) (#266)
1.Add metafiles for 6 kd algos.
2.Add model and log links. 
3.Revise data_samples in datafreedistillation for new feature of mmengine.
2022-08-31 22:12:25 +08:00
P.Huang
5d9fcd8070
[feature] add A Comprehensive Overhaul of Feature Distillation (#244)
* init

* init

* linting

* add README

* add wrn docstrings

* add vanilla wrn configs

* fix UT

* OFD DONE

* update OFD readme

* update config path

* rename vanilla model files

* rename vanilla models config files

Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-08-29 14:06:58 +08:00
zhongyu zhang
1c0da58dae
[Feature] Add FBKD algorithm and torch_connectors (#248)
* 1.Add FBKD

* 1.Add torch_connector and its ut. 2.Revise readme and fbkd config.

* 1.Revise UT for torch_connectors

* 1.Revise nonlocalblock into a subclass of NonLocal2d in mmcv.cnn
2022-08-29 10:05:32 +08:00
zhongyu zhang
f3b964c521
[Feature] Add DFAD algorithm (#247)
1.Add DFAD algorithm. 2.Add L1Loss and its UT.
2022-08-25 15:46:45 +08:00
zhongyu zhang
876b2ac033
[Feature] Add Zero-shot Knowledge Transfer via Adversarial Belief Matching (#241)
* 1.Add ZSKT algorithm with zskt_generator, at_loss. 2.Add teacher_detach in kl_divergence.

* 1.Amend readme. 2.Revise UT bugs of test_graph and test_distill.

* 1.Amend docstring of zskt_generator

* 1.Add torch version judgment in test_distillation_loss.

* 1.Revise defaults of batch_size to 1 in generators. 2.Revise mmcls.data to mmcls.structures

* 1.Rename function "at" to "calc_attention_matrix".
2022-08-24 14:57:35 +08:00
P.Huang
afb95a40e7
[feature] add Factor Transfer (#237)
* init

* init

* update

* done

* update README

* update README

* update README

* update README

* update README

* fix docstring && fix typing

* update

Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-08-23 19:52:35 +08:00
zhongyu zhang
57aec1f730
[Feature] Add DAFL Distillation (#235)
* 1.Add DAFL, including config, DAFLLoss and readme. 2.Add DataFreeDistillationtillation. 3.Add Generator, including base_generator and dafl_generator. 4.Add get_module_device and set_requires_grad functions in utils.

* 1.Amend the file that report error in mypy test under py37, including gather_tensors, datafree_distillation, base_generator. 2.Revise other linting error.

* 1.Revise some docstrings.

* 1.Add UT for datafreedistillation. 2.Add all typing.hints.

* 1.Add UT for generators and gather_tensors.

* 1.Add assert of batch_size in base_generator

* 1.Isort

Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-08-23 10:47:34 +08:00
zengyi
c6e8dcd209
[Feature] Add BYOT Distillation (#232)
* byot connector & distiller

* fix config

* fix connector

* tmpsave

* add byot & kdsoftce loss

* update dev-1.x

* fx wsld

* Update README.md

* Update README.md

* fix md

* add ut & REQUIRE REVIEW part

* fix md

* add SelfDistillValLoop UT

* fix comments

* fix comments v2

* fix comments v3

* add connector_idx=None to ConfigurableDistiller.get_record()

Co-authored-by: zengyi.vendor <zengyi.vendor@sensetime.com>
2022-08-22 14:08:02 +08:00
spynccat
a1937fd5a6
[Feature] Add Decoupled KD Loss (#222)
* add DKDLoss, config

* linting

* linting

* reset default reduction

* dkd ut

* Update decoupled_kd.py

* Update decoupled_kd.py

* Update decoupled_kd.py

* fix commit

* fix readme

* fix comments

* linting comment

* rename loss params

* fix docstring

* Update decoupled_kd.py

* fix gt from config

* merge fix

* fix ut & wsld

* Update README.md

* Update README.md

* add Acknowledgement

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* fix readme style

* fix md

Co-authored-by: zengyi.vendor <zengyi.vendor@sensetime.com>
2022-08-15 14:59:24 +08:00
zhongyu zhang
e88e92cac4
[Feature] Add Activation Boundaries Loss (#214)
* 1.Add ABLoss and its config, readme and pipeline image. 2.Merge all connectors in general_connector into convconnector.

* 1.Improve convconnector to convmoduleconnecotr which aligns with mmcv. 2.Revise UT of test_connector. 3.Revise config of fitnet and abloss. 4.Revise mmcls import of darts_subnet_head to align with the newest mmcls-dev-1.x.

* 1.Simplify ConvModuleConncetor by ConvModule.

Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-08-05 14:14:47 +08:00
wilxy
a94aaba986
[Feature] Add connector components and FitNet (#212)
* Fix spelling mistakes

* 1.Rename general connectors. 2.Replace nn.conv2 to build_conv_layer, replace nn.bn to build_norm_layer.

* 1. Rename function init_parameters to init_weights in SingleConvConnector to realize automatically invocation.

* 1. Add norm_cfg in config and general_connector

* 1.Move calculate_student_loss to distillation algorithm. 2.Move mmrazor.models.connector to mmrazor.models.architectures. 3.Merge stu_connectors and tea_connectors into connectors, and call connectors by their connector_name.

* 1.Replace connector_name to connector in record_info. 2.Add assert that each connector must be in connectors.

Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-07-28 19:17:42 +08:00