60469c0a58
[Feature] Add mask channel in MGD Loss ( #461 )
...
* [Feature]
Add mask channel in MGD Loss
* fix lint
---------
Co-authored-by: pppppM <67539920+pppppM@users.noreply.github.com>
Co-authored-by: pppppM <gjf_mail@126.com>
2023-03-01 18:34:38 +08:00
0919f69287
[feature] Support for specifying epochs to stop knowledge distillation ( #455 )
...
* support distillation early stop
* add norm arg to ChannelWiseDivergence
* add norm connector and support a list of connectors
* delete useless codes in cwd because of the usage of the norm connector
* fix fpn distill
* fix pytest
* rename stop distillation hook
* rename stop_epoch and add doc
* rename
* replace = with >=
* set _is_init private attribute of the teacher model to True after loading checkpoint
2023-03-01 17:10:22 +08:00
a27952dbb1
[Improvement] Update NasMutator to build search_space in NAS ( #426 )
...
* update space_mixin
* update NAS algorithms with SpaceMixin
* update pruning algorithms with SpaceMixin
* fix ut
* fix comments
* revert _load_fix_subnet_by_mutator
* fix dcff test
* add ut for registry
* update autoslim_greedy_search
* fix repeat-mutables bug
* fix slice_weight in export_fix_subnet
* Update NasMutator:
1. unify mutators for NAS algorithms as the NasMutator;
2. regard ChannelMutator as pruning-specified;
3. remove value_mutators & module_mutators;
4. set GroupMixin only for NAS;
5. revert all changes in ChannelMutator.
* update NAS algorithms using NasMutator
* update channel mutator
* update one_shot_channel_mutator
* fix comments
* update UT for NasMutator
* fix isort version
* fix comments
---------
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: liukai <your_email@abc.example>
2023-02-01 22:51:38 +08:00
5ebf839a30
[CodeCamp #122 ] Support KD algorithm MGD for detection. ( #377 )
...
* [Feature] Support KD algorithm MGD for detection.
* use connector to beauty mgd.
* fix typo, add unitest.
* fix mgd loss unitest.
* fix mgd connector unitest.
* add model pth and log file.
* add mAP.
2023-01-03 17:21:42 +08:00
42e8de73af
[Improvement] Adapt OFA series with SearchableMobileNetV3 ( #385 )
...
* fix mutable bug in AttentiveMobileNetV3
* remove unness code
* update ATTENTIVE_SUBNET_A0-A6.yaml with optimized names
* unify the sampling usage in sandwich_rule-based NAS
* use alias to export subnet
* update OFA configs
* fix attr bug
* fix comments
* update convert_supernet2subnet.py
* correct the way to dump DerivedMutable
* fix convert index bug
* update OFA configs & models
* fix dynamic2static
* generalize convert_ofa_ckpt.py
* update input_resizer
* update README.md
* fix ut
* update export_fix_subnet
* update _dynamic_to_static
* update fix_subnet UT & minor fix bugs
* fix ut
* add new autoaug compared to attentivenas
* clean
* fix act
* fix act_cfg
* update fix_subnet
* fix lint
* add docstring
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: aptsunny <aptsunny@tongji.edu.cn>
2022-12-15 22:19:55 +08:00
1c03a07350
Enhance the Abilities of the Tracer for Pruning. ( #371 )
...
* tmp
* add new mmdet models
* add docstring
* pass test and pre-commit
* rm razor tracer
* update fx tracer, now it can automatically wrap methods and functions.
* update tracer passed models
* add warning for torch <1.12.0
fix bug for python3.6
update placeholder to support placeholder.XXX
* fix bug
* update docs
* fix lint
* fix parse_cfg in configs
* restore mutablechannel
* test ite prune algorithm when using dist
* add get_model_from_path to MMModelLibrrary
* add mm models to DefaultModelLibrary
* add uts
* fix bug
* fix bug
* add uts
* add uts
* add uts
* add uts
* fix bug
* restore ite_prune_algorithm
* update doc
* PruneTracer -> ChannelAnalyzer
* prune_tracer -> channel_analyzer
* add test for fxtracer
* fix bug
* fix bug
* PruneTracer -> ChannelAnalyzer
refine
* CustomFxTracer -> MMFxTracer
* fix bug when test with torch<1.12
* update print log
* fix lint
* rm unuseful code
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: jacky <jacky@xx.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: liukai <your_email@abc.example>
2022-12-08 15:59:27 +08:00
b0b3fbdb49
[Feature] Add BigNAS algorithm ( #219 )
...
* add calibrate-bn-statistics
* add test calibrate-bn-statistics
* fix mixins
* fix mixins
* fix mixin tests
* remove slimmable channel mutable and refactor dynamic op
* refact dynamic batch norm
* add progressive dynamic conv2d
* add center crop dynamic conv2d
* refactor dynamic directory
* refactor dynamic sequential
* rename length to depth in dynamic sequential
* add test for derived mutable
* refactor dynamic op
* refactor api of dynamic op
* add derive mutable mixin
* addbignas algorithm
* refactor bignas structure
* add input resizer
* add input resizer to bignas
* move input resizer from algorithm into classifier
* remove compnents
* add attentive mobilenet
* delete json file
* nearly(less 0.2) align inference accuracy with gml
* move mutate seperated in bignas mobilenet backbone
* add zero_init_residual
* add set_dropout
* set dropout in bignas algorithm
* fix registry
* add subnet yaml and nearly align inference accuracy with gml
* add rsb config for bignas
* remove base in config
* add gml bignas config
* convert to iter based
* bignas forward and backward fly
* fix merge conflict
* fix dynamicseq bug
* fix bug and refactor bignas
* arrange configs of bignas
* fix typo
* refactor attentive_mobilenet
* fix channel mismatch due to registion of DerivedMutable
* update bignas & fix se channel mismatch
* add AutoAugmentV2 & remove unness configs
* fix lint
* recover channel assertion in channel unit
* fix a group bug
* fix comments
* add docstring
* add norm in dynamic_embed
* fix search loop & other minor changes
* fix se expansion
* minor change
* add ut for bignas & attentive_mobilenet
* fix ut
* update bignas readme
* rm unness ut & supplement get_placeholder
* fix lint
* fix ut
* add subnet deployment in downstream tasks.
* minor change
* update ofa backbone
* minor fix
* Continued improvements of searchable backbone
* minor change
* drop ratio in backbone
* fix comments
* fix ci test
* fix test
* add dynamic shortcut UT
* modify strategy to fit bignas
* fix test
* fix bug in neck
* fix error
* fix error
* fix yaml
* save subnet ckpt
* merge autoslim_val/test_loop into subnet_val_loop
* move calibrate_bn_mixin to utils
* fix bugs and add docstring
* clean code
* fix register bug
* clean code
* update
Co-authored-by: wangshiguang <wangshiguang@sensetime.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: aptsunny <aptsunny@tongji.edu.cn>
Co-authored-by: sunyue1 <sunyue1@sensetime.com>
2022-12-07 11:28:10 +08:00
76c3773e83
[Feature] Add DCFF ( #295 )
...
* add ChannelGroup (#250 )
* rebase new dev-1.x
* modification for adding config_template
* add docstring to channel_group.py
* add docstring to mutable_channel_group.py
* rm channel_group_cfg from Graph2ChannelGroups
* change choice type of SequentialChannelGroup from float to int
* add a warning about group-wise conv
* restore __init__ of dynamic op
* in_channel_mutable -> mutable_in_channel
* rm abstractproperty
* add a comment about VT
* rm registry for ChannelGroup
* MUTABLECHANNELGROUP -> ChannelGroupType
* refine docstring of IndexDict
* update docstring
* update docstring
* is_prunable -> is_mutable
* update docstring
* fix error in pre-commit
* update unittest
* add return type
* unify init_xxx apit
* add unitest about init of MutableChannelGroup
* update according to reviews
* sequential_channel_group -> sequential_mutable_channel_group
Co-authored-by: liukai <liukai@pjlab.org.cn>
* Add BaseChannelMutator and refactor Autoslim (#289 )
* add BaseChannelMutator
* add autoslim
* tmp
* make SequentialMutableChannelGroup accpeted both of num and ratio as choice. and supports divisior
* update OneShotMutableChannelGroup
* pass supernet training of autoslim
* refine autoslim
* fix bug in OneShotMutableChannelGroup
* refactor make_divisible
* fix spell error: channl -> channel
* init_using_backward_tracer -> init_from_backward_tracer
init_from_fx_tracer -> init_from_fx_tracer
* refine SequentialMutableChannelGroup
* let mutator support models with dynamicop
* support define search space in model
* tracer_cfg -> parse_cfg
* refine
* using -> from
* update docstring
* update docstring
Co-authored-by: liukai <liukai@pjlab.org.cn>
* tmpsave
* migrate ut
* tmpsave2
* add loss collector
* refactor slimmable and add l1-norm (#291 )
* refactor slimmable and add l1-norm
* make l1-norm support convnd
* update get_channel_groups
* add l1-norm_resnet34_8xb32_in1k.py
* add pretrained to resnet34-l1
* remove old channel mutator
* BaseChannelMutator -> ChannelMutator
* update according to reviews
* add readme to l1-norm
* MBV2_slimmable -> MBV2_slimmable_config
Co-authored-by: liukai <liukai@pjlab.org.cn>
* update config
* fix md & pytorch support <1.9.0 in batchnorm init
* Clean old codes. (#296 )
* remove old dynamic ops
* move dynamic ops
* clean old mutable_channels
* rm OneShotMutableChannel
* rm MutableChannel
* refine
* refine
* use SquentialMutableChannel to replace OneshotMutableChannel
* refactor dynamicops folder
* let SquentialMutableChannel support float
Co-authored-by: liukai <liukai@pjlab.org.cn>
* fix ci
* ci fix py3.6.x & add mmpose
* ci fix py3.6.9 in utils/index_dict.py
* fix mmpose
* minimum_version_cpu=3.7
* fix ci 3.7.13
* fix pruning &meta ci
* support python3.6.9
* fix py3.6 import caused by circular import patch in py3.7
* fix py3.6.9
* Add channel-flow (#301 )
* base_channel_mutator -> channel_mutator
* init
* update docstring
* allow omitting redundant configs for channel
* add register_mutable_channel_to_a_module to MutableChannelContainer
* update according to reviews 1
* update according to reviews 2
* update according to reviews 3
* remove old docstring
* fix error
* using->from
* update according to reviews
* support self-define input channel number
* update docstring
* chanenl -> channel_elem
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: jacky <jacky@xx.com>
* support >=3.7
* support py3.6.9
* Rename: ChannelGroup -> ChannelUnit (#302 )
* refine repr of MutableChannelGroup
* rename folder name
* ChannelGroup -> ChannelUnit
* filename in units folder
* channel_group -> channel_unit
* groups -> units
* group -> unit
* update
* get_mutable_channel_groups -> get_mutable_channel_units
* fix bug
* refine docstring
* fix ci
* fix bug in tracer
Co-authored-by: liukai <liukai@pjlab.org.cn>
* update new channel config format
* update pruning refactor
* update merged pruning
* update commit
* fix dynamic_conv_mixin
* update comments: readme&dynamic_conv_mixins.py
* update readme
* move kl softmax channel pooling to op by comments
* fix comments: fix redundant & split README.md
* dcff in ItePruneAlgorithm
* partial dynamic params for fuseconv
* add step_freq & prune_time check
* update comments
* update comments
* update comments
* fix ut
* fix gpu ut & revise step_freq in ItePruneAlgorithm
* update readme
* revise ItePruneAlgorithm
* fix docs
* fix dynamic_conv attr
* fix ci
Co-authored-by: LKJacky <108643365+LKJacky@users.noreply.github.com>
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: zengyi.vendor <zengyi.vendor@sensetime.com>
Co-authored-by: jacky <jacky@xx.com>
2022-11-23 09:55:33 +08:00
fb42405af8
[Feature] Add Autoformer algorithm ( #315 )
...
* update candidates
* update subnet_sampler_loop
* update candidate
* add readme
* rename variable
* rename variable
* clean
* update
* add doc string
* Revert "[Improvement] Support for candidate multiple dimensional search constraints."
* [Improvement] Update Candidate with multi-dim search constraints. (#322 )
* update doc
* add support type
* clean code
* update candidates
* clean
* xx
* set_resource -> set_score
* fix ci bug
* py36 lint
* fix bug
* fix check constrain
* py36 ci
* redesign candidate
* fix pre-commit
* update cfg
* add build_resource_estimator
* fix ci bug
* remove runner.epoch in testcase
* [Feature] Autoformer architecture and dynamicOPs (#327 )
* add DynamicSequential
* dynamiclayernorm
* add dynamic_pathchembed
* add DynamicMultiheadAttention and DynamicRelativePosition2D
* add channel-level dynamicOP
* add autoformer algo
* clean notes
* adapt channel_mutator
* vit fly
* fix import
* mutable init
* remove annotation
* add DynamicInputResizer
* add unittest for mutables
* add OneShotMutableChannelUnit_VIT
* clean code
* reset unit for vit
* remove attr
* add autoformer backbone UT
* add valuemutator UT
* clean code
* add autoformer algo UT
* update classifier UT
* fix test error
* ignore
* make lint
* update
* fix lint
* mutable_attrs
* fix test
* fix error
* remove DynamicInputResizer
* fix test ci
* remove InputResizer
* rename variables
* modify type
* Continued improvements of ChannelUnit
* fix lint
* fix lint
* remove OneShotMutableChannelUnit
* adjust derived type
* combination mixins
* clean code
* fix sample subnet
* search loop fly
* more annotations
* avoid counter warning and modify batch_augment cfg by gy
* restore
* source_value_mutables restriction
* simply arch_setting api
* update
* clean
* fix ut
2022-11-14 13:01:04 +08:00
d37829eb60
[Refactor] Refactor Mutables and Mutators ( #324 )
...
* refactor mutables
* update load fix subnet
* add DumpChosen Typehint
* adapt UTs
* fix lint
* Add GroupMixin to ChannelMutator (temporarily)
* fix type hints
* add GroupMixin doc-string
* modified by comments
* fix type hits
* update subnet format
* fix channel group bugs and add UTs
* fix doc string
* fix comments
* refactor diff module forward
* fix error in channel mutator doc
* fix comments
Co-authored-by: liukai <liukai@pjlab.org.cn>
2022-11-01 12:49:42 +08:00
b4b7e2432a
merge pruning into dev-1.x ( #312 )
...
* add ChannelGroup (#250 )
* rebase new dev-1.x
* modification for adding config_template
* add docstring to channel_group.py
* add docstring to mutable_channel_group.py
* rm channel_group_cfg from Graph2ChannelGroups
* change choice type of SequentialChannelGroup from float to int
* add a warning about group-wise conv
* restore __init__ of dynamic op
* in_channel_mutable -> mutable_in_channel
* rm abstractproperty
* add a comment about VT
* rm registry for ChannelGroup
* MUTABLECHANNELGROUP -> ChannelGroupType
* refine docstring of IndexDict
* update docstring
* update docstring
* is_prunable -> is_mutable
* update docstring
* fix error in pre-commit
* update unittest
* add return type
* unify init_xxx apit
* add unitest about init of MutableChannelGroup
* update according to reviews
* sequential_channel_group -> sequential_mutable_channel_group
Co-authored-by: liukai <liukai@pjlab.org.cn>
* Add BaseChannelMutator and refactor Autoslim (#289 )
* add BaseChannelMutator
* add autoslim
* tmp
* make SequentialMutableChannelGroup accpeted both of num and ratio as choice. and supports divisior
* update OneShotMutableChannelGroup
* pass supernet training of autoslim
* refine autoslim
* fix bug in OneShotMutableChannelGroup
* refactor make_divisible
* fix spell error: channl -> channel
* init_using_backward_tracer -> init_from_backward_tracer
init_from_fx_tracer -> init_from_fx_tracer
* refine SequentialMutableChannelGroup
* let mutator support models with dynamicop
* support define search space in model
* tracer_cfg -> parse_cfg
* refine
* using -> from
* update docstring
* update docstring
Co-authored-by: liukai <liukai@pjlab.org.cn>
* refactor slimmable and add l1-norm (#291 )
* refactor slimmable and add l1-norm
* make l1-norm support convnd
* update get_channel_groups
* add l1-norm_resnet34_8xb32_in1k.py
* add pretrained to resnet34-l1
* remove old channel mutator
* BaseChannelMutator -> ChannelMutator
* update according to reviews
* add readme to l1-norm
* MBV2_slimmable -> MBV2_slimmable_config
Co-authored-by: liukai <liukai@pjlab.org.cn>
* Clean old codes. (#296 )
* remove old dynamic ops
* move dynamic ops
* clean old mutable_channels
* rm OneShotMutableChannel
* rm MutableChannel
* refine
* refine
* use SquentialMutableChannel to replace OneshotMutableChannel
* refactor dynamicops folder
* let SquentialMutableChannel support float
Co-authored-by: liukai <liukai@pjlab.org.cn>
* Add channel-flow (#301 )
* base_channel_mutator -> channel_mutator
* init
* update docstring
* allow omitting redundant configs for channel
* add register_mutable_channel_to_a_module to MutableChannelContainer
* update according to reviews 1
* update according to reviews 2
* update according to reviews 3
* remove old docstring
* fix error
* using->from
* update according to reviews
* support self-define input channel number
* update docstring
* chanenl -> channel_elem
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: jacky <jacky@xx.com>
* Rename: ChannelGroup -> ChannelUnit (#302 )
* refine repr of MutableChannelGroup
* rename folder name
* ChannelGroup -> ChannelUnit
* filename in units folder
* channel_group -> channel_unit
* groups -> units
* group -> unit
* update
* get_mutable_channel_groups -> get_mutable_channel_units
* fix bug
* refine docstring
* fix ci
* fix bug in tracer
Co-authored-by: liukai <liukai@pjlab.org.cn>
* Merge dev-1.x to pruning (#311 )
* [feature] CONTRASTIVE REPRESENTATION DISTILLATION with dataset wrapper (#281 )
* init
* TD: CRDLoss
* complete UT
* fix docstrings
* fix ci
* update
* fix CI
* DONE
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* add UT: CRD_ClsDataset
* init
* TODO: UT test formatting.
* init
* crd dataset wrapper
* update docstring
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
* [Improvement] Update estimator with api revision (#277 )
* update estimator usage and fix bugs
* refactor api of estimator & add inner check methods
* fix docstrings
* update search loop and config
* fix lint
* update unittest
* decouple mmdet dependency and fix lint
Co-authored-by: humu789 <humu@pjlab.org.cn>
* [Fix] Fix tracer (#273 )
* test image_classifier_loss_calculator
* fix backward tracer
* update SingleStageDetectorPseudoLoss
* merge
* [Feature] Add Dsnas Algorithm (#226 )
* [tmp] Update Dsnas
* [tmp] refactor arch_loss & flops_loss
* Update Dsnas & MMRAZOR_EVALUATOR:
1. finalized compute_loss & handle_grads in algorithm;
2. add MMRAZOR_EVALUATOR;
3. fix bugs.
* Update lr scheduler & fix a bug:
1. update param_scheduler & lr_scheduler for dsnas;
2. fix a bug of switching to finetune stage.
* remove old evaluators
* remove old evaluators
* update param_scheduler config
* merge dev-1.x into gy/estimator
* add flops_loss in Dsnas using ResourcesEstimator
* get resources before mutator.prepare_from_supernet
* delete unness broadcast api from gml
* broadcast spec_modules_resources when estimating
* update early fix mechanism for Dsnas
* fix merge
* update units in estimator
* minor change
* fix data_preprocessor api
* add flops_loss_coef
* remove DsnasOptimWrapper
* fix bn eps and data_preprocessor
* fix bn weight decay bug
* add betas for mutator optimizer
* set diff_rank_seed=True for dsnas
* fix start_factor of lr when warm up
* remove .module in non-ddp mode
* add GlobalAveragePoolingWithDropout
* add UT for dsnas
* remove unness channel adjustment for shufflenetv2
* update supernet configs
* delete unness dropout
* delete unness part with minor change on dsnas
* minor change on the flag of search stage
* update README and subnet configs
* add UT for OneHotMutableOP
* [Feature] Update train (#279 )
* support auto resume
* add enable auto_scale_lr in train.py
* support '--amp' option
* [Fix] Fix darts metafile (#278 )
fix darts metafile
* fix ci (#284 )
* fix ci for circle ci
* fix bug in test_metafiles
* add pr_stage_test for github ci
* add multiple version
* fix ut
* fix lint
* Temporarily skip dataset UT
* update github ci
* add github lint ci
* install wheel
* remove timm from requirements
* install wheel when test on windows
* fix error
* fix bug
* remove github windows ci
* fix device error of arch_params when DsnasDDP
* fix CRD dataset ut
* fix scope error
* rm test_cuda in workflows of github
* [Doc] fix typos in en/usr_guides
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
Co-authored-by: P.Huang <37200926+FreakieHuang@users.noreply.github.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: Yang Gao <Gary1546308416AL@gmail.com>
Co-authored-by: humu789 <humu@pjlab.org.cn>
Co-authored-by: whcao <41630003+HIT-cwh@users.noreply.github.com>
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
* Refine pruning branch (#307 )
* [feature] CONTRASTIVE REPRESENTATION DISTILLATION with dataset wrapper (#281 )
* init
* TD: CRDLoss
* complete UT
* fix docstrings
* fix ci
* update
* fix CI
* DONE
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* add UT: CRD_ClsDataset
* init
* TODO: UT test formatting.
* init
* crd dataset wrapper
* update docstring
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
* [Improvement] Update estimator with api revision (#277 )
* update estimator usage and fix bugs
* refactor api of estimator & add inner check methods
* fix docstrings
* update search loop and config
* fix lint
* update unittest
* decouple mmdet dependency and fix lint
Co-authored-by: humu789 <humu@pjlab.org.cn>
* [Fix] Fix tracer (#273 )
* test image_classifier_loss_calculator
* fix backward tracer
* update SingleStageDetectorPseudoLoss
* merge
* [Feature] Add Dsnas Algorithm (#226 )
* [tmp] Update Dsnas
* [tmp] refactor arch_loss & flops_loss
* Update Dsnas & MMRAZOR_EVALUATOR:
1. finalized compute_loss & handle_grads in algorithm;
2. add MMRAZOR_EVALUATOR;
3. fix bugs.
* Update lr scheduler & fix a bug:
1. update param_scheduler & lr_scheduler for dsnas;
2. fix a bug of switching to finetune stage.
* remove old evaluators
* remove old evaluators
* update param_scheduler config
* merge dev-1.x into gy/estimator
* add flops_loss in Dsnas using ResourcesEstimator
* get resources before mutator.prepare_from_supernet
* delete unness broadcast api from gml
* broadcast spec_modules_resources when estimating
* update early fix mechanism for Dsnas
* fix merge
* update units in estimator
* minor change
* fix data_preprocessor api
* add flops_loss_coef
* remove DsnasOptimWrapper
* fix bn eps and data_preprocessor
* fix bn weight decay bug
* add betas for mutator optimizer
* set diff_rank_seed=True for dsnas
* fix start_factor of lr when warm up
* remove .module in non-ddp mode
* add GlobalAveragePoolingWithDropout
* add UT for dsnas
* remove unness channel adjustment for shufflenetv2
* update supernet configs
* delete unness dropout
* delete unness part with minor change on dsnas
* minor change on the flag of search stage
* update README and subnet configs
* add UT for OneHotMutableOP
* [Feature] Update train (#279 )
* support auto resume
* add enable auto_scale_lr in train.py
* support '--amp' option
* [Fix] Fix darts metafile (#278 )
fix darts metafile
* fix ci (#284 )
* fix ci for circle ci
* fix bug in test_metafiles
* add pr_stage_test for github ci
* add multiple version
* fix ut
* fix lint
* Temporarily skip dataset UT
* update github ci
* add github lint ci
* install wheel
* remove timm from requirements
* install wheel when test on windows
* fix error
* fix bug
* remove github windows ci
* fix device error of arch_params when DsnasDDP
* fix CRD dataset ut
* fix scope error
* rm test_cuda in workflows of github
* [Doc] fix typos in en/usr_guides
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
* fix bug when python=3.6
* fix lint
* fix bug when test using cpu only
* refine ci
* fix error in ci
* try ci
* update repr of Channel
* fix error
* mv init_from_predefined_model to MutableChannelUnit
* move tests
* update SquentialMutableChannel
* update l1 mutable channel unit
* add OneShotMutableChannel
* candidate_mode -> choice_mode
* update docstring
* change ci
Co-authored-by: P.Huang <37200926+FreakieHuang@users.noreply.github.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: Yang Gao <Gary1546308416AL@gmail.com>
Co-authored-by: humu789 <humu@pjlab.org.cn>
Co-authored-by: whcao <41630003+HIT-cwh@users.noreply.github.com>
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
Co-authored-by: liukai <liukai@pjlab.org.cn>
Co-authored-by: jacky <jacky@xx.com>
Co-authored-by: P.Huang <37200926+FreakieHuang@users.noreply.github.com>
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
Co-authored-by: Yang Gao <Gary1546308416AL@gmail.com>
Co-authored-by: humu789 <humu@pjlab.org.cn>
Co-authored-by: whcao <41630003+HIT-cwh@users.noreply.github.com>
Co-authored-by: pppppM <gjf_mail@126.com>
Co-authored-by: gaoyang07 <1546308416@qq.com>
Co-authored-by: SheffieldCao <1751899@tongji.edu.cn>
2022-10-10 17:30:25 +08:00
eb25bb7577
[feature] CONTRASTIVE REPRESENTATION DISTILLATION with dataset wrapper ( #281 )
...
* init
* TD: CRDLoss
* complete UT
* fix docstrings
* fix ci
* update
* fix CI
* DONE
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* maintain CRD dataset unique funcs as a mixin
* add UT: CRD_ClsDataset
* init
* TODO: UT test formatting.
* init
* crd dataset wrapper
* update docstring
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-09-13 20:53:43 +08:00
41464c6af3
Regression Benchmark ( #271 )
...
* update benchmark test
* fix circle ci gpu config
* move delivery, recorder, tracer from structures to task modules
* move ops from models to models.architectures
* rename dynamic_op to dynamic_ops
* fix configs and metafiles
* remove some github ci
* fix configs / readme / metafile
Co-authored-by: gaojianfei <gaojianfei@sensetime.com>
2022-09-01 11:54:18 +08:00
1c0da58dae
[Feature] Add FBKD algorithm and torch_connectors ( #248 )
...
* 1.Add FBKD
* 1.Add torch_connector and its ut. 2.Revise readme and fbkd config.
* 1.Revise UT for torch_connectors
* 1.Revise nonlocalblock into a subclass of NonLocal2d in mmcv.cnn
2022-08-29 10:05:32 +08:00
876b2ac033
[Feature] Add Zero-shot Knowledge Transfer via Adversarial Belief Matching ( #241 )
...
* 1.Add ZSKT algorithm with zskt_generator, at_loss. 2.Add teacher_detach in kl_divergence.
* 1.Amend readme. 2.Revise UT bugs of test_graph and test_distill.
* 1.Amend docstring of zskt_generator
* 1.Add torch version judgment in test_distillation_loss.
* 1.Revise defaults of batch_size to 1 in generators. 2.Revise mmcls.data to mmcls.structures
* 1.Rename function "at" to "calc_attention_matrix".
2022-08-24 14:57:35 +08:00
afb95a40e7
[feature] add Factor Transfer ( #237 )
...
* init
* init
* update
* done
* update README
* update README
* update README
* update README
* update README
* fix docstring && fix typing
* update
Co-authored-by: huangpengsheng <huangpengsheng@sensetime.com>
2022-08-23 19:52:35 +08:00
57aec1f730
[Feature] Add DAFL Distillation ( #235 )
...
* 1.Add DAFL, including config, DAFLLoss and readme. 2.Add DataFreeDistillationtillation. 3.Add Generator, including base_generator and dafl_generator. 4.Add get_module_device and set_requires_grad functions in utils.
* 1.Amend the file that report error in mypy test under py37, including gather_tensors, datafree_distillation, base_generator. 2.Revise other linting error.
* 1.Revise some docstrings.
* 1.Add UT for datafreedistillation. 2.Add all typing.hints.
* 1.Add UT for generators and gather_tensors.
* 1.Add assert of batch_size in base_generator
* 1.Isort
Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-08-23 10:47:34 +08:00
c6e8dcd209
[Feature] Add BYOT Distillation ( #232 )
...
* byot connector & distiller
* fix config
* fix connector
* tmpsave
* add byot & kdsoftce loss
* update dev-1.x
* fx wsld
* Update README.md
* Update README.md
* fix md
* add ut & REQUIRE REVIEW part
* fix md
* add SelfDistillValLoop UT
* fix comments
* fix comments v2
* fix comments v3
* add connector_idx=None to ConfigurableDistiller.get_record()
Co-authored-by: zengyi.vendor <zengyi.vendor@sensetime.com>
2022-08-22 14:08:02 +08:00
d190037a5e
[Feature] Add dynamic bricks ( #228 )
...
* add dynamic bricks
* add dynamic conv2d test
* add tests for dynamic linear and dynamic norm
* add docstring for dynamic conv2d
* add docstring for dynamic linear
* add docstring for dynamic batchnorm
* Refactor the dynamic op ( put more logic into the mixin )
* fix UT
* Fix UT ( fileio was moved to mmengine)
* derived mutable adds choices property
* Unify the register interface of mutable in dynamic op
* Unified getter interface of mutable in dynamic op
Co-authored-by: gaojianfei <gaojianfei@sensetime.com>
Co-authored-by: pppppM <gjf_mail@126.com>
2022-08-19 15:15:38 +08:00
a5435d74b9
[Fix] Fix AssertionError in test_connector ( #231 )
...
fix connector UT bug
2022-08-15 14:59:58 +08:00
e88e92cac4
[Feature] Add Activation Boundaries Loss ( #214 )
...
* 1.Add ABLoss and its config, readme and pipeline image. 2.Merge all connectors in general_connector into convconnector.
* 1.Improve convconnector to convmoduleconnecotr which aligns with mmcv. 2.Revise UT of test_connector. 3.Revise config of fitnet and abloss. 4.Revise mmcls import of darts_subnet_head to align with the newest mmcls-dev-1.x.
* 1.Simplify ConvModuleConncetor by ConvModule.
Co-authored-by: zhangzhongyu.vendor < zhangzhongyu.vendor@sensetime.com>
2022-08-05 14:14:47 +08:00
2c640d0586
Reorganize Dirs ( #205 )
...
* reorg dirs
* rename deliver to delivery
2022-07-25 13:28:08 +08:00
ae205ac0c6
Refactor darts ( #204 )
...
* add separate optim wrapper
* refactor darts related modules
* refactor darts algorithm
* fix some bugs
* update darts related modules
* update unittest
* update darts configs
2022-07-25 09:52:39 +08:00
5ddfed5040
[Feature] Add slimmable algorithm
2022-07-15 23:05:07 +08:00
6c920c88ee
Align SPOS and DetNAS to MMRazor2.0
2022-07-15 23:04:38 +08:00
2d5e8bc675
Refactor subnet sampler
2022-07-15 23:04:38 +08:00
0939aecdb6
refactor name of mutable module
2022-07-15 23:04:38 +08:00
56afc69d85
Refactor DartsBackbone
2022-07-15 23:04:13 +08:00
99e7993376
Refactor backbone
2022-07-15 23:04:13 +08:00
Ming-Hsuan-Tu
whcao
Yang Gao
Xianpan Zhou
LKJacky
qiufeng
zengyi
Yue Sun
pppppM
P.Huang
zhongyu zhang
PJDong
humu789