7 Commits (0a2f60ba0198f8d567b536313bfba329588f9c3f)

Author	SHA1	Message	Date
takuoko	d510b8b174	[Feature] Support LayerScale in FFN (#2451) ... * add layer scale * add layer scale * add layer scale * Update mmcv/cnn/bricks/transformer.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * Update mmcv/cnn/bricks/transformer.py Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com> * add layer scale * move LayerScale * add layer_scale_init_value * add typehint * fix for tensor with any dim * fix layer scale rule * fix layer scale rule * fix test * add docs Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>	2022-12-11 17:58:19 +08:00
Mashiro	19a024155a	[Refactor] Use MODELS registry in mmengine and delete basemodule (#2172) ... * change MODELS to mmengine, delete basemodule * fix unit test * remove build from cfg * add comment and rename TARGET_MODELS to registry * refine cnn docs * remove unnecessary check * refine as comment * refine build_xxx_conv error message * fix lint * fix import registry from mmcv * remove unused file	2022-08-19 11:18:47 +08:00
Zaida Zhou	6e9ce18323	Add copyright pre-commit-hook (#1742) ... * first commit * Add copyright pre-commit-hook	2022-02-24 09:24:25 +08:00
Mashiro	26aba2f533	[Fix] Fix an incorrect assignment in the BaseTransformerLayer (#1670) ... * fix BaseTransformerLayer * Add BaseTransformerLayer unit test without ffn_cfg embed_dims * Optimize code logic	2022-01-28 23:43:53 +08:00
Shilong Zhang	001f3f6610	[Feature]Add patch embed and patch merge (#1305)	2021-12-14 21:52:37 +08:00
Tong Gao	c522b47e6a	fix the wrong function reference bug in BaseTransformerLayer when batch_first is True (#1418)	2021-11-02 20:57:50 +08:00
Shilong Zhang	e05fb56031	Refactor the baseclass related to transformer (#978) ... * minor changes * change to modulist * change to Sequential * replace dropout with attn_drop and proj_drop in MultiheadAttention * add operation_name for attn * add drop path and move all ffn args to ffncfgs * fix typo * fix a bug when use default value of ffn_cfgs * fix ffns * add deprecate warning * fix deprecate warning * change to pop kwargs * support register FFN of transformer * support batch first * fix batch first wapper * fix forward wapper * fix typo * fix lint * add unitest for transformer * fix unitest * fix equal * use allclose * fix comments * fix comments * change configdict to dict * move drop to a file * add comments for drop path * add noqa 501 * move bnc wapper to MultiheadAttention * move bnc wapper to MultiheadAttention * use dep warning * resolve comments * add unitest: * rename residual to identity * revert runner * msda residual to identity * rename inp_identity to identity * fix name * fix transformer * remove key in msda * remove assert for key Co-authored-by: HIT-cwh <2892770585@qq.com> Co-authored-by: bkhuang <congee524@gmail.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>	2021-06-11 18:09:31 +08:00