Commit Graph

6 Commits (53c0a07cae34baf6ea994e18a67e59d871fbf332)

Author SHA1 Message Date
Francisco Massa ab5715372d
Add deit_base_distilled_patch16_384 (#55) 2021-01-18 12:19:47 +01:00
Francisco Massa 277659c937
Add distilled and finetuned models to MODEL_ZOO (#50)
* Add results to README

* Add new model URLs

* Fill table with new models

* Use details in README

* Add URLs in the command-lines

* Add Training commands

* Bugfix

* Add models to hubconf

* Fix models
2021-01-18 10:43:54 +01:00
Francisco Massa a8e90967a3
Add option to finetune on larger resolution (#43)
* Add option for finetuning a model

* Fixes

* Keep model in eval mode during finetuning

* Only skip head weights if size mismatch

* Remove finetune-epochs

Might not be needed

* Raise error if distillation + finetune are enabled
2021-01-15 10:13:52 +01:00
Francisco Massa 8eae3269da
Add Knowledge-Distillation (#42)
* Add knowledge distillation

* Bugfix

* Bugfix

* Make names more readable and use single torch.cat call

* Remove criterion.train() in engine

The teacher should stay in eval mode

* Change default argument for teacher-model

* Return the average of classifiers during inference

* Cleanup unused code

* Add docstring for DistillationLoss

* Remove warnings from newer PyTorch

Also uses more stable variant, instead of using softmax + log, use directly log_softmax
2021-01-13 14:19:23 +01:00
Francisco Massa 0c4b8f60bd Change LICENSE to Apache 2.0 2021-01-08 10:51:58 +01:00
Francisco Massa 1d38fa4c37 Initial commit 2020-12-23 10:47:58 -08:00