Commit Graph

3 Commits (d9932c08b5108989f94ca1029a439d2487d1f3b3)

Author SHA1 Message Date
Francisco Massa 8eae3269da
Add Knowledge-Distillation (#42)
* Add knowledge distillation

* Bugfix

* Bugfix

* Make names more readable and use single torch.cat call

* Remove criterion.train() in engine

The teacher should stay in eval mode

* Change default argument for teacher-model

* Return the average of classifiers during inference

* Cleanup unused code

* Add docstring for DistillationLoss

* Remove warnings from newer PyTorch

Also uses more stable variant, instead of using softmax + log, use directly log_softmax
2021-01-13 14:19:23 +01:00
Francisco Massa 0c4b8f60bd Change LICENSE to Apache 2.0 2021-01-08 10:51:58 +01:00
Francisco Massa 1d38fa4c37 Initial commit 2020-12-23 10:47:58 -08:00