Research questionHow can contrastive distillation efficiently transfer representations to smaller students without memory banks or fixed temperatures?Contrastive distillation can improve student representations, but negative sampling may require external memory banks and fixed temperatures may not suit different training stages or teacher–student pairs. Latest papersRecent research connected to this question, newest first.Discriminative and Consistent Representation DistillationThe source addresses representation distillation for image classification, object detection, and cross-dataset transfer. It proposes in-batch sampling, cross-model similarity consistency, and learnable scale and bias parameters, with experiments on CIFAR-100, ImageNet, MS-COCO, STL-10, and Tiny ImageNet.research paper · Sep 2, 2026