Browse by author
Lookup NU author(s): Dr Zhenyu Liu, Dr Huizhi LiangORCiD, Xinrun Li, Dr Varun OjhaORCiD
This is the authors' accepted manuscript of a conference proceedings (inc. abstract) published in its final definitive form in 2025. For re-use rights please refer to the publishers terms and conditions.
Adversarial distillation (AD) is a knowledge distillation technique that facilitates the transfer of robustness from teacher deep neural network (DNN) models to lightweight target (student) DNN models, enabling the target models to perform better than only training the student model independently. Some previous works focus on using a small, learnable teacher (guide) model to improve the robustness of a student model. Since a learnable guide model starts learning from scratch, maintaining its optimal state for effective knowledge transfer during co-training is challenging. Therefore, we propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method. Our method, AdaGAT, dynamically adjusts the training state of the guide model to install robustness to the target model. Specifically, we develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state. We evaluated our approach via extensive experiments on three datasets: CIFAR-10, CIFAR-100, and TinyImageNet, using the WideResNet-34-10 model as the target model. Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model’s robustness across various adversarial attacks compared to a variety of baseline models.
Author(s): Zhenyu Liu, Huizhi Liang, Xinrun Li, Vaclav Snasel, Varun Ojha
Publication type: Conference Proceedings (inc. Abstract)
Publication status: In Press
Conference Name: The 8th Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2025)
Year of Conference: 2025
Acceptance date: 22/08/2025
Date deposited: 18/09/2025
