AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks

Zhenyu, Liu; Huizhi, Liang; Xinrun, Li; Vaclav, Snasel; Varun, Ojha

doi:10.1007/978-981-95-5699-1_13

AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks

Lookup NU author(s): Dr Zhenyu Liu, Dr Huizhi Liang ORCiD, Xinrun Li, Dr Varun Ojha ORCiD

Downloads

Accepted version [.pdf]

Licence

This is the authors' accepted manuscript of a conference proceedings (inc. abstract) that has been published in its final definitive form by Springer, 2026.

For re-use rights please refer to the publisher's terms and conditions.

Abstract

Adversarial distillation (AD) is a knowledge distillation technique that facilitates the transfer of robustness from teacher deep neural network (DNN) models to lightweight target (student) DNN models, enabling the target models to perform better than only training the student model independently. Some previous works focus on using a small, learnable teacher (guide) model to improve the robustness of a student model. Since a learnable guide model starts learning from scratch, maintaining its optimal state for effective knowledge transfer during co-training is challenging. Therefore, we propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method. Our method, AdaGAT, dynamically adjusts the training state of the guide model to install robustness to the target model. Specifically, we develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state. We evaluated our approach via extensive experiments on three datasets: CIFAR-10, CIFAR-100, and TinyImageNet, using the WideResNet-34-10 model as the target model. Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model’s robustness across various adversarial attacks compared to a variety of baseline models.