文献类型：期刊文献

英文题名：Unleashing the Power of Each Distilled Image

作者：Zhang, Jingxuan[1];Chen, Zhihua[1];Dai, Lei[1]

机构：[1]East China Univ Sci & Technol, Dept Comp Sci & Engn, Shanghai 200237, Peoples R China

年份：2025

卷号：34

起止页码：7050

外文期刊名：IEEE TRANSACTIONS ON IMAGE PROCESSING

收录：;EI(收录号：20254419434632);WOS:【SCI-EXPANDED(收录号:WOS:001615337100007)】；

基金：This work was supported by the National Natural Science Foundation of China under Grant 62572188, Grant 62272164, and Grant 62306113. The

语种：英文

外文关键词：Training; Synthetic data; Artificial neural networks; Overfitting; Data models; Computational modeling; Generative adversarial networks; Feature extraction; Bidirectional control; Accuracy; Dataset distillation; dynamic dataset pruning; knowledge distillation; image classification

摘要：Dataset distillation (DD) aims to accelerate the training speed of neural networks (NNs) by synthesizing a reduced dataset. NNs trained on the smaller dataset are expected to obtain almost the same test set accuracy as they do on the larger one. Previous DD research treated the obtained distilled dataset as a regular dataset for training, neglecting the overfitting issue caused by the limited number of original distilled images. In this paper, we propose a new DD paradigm. Specifically, in the deployment stage, distilled images are augmented by amplifying their local information since the teacher network can produce diverse supervision signals when receiving inputs from different regions. Efficient and diverse augmentation methods for each distilled image are devised, while ensuring the authenticity of augmented samples. Additionally, to alleviate the increased training cost caused by data augmentation, we design a bi-directional dynamic dataset pruning technique to prune the original distilled dataset and augmented distilled dataset. A new pruning strategy and scheduling are proposed based on experimental findings. Experiments on 9 benchmark datasets (CIFAR10, CIFAR100, ImageWoof, ImageCat, ImageFruit, ImageNette, ImageNet10, ImageNet100 and ImageNet1K) demonstrate the effectiveness of our approach. For instance, on the ImageNet1K dataset with a ResNet18 architecture and 50 distilled images per class, our algorithm surpasses the second-ranked MiniMax algorithm by 7.6%, achieving a distilled accuracy of 66.2%.