文献类型：期刊文献

英文题名：Freezing partial source representations matters for image inpainting under limited data

作者：Zhang, Yanbing[1,2];Yang, Mengping[1,2];Xiao, Ting[1];Wang, Zhe[1];Chi, Ziqiu[1,2]

机构：[1]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China;[2]East China Univ Sci & Technol, Dept Comp Sci & Engn, Shanghai 200237, Peoples R China

年份：2024

卷号：133

外文期刊名：ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE

收录：;EI(收录号：20240815609034);WOS:【SCI-EXPANDED(收录号:WOS:001187232300001)】；

基金：This work is supported by Natural Science Foundation of China under Grant No. 62076094, Shanghai Science and Technology Program "Federated based cross-domain and cross-task incremental learning" under Grant No. 21511100800, Chinese Defense Program of Science and Technology under Grant No. 2021-JCJQ-JJ-0041, and China Aerospace Science and Technology Corporation Industry-University-Research Cooperation Foundation of the Eighth Research Institute under Grant No. SAST2021-007.

语种：英文

外文关键词：Image inpainting; Transfer learning; Limited data

摘要：Recent years have seen significant advances in image inpainting for any shape of missing regions. However, the performance of existing methods degrades drastically when insufficient data is given (e.g., 100), which has drawn limited attention in the community. This work provides an appropriate solution for image inpainting on the challenging limited data regime. Specifically, we first make an in-depth comparison of fine-tuning and training from scratch and find that, although the former advances the performance than the latter, the overall structural consistency and fine details are still unsatisfactory. Consequently, we propose a two -stage method based on transfer learning, namely T2inpaint. To capture the global structures of the target domain, we exclusively refine the domain -specific weights during the first stage, directing the model's attention towards the acquisition of high-level features. Subsequently, in the second stage, we train additional parameters integrated into the frozen model from the first phase. This approach aims to attain detailed textures while concurrently alleviating overfitting. As a result, the reusable knowledge from the source domain plays a crucial role in guiding the optimization process, preventing the inclusion of ambiguous content. Extensive experiments conducted on various low -data regime datasets demonstrate that our T2inpaint produces plausible images, achieving state-of-the-art performance, particularly in scenarios with training data below 100 samples. Meanwhile, rich ablation studies elucidate the nuanced aspects of our approach. Moreover, an empirical study on the source domains, data regimes, and various data augmentation is conducted, facilitating potential interesting works.