文献类型：期刊文献

英文题名：DGattGAN: Cooperative Up-Sampling Based Dual Generator Attentional GAN on Text-to-Image Synthesis

作者：Zhang, Han[1];Zhu, Hongqing[1];Yang, Suyi[2];Li, Wenhao[1]

机构：[1]East China Univ Sci & Technol, Sch Informat Sci & Engn, Shanghai 200237, Peoples R China;[2]Kings Coll London, Dept Math Nat Math & Engn Sci, London WC2R 2LS, England

年份：2021

卷号：9

起止页码：29584

外文期刊名：IEEE ACCESS

收录：;EI(收录号：20210809949267);WOS:【SCI-EXPANDED(收录号:WOS:000622085100001)】；

基金：This work was supported by the National Nature Science Foundation of China under Grant 61872143.

语种：英文

外文关键词：Generators; Image resolution; Generative adversarial networks; Gallium nitride; Task analysis; Image synthesis; Visualization; Asymmetric information feeding; cooperative up-sampling; dual generator; generative adversarial networks; text-to-image synthesis

摘要：Text-to-image synthesis task aims at generating images consistent with input text descriptions and is well developed by the Generative Adversarial Network (GAN). Although GAN based image generation approaches have achieved promising results, synthesizing quality is sometimes unsatisfied due to discursive generation of background and object. In this article, we propose a cooperative up-sampling based Dual Generator attentional GAN (DGattGAN) to generate high-quality images from text description. To achieve this, two generators with individual generation purpose are established to decouple object and background generation. In particular, we introduce a cooperative up-sampling mechanism to build cooperation between object and background generators during training. This strategy is potentially very useful as any dual generator architecture in GAN models can benefit from this mechanism. Furthermore, we propose an asymmetric information feeding scheme to distinguish two synthesis tasks, such that each generator only synthesizes based on semantic information they accept. Taking advantage of effective dual generator, the attention mechanism we incorporated on object generator could devote to fine-grained details generation on actual targeted objects. Experiments on Caltech-UCSD Bird (CUB) and Oxford-102 datasets suggest that generated images by the proposed model are more realistic and consistent with input text, and DGattGAN is competent compared to state-of-the-art methods according to Inception Score (IS) and R-precision metrics. Our codes are available at: https://github.com/ecfish/DGattGAN.