文献类型：期刊文献

英文题名：Multitask GANs for Semantic Segmentation and Depth Completion With Cycle Consistency

作者：Zhang, Chongzhen[1];Tang, Yang[1];Zhao, Chaoqiang[1];Sun, Qiyu[1];Ye, Zhencheng[1];Kurths, Jurgen[2,3,4]

机构：[1]East China Univ Sci & Technol, Key Lab Smart Mfg Energy Chem Proc, Minist Educ, Shanghai 200237, Peoples R China;[2]Potsdam Inst Climate Impact Res, D-14473 Potsdam, Germany;[3]Humboldt Univ, Inst Phys, D-12489 Berlin, Germany;[4]Sechenov First Moscow State Med Univ, World Class Res Ctr Digital Biodesign & Personali, Ctr Anal Complex Syst, Moscow 119146, Russia

年份：2021

卷号：32

期号：12

起止页码：5404

外文期刊名：IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

收录：;EI(收录号：20212010375183);WOS:【SCI-EXPANDED(收录号:WOS:000724480600019)】；

基金：This work was supported in part by the National Natural Science Foundation of China through the Basic Science Center Program under Grant 61988101, in part by the International (Regional) Cooperation and Exchange Project under Grant 61720106008, in part by the Program of Shanghai Academic Research Leader under Grant 20XD1401300, in part by the Ministry of Science and Higher Education of the Russian Federation within the framework of state support for the creation and development of the World-Class Research Center "Digital Biodesign and Personalized Healthcare" under Grant 075-15-2020-926, and in part by the Programme of Introducing Talents of Discipline to Universities (the 111 Project) under Grant B17017.

语种：英文

外文关键词：Semantics; Task analysis; Image segmentation; Image reconstruction; Buildings; Vegetation mapping; Feature extraction; Depth completion; generative adversarial networks; image-to-image translation; semantic segmentation

摘要：Semantic segmentation and depth completion are two challenging tasks in scene understanding, and they are widely used in robotics and autonomous driving. Although several studies have been proposed to jointly train these two tasks using some small modifications, such as changing the last layer, the result of one task is not utilized to improve the performance of the other one despite that there are some similarities between these two tasks. In this article, we propose multitask generative adversarial networks (Multitask GANs), which are not only competent in semantic segmentation and depth completion but also improve the accuracy of depth completion through generated semantic images. In addition, we improve the details of generated semantic images based on CycleGAN by introducing multiscale spatial pooling blocks and the structural similarity reconstruction loss. Furthermore, considering the inner consistency between semantic and geometric structures, we develop a semantic-guided smoothness loss to improve depth completion results. Extensive experiments on the Cityscapes data set and the KITTI depth completion benchmark show that the Multitask GANs are capable of achieving competitive performance for both semantic segmentation and depth completion tasks.