文献类型：会议论文

英文题名：Semantic-guided Masked Mutual Learning for Multi-modal Brain Tumor Segmentation with Arbitrary Missing Modalities

作者：Liang, Guoyan[1,2];Zhou, Qin[3];Wang, Zhe[3];Chen, Jingyuan[1];Gu, Lin[4];Yao, Chang[1,2];Wu, Sai[1,2];Huang, Bingcang[5];Chen, Kai[5]

机构：[1]Zhejiang Univ, Hangzhou, Peoples R China;[2]Hangzhou High Tech Zone Binjiang Inst Blockchain, Hangzhou, Peoples R China;[3]ECUST, Dept Comp Sci & Engn, Shanghai, Peoples R China;[4]Univ Tokyo, RIKEN AIP, Tokyo, Japan;[5]Gongli Hosp Shanghai Pudong New Area, Shanghai, Peoples R China

会议论文集：39th AAAI Conference on Artificial Intelligence

会议日期：FEB 25-MAR 04, 2025

会议地点：Philadelphia, PA

语种：英文

摘要：Malignant brain tumors have become an aggressive and dangerous disease that leads to death worldwide. Multi-modal MRI data is crucial for accurate brain tumor segmentation, but missing modalities common in clinical practice can severely degrade the segmentation performance. While incomplete multi-modal learning methods attempt to address this, learning robust and discriminative features from arbitrary missing modalities remains challenging. To address this challenge, we propose a novel Semantic-guided Masked Mutual Learning (SMML) approach to distill robust and discriminative knowledge across diverse missing modality scenarios. Specifically, we propose a novel dual-branch masked mutual learning scheme guided by Hierarchical Consistency Constraints (HCC) to ensure multi-level consistency, thereby enhancing mutual learning in incomplete multi-modal scenarios. The HCC framework comprises a pixel-level constraint that selects and exchanges reliable knowledge to guide the mutual learning process. Additionally, it includes a feature-level constraint that uncovers robust inter-sample and interclass relational knowledge within the latent feature space. To further enhance multi-modal learning from missing modality data, we integrate a refinement network into each student branch. This network leverages semantic priors from the Segment Anything Model (SAM) to provide supplementary information, effectively complementing the masked mutual learning strategy in capturing auxiliary discriminative knowledge. Extensive experiments on three challenging brain tumor segmentation datasets demonstrate that our method significantly improves performance over state-of-the-art methods in diverse missing modality settings.