CANet: Co-attention network for RGB-D semantic segmentation

Highlights：

• We propose a novel CANet for RGB-D semantic segmentation, and the key co-attention fusion part consists of three modules, i.e. the PCFM, the CCFM, and the FCM, where the PCFM and CCFM aggregate the position-wise and channel-wise features of RGB and depth images, and the FCM produces the final fused features by integrating the output features from the PCFM, the CCFM, and the mixture branch.

• We perform extensive experiments on the NYUDv2 and SUN-RGBD datasets, where the CANet significantly improves RGB-D semantic segmentation results, achieving state-ofthe- art performance on these two popular RGB-D benchmarks.