Human-Centric Image Captioning

作者：

Highlights：

• We propose a new task of Human-Centric Image Captioning and build a dataset - HC-COCO.

• We introduce the Human-Centric Feature Hierarchization to hierarchize image features more explicitly for human-centric captioning by incorporating human body part information.

• We propose a novel three-branch architecture for the separate information flow control and optimization, which helps generating more detailed captions for human activities.

• Our proposed method achieves state-of-the-art performance on HC-COCO, outperforming the previous state of the art by a clear margin.

摘要

•We propose a new task of Human-Centric Image Captioning and build a dataset - HC-COCO.•We introduce the Human-Centric Feature Hierarchization to hierarchize image features more explicitly for human-centric captioning by incorporating human body part information.•We propose a novel three-branch architecture for the separate information flow control and optimization, which helps generating more detailed captions for human activities.•Our proposed method achieves state-of-the-art performance on HC-COCO, outperforming the previous state of the art by a clear margin.

论文关键词：Human-centric,Image captioning,Feature hierarchization

论文评审过程：Received 9 February 2021, Revised 1 January 2022, Accepted 21 January 2022, Available online 22 January 2022, Version of Record 6 February 2022.

论文官网地址：https://doi.org/10.1016/j.patcog.2022.108545