Repurposing existing deep networks for caption and aesthetic-guided image cropping

Highlights：

• The core research question of this paper is how can we find the image part described by a user, such that the output image crop will represent and preserve the caption information meanwhile result in an aesthetically pleasing output?

• We have proposed a caption and aesthetics guided framework for cropping images according to the user’s intention. Our framework is the first to account for the user’s intention directly from the provided image caption.

• We argue that the currently available image cropping and caption grounding datasets are not suitable for our description-based image cropping task. Therefore, we proposed a novel dataset with multiple ground truth bounding box annotations for each caption.

• The experiments in Section 4.2 show that we can achieve better performance than the baseline methods for caption-based image cropping by re-proposing existing deep networks.