Hallucinating uncertain motion and future for static image action recognition

摘要

Static image action recognition, aiming to recognize human action in a single image, is a challenging task due to the lack of motion and temporal information in static images. Therefore, some previous works have leveraged freely available unlabeled videos to assist image action recognition, which can be categorized into the following two research lines: predict motion or future information of static images using the predictor learned from unlabeled videos. In this paper, following the above two research lines, we propose a novel Multi-modal Motion feature Generator (MMG) and a novel Multi-modal Future feature Generator (MFG) to hallucinate multiple plausible motion features and future visual features for a static image, which could significantly facilitate the image action recognition task. Extensive experiments on two video datasets and two static action image datasets demonstrate the effectiveness of our methods.