Local detection of occlusion boundaries in video

摘要

Occlusion boundaries are notoriously difficult for many patch-based computer vision algorithms, but they also provide potentially useful information about scene structure and shape. Using short video clips, we present a novel method for scoring the degree to which occlusion is visible at detected edges. We first utilise a spatio-temporal edge detector which estimates edge strength, orientation, and normal motion. By then extracting patches from either side of each detected (possibly moving) edge pixel, we can estimate and compare motion to determine if occlusion is present. In experiments on synthetic and natural images, we demonstrate our ability to differentiate occlusion boundary pixels from simple edge pixels by using motion information. In terms of precision versus recall, our occlusion scoring metric outperforms a rank-based motion inconsistency measure from the literature. The completely local, bottom-up approach described here is intended to provide powerful low-level information for use by higher-level reasoning methods.