Fine Tuning Dual Streams Deep Network with Multi-scale Pyramid Decision for Heterogeneous Face Recognition

摘要

In this paper, we propose a novel method called fine tuning dual streams deep network (FTDSDN) with multi-scale pyramid decision (MsPD) for solving heterogeneous face recognition task. As an extension of classical CNNs, FTDSDN can remove highly non-linear modality information and reserve the discriminative information using Rayleigh quotient objective function. Furthermore, we develop a powerful joint decision strategy called MsPD to adaptively adjust the weight of sub structure and obtain more robust classification performance. Experimental results show our proposed method achieves better performance on the challenging CASIA NIR-VIS 2.0 database, the heterogeneous face biometrics database, the CUHK face sketch FERET database, and the CUHK face sketch database, which demonstrates the effectiveness of our proposed approach.