Fine-grained image classification with factorized deep user click feature

Highlights：

• We construct factorized TF-IDF vectors using word vocabularies of different POS to represent images. Compared with traditional TF-IDF vector built on an entire word vocabulary, the factorized TF-IDF vectors discover better semantics with complementary information of different POS.

• We develop the click feature tensor to integrate the factorized TF-IDF vectors, and devise an end-to-end deep neural network on the click feature tensors to learn a structured click feature. The deep click model further enhances image representations by capturing hierarchical semantics.

摘要

•We propose a novel image classification framework with deep click feature. Compared with previous visual-feature-based approaches, the proposed approach captures image semantics in a better manner.•We construct factorized TF-IDF vectors using word vocabularies of different POS to represent images. Compared with traditional TF-IDF vector built on an entire word vocabulary, the factorized TF-IDF vectors discover better semantics with complementary information of different POS.•We develop the click feature tensor to integrate the factorized TF-IDF vectors, and devise an end-to-end deep neural network on the click feature tensors to learn a structured click feature. The deep click model further enhances image representations by capturing hierarchical semantics.

论文评审过程：Received 31 July 2019, Revised 15 November 2019, Accepted 16 December 2019, Available online 31 December 2019, Version of Record 31 December 2019.