A sparse-response deep belief network based on rate distortion theory
作者：
Highlights：
• A novel deep belief network based on rate distortion theory for feature extraction is proposed.
• Sparse response regularization induced by L1-norm of codes is used to achieve a small rate.
• KL divergence is considered as a distortion function.
• Hierarchical representations mimicking computations in the cortical hierarchy are learnt.
• More discriminative representation than other algorithms in deep belief networks is yielded.
摘要
Highlights•A novel deep belief network based on rate distortion theory for feature extraction is proposed.•Sparse response regularization induced by L1-norm of codes is used to achieve a small rate.•KL divergence is considered as a distortion function.•Hierarchical representations mimicking computations in the cortical hierarchy are learnt.•More discriminative representation than other algorithms in deep belief networks is yielded.
论文关键词：Deep belief network,Kullback–Leibler divergence,Information entropy,Rate distortion theory,Unsupervised feature learning
论文评审过程：Received 26 April 2013, Accepted 26 March 2014, Available online 12 April 2014.
论文官网地址：https://doi.org/10.1016/j.patcog.2014.03.025
