Convolutional Neural Networks and Long Short-Term Memory for skeleton-based human activity and hand gesture recognition

作者：

Highlights：

• Combination of a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) recurrent network for skeleton-based human activity and hand gesture recognition.

• Two-stage training strategy which firstly focuses on the CNN training and, secondly, adjusts the full method CNN+LSTM.

• A method for data augmentation in the context of spatiotemporal 3D data sequences.

• An exhaustive experimental study on publicly available data benchmarks with respect to the state-of-the-art most representative methods.

• Comparison among different CPU and GPU platforms.

摘要

•Combination of a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM) recurrent network for skeleton-based human activity and hand gesture recognition.•Two-stage training strategy which firstly focuses on the CNN training and, secondly, adjusts the full method CNN+LSTM.•A method for data augmentation in the context of spatiotemporal 3D data sequences.•An exhaustive experimental study on publicly available data benchmarks with respect to the state-of-the-art most representative methods.•Comparison among different CPU and GPU platforms.

论文关键词：Deep learning,Convolutional Neural Network,Recurrent neural network,Long Short-Term Memory,Human activity recognition,Hand gesture recognition,Real-time

论文评审过程：Received 2 December 2016, Revised 6 October 2017, Accepted 24 October 2017, Available online 21 December 2017, Version of Record 21 December 2017.

论文官网地址：https://doi.org/10.1016/j.patcog.2017.10.033