Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis

摘要

Long short-term memory networks (LSTM) and classical convolutional neural networks (CNN) are two critical methods for the task of targeted sentiment analysis, but LSTM are difficult to parallelize and time-inefficient, and classical CNN can only capture local semantic features. To this end, this paper first proposes a sparse attention based separable dilated convolutional neural network (SA-SDCCN), which consists of multichannel embedding layer, separable dilated convolution module, sparse attention layer, and output layer. Specifically, our work is mainly concentrated on the first three parts. In multichannel embedding layer, semantic and sentiment embeddings are incorporated into an embedding tensor, which builds richer representations over the input sequence. In separable dilated convolution module, long-range contextual semantic information is explored and multi-scale contextual semantic dependencies are aggregated simultaneously through diverse dilation rates. Moreover, the separable structure further reduces the model parameters. In sparse attention layer, sentiment-oriented components are noticed according to the features of specific target entity. Finally, some experiments on three benchmark datasets demonstrate that SA-SDCCN achieves comparable or even better performance than state-of-the-art methods in terms of higher parallelism and lower computational cost.