Improving tree-based neural machine translation with dynamic lexicalized dependency encoding

摘要

Tree-to-sequence neural machine translation models have proven to be effective in learning the semantic representations from the exploited syntactic structure. Despite their success, tree-to-sequence models have two major issues: (1) the embeddings of constituents at the higher tree levels tend to contribute less in translation; and (2) using a single set of model parameters is difficult to fully capture the syntactic and semantic richness of linguistic phrases. To address the first problem, we proposed a lexicalized dependency model, in which the source-side lexical representations are learned in a head-dependent fashion following a dependency graph. Since the number of dependents is variable, we proposed a variant recurrent neural network (RNN) to jointly consider the long-distance dependencies and the sequential information of words. Concerning the second problem, we adopt a latent vector to dynamically condition the parameters for the composition of each node representation. Experimental results reveal that the proposed model significantly outperforms the recently proposed tree-based methods in English–Chinese and English–German translation tasks with even far fewer parameters.