No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Modeling Compositionality with Dependency Graph for Dialogue Generation
ResearchGate has not been able to resolve any citations for this publication.
Recently, non-recurrent architectures (convolutional, self-attentional) have outperformed RNNs in neural machine translation. CNNs and self-attentional networks can connect distant words via shorter network paths than RNNs, and it has been speculated that this improves their ability to model long-range dependencies. However, this theoretical argument has not been tested empirically, nor have alternative explanations for their strong performance been explored in-depth. We hypothesize that the strong performance of CNNs and self-attentional networks could also be due to their ability to extract semantic features from the source text, and we evaluate RNNs, CNNs and self-attention networks on two tasks: subject-verb agreement (where capturing long-range dependencies is required) and word sense disambiguation (where semantic feature extraction is required). Our experimental results show that: 1) self-attentional networks and CNNs do not outperform RNNs in modeling subject-verb agreement over long distances; 2) self-attentional networks perform distinctly better than RNNs and CNNs on word sense disambiguation.
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
Improved semantic representations from tree-structured long short-term memory networks
- Kai Sheng Tai
- Richard Socher
- Christopher D Manning
Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from
tree-structured long short-term memory networks. In
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th
International Joint Conference on Natural Language
Processing (Volume 1: Long Papers), pages 1556-1566.
Hierarchical recurrent attention network for response generation
- Chen Xing
- Yu Wu
- Wei Wu
- Yalou Huang
- Ming Zhou
Chen Xing, Yu Wu, Wei Wu, Yalou Huang, and Ming
Zhou. 2018. Hierarchical recurrent attention network
for response generation. In Proceedings of the AAAI
Conference on Artificial Intelligence, volume 32.