No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
A Method for Building a Commonsense Inference Dataset based on Basic EventsA Method for Building a Commonsense Inference Dataset based on Basic Events
ResearchGate has not been able to resolve any citations for this publication.
For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively tailored to any one specific task or dataset. In pursuit of this objective, we introduce the General Language Understanding Evaluation benchmark (GLUE), a tool for evaluating and analyzing the performance of models across a diverse range of existing NLU tasks. GLUE is model-agnostic, but it incentivizes sharing knowledge across tasks because certain tasks have very limited training data. We further provide a hand-crafted diagnostic test suite that enables detailed linguistic analysis of NLU models. We evaluate baselines based on current methods for multi-task and transfer learning and find that they do not immediately give substantial improvements over the aggregate performance of training a separate model per task, indicating room for improvement in developing general and robust NLU systems.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- J Devlin
- M.-W Chang
- K Lee
- K Toutanova
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
- A Talmor
- J Herzig
- N Lourie
- J Berant
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
- A Wang
- Y Pruksachatkun
- N Nangia
- A Singh
- J Michael
- F Hill
- O Levy
- S R Bowman