-
[show abstract]
[hide abstract]
ABSTRACT: We explore a new way to collect human an-notated relations in text using Amazon Me-chanical Turk. Given a knowledge base of relations and a corpus, we identify sentences which mention both an entity and an attribute that have some relation in the knowledge base. Each noisy sentence/relation pair is presented to multiple turkers, who are asked whether the sentence expresses the relation. We describe a design which encourages user efficiency and aids discovery of cheating. We also present results on inter-annotator agreement.
07/2010;
-
COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference, 23-27 August 2010, Beijing, China; 01/2010
-
COLING 2010, 23rd International Conference on Computational Linguistics, Posters Volume, 23-27 August 2010, Beijing, China; 01/2010
-
[show abstract]
[hide abstract]
ABSTRACT: The HLTCOE participated in the entity link-ing and slot filling tasks at TAC 2009. A ma-chine learning-based approach to entity link-ing, operating over a wide range of feature types, yielded good performance on the entity linking task. Slot-filling based on sentence se-lection, application of weak patterns and ex-ploitation of redundancy was ineffective in the slot filling task.
-
[show abstract]
[hide abstract]
ABSTRACT: We describe our experience using both Amazon Mechanical Turk (MTurk) and Crowd-Flower to collect simple named entity annotations for Twitter status updates. Unlike most genres that have traditionally been the focus of named entity experiments, Twitter is far more informal and abbreviated. The collected annotations and annotation techniques will provide a first step towards the full study of named entity recognition in domains like Facebook and Twitter. We also briefly describe how to use MTurk to collect judgements on the quality of "word clouds."