Figure 4 - uploaded by Jeffrey Lund
Content may be subject to copyright.
Plot showing human agreement with each model type. CopulaLDA performs slightly worse than LDA. Humans preferred topic assignments from Anchor Words by a wide margin.
Source publication
Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments. Token-level assignments are important for downstream tasks such as classification. Even recent models, which aim to improve the quality of these token-l...
Contexts in source publication
Context 1
... previous work indicates that when properly aggregated, we can still filter out noisy judgments and obtain reasonable opinions (Nowak and Rüger, 2010). Figure 4 summarizes the human agreement with the three different model types. Surprisingly, despite claiming to produce superior local topic quality, CopulaLDA actually performs slightly worse than LDA according to our results with the topicword matching task. ...
Context 2
... previous work indicates that when properly aggregated, we can still filter out noisy judgments and obtain reasonable opinions (Nowak and Rüger, 2010). Figure 4 summarizes the human agreement with the three different model types. Surprisingly, despite claiming to produce superior local topic quality, CopulaLDA actually performs slightly worse than LDA according to our results with the topicword matching task. ...
Similar publications
A key challenge on the path to developing agents that learn complex human-like behavior is the need to quickly and accurately quantify human-likeness. While human assessments of such behavior can be highly accurate, speed and scalability are limited. We address these limitations through a novel automated Navigation Turing Test (ANTT) that learns to...
