Comparison of Recall, Precision and F1 on overall classification of each coherence level.

Comparison of Recall, Precision and F1 on overall classification of each coherence level.

Source publication
Preprint
Full-text available
This paper evaluates the utility of Rhetorical Structure Theory (RST) trees and relations in discourse coherence evaluation. We show that incorporating silver-standard RST features can increase accuracy when classifying coherence. We demonstrate this through our tree-recursive neural model, namely RST-Recursive, which takes advantage of the text's...

Context in source publication

Context 1
... demonstrated in Figure 5, coherence classifiers have difficulty predicting the neutral class (2), experiencing modal collapse towards the extreme ends in the best performing models. Early experiments using alternative objective functions such as the Ordinal Loss or Mean Squared Error resulted in a similar modal collapse or poor overall performance. ...

Similar publications

Conference Paper
Full-text available
We present a freely available, genre-balanced English web corpus totaling 4M tokens and featuring a large number of high-quality automatic annotation layers, including dependency trees, non-named entity annotations, coreference resolution, and discourse trees in Rhetorical Structure Theory. By tapping open online data sources the corpus is meant to...
Article
Full-text available
This paper proposes a two-stage automatic text summarization method based on discourse structure, aiming to improve the accuracy and coherence of the summary. In the extractive stage, a text encoder divides the long text into elementary discourse units (EDUs). Then a parse tree based on rhetorical structure theory is constructed for the whole disco...