Machine learning-based coreference resolution of concepts in clinical documents

M*Modal, Inc., Morgantown, West Virginia 26505, USA.
Journal of the American Medical Informatics Association (Impact Factor: 3.5). 05/2012; 19(5):883-7. DOI: 10.1136/amiajnl-2011-000774
Source: PubMed


Coreference resolution of concepts, although a very active area in the natural language processing community, has not yet been widely applied to clinical documents. Accordingly, the 2011 i2b2 competition focusing on this area is a timely and useful challenge. The objective of this research was to collate coreferent chains of concepts from a corpus of clinical documents. These concepts are in the categories of person, problems, treatments, and tests.
A machine learning approach based on graphical models was employed to cluster coreferent concepts. Features selected were divided into domain independent and domain specific sets. Training was done with the i2b2 provided training set of 489 documents with 6949 chains. Testing was done on 322 documents.
The learning engine, using the un-weighted average of three different measurement schemes, resulted in an F measure of 0.8423 where no domain specific features were included and 0.8483 where the feature set included both domain independent and domain specific features.
Our machine learning approach is a promising solution for recognizing coreferent concepts, which in turn is useful for practical applications such as the assembly of problem and medication lists from clinical documents.

Download full-text


Available from: Vasudevan Jagannathan, Oct 28, 2015
  • Source
    • "In the clinical field, it includes for example relation between disease and drug. • Co-reference Analysis task, is a task which determine linguistic expressions that refer to the same real-world entity in natural language, has not yet been widely applied to clinical documents [40] "

    01/2015; 3(3):16. DOI:10.9781/ijimai.2015.332
  • Source
    • "So, we are unable to compare whether their system is as robust as ours for different evaluation metrics. Ware et al. [35] achieved an unweighted F 1 score of 0.848 on the full official test corpus (lower than our results). Their system performs poorly for MUC metrics, e.g. it obtains only 0.254 MUC F 1 score for Test chain type. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Identification of co-referent entity mentions inside text has significant importance for other natural language processing (NLP) tasks (e.g.event linking). However, this task, known as co-reference resolution, remains a complex problem, partly because of the confusion over different evaluation metrics and partly because the well-researched existing methodologies do not perform well on new domains such as clinical records. This paper presents a variant of the influential mention-pair model for co-reference resolution. Using a series of linguistically and semantically motivated constraints, the proposed approach controls generation of less-informative/sub-optimal training and test instances. Additionally, the approach also introduces some aggressive greedy strategies in chain clustering. The proposed approach has been tested on the official test corpus of the recently held i2b2/VA 2011 challenge. It achieves an unweighted average F1 score of 0.895, calculated from multiple evaluation metrics (MUC,B(3) and CEAF scores). These results are comparable to the best systems of the challenge. What makes our proposed system distinct is that it also achieves high average F1 scores for each individual chain type (Test: 0.897, Person: 0.852, Problem: 0.855, Treatment: 0.884). Unlike other works, it obtains good scores for each of the individual metrics rather than being biased towards a particular metric.
    Journal of Biomedical Informatics 04/2013; 46(3). DOI:10.1016/j.jbi.2013.03.007 · 2.19 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The fifth i2b2/VA Workshop on Natural Language Processing Challenges for Clinical Records conducted a systematic review on resolution of noun phrase coreference in medical records. Informatics for Integrating Biology and the Bedside (i2b2) and the Veterans Affair (VA) Consortium for Healthcare Informatics Research (CHIR) partnered to organize the coreference challenge. They provided the research community with two corpora of medical records for the development and evaluation of the coreference resolution systems. These corpora contained various record types (ie, discharge summaries, pathology reports) from multiple institutions. The coreference challenge provided the community with two annotated ground truth corpora and evaluated systems on coreference resolution in two ways: first, it evaluated systems for their ability to identify mentions of concepts and to link together those mentions. Second, it evaluated the ability of the systems to link together ground truth mentions that refer to the same entity. Twenty teams representing 29 organizations and nine countries participated in the coreference challenge. The teams' system submissions showed that machine-learning and rule-based approaches worked best when augmented with external knowledge sources and coreference clues extracted from document structure. The systems performed better in coreference resolution when provided with ground truth mentions. Overall, the systems struggled in solving coreference resolution for cases that required domain knowledge.
    Journal of the American Medical Informatics Association 02/2012; 19(5):786-91. DOI:10.1136/amiajnl-2011-000784 · 3.50 Impact Factor
Show more