No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Evidence Types, Credibility Factors, and Patterns or Soft Rules for Weighing Conflicting Evidence: Argument Mining in the Context of Legal Rules Governing Evidence Assessment
... Previous AM studies have explored materials like student essays [1,48], peer reviews [17], and comments on online forums [37]. Regarding the key aspect of argumentation in social and political science [54], AM has been particularly prominent at the intersection of Artificial Intelligence (AI) and Law [7,14,42,56]. Specifically, different types of legal documents have been explored as AM materials including court decisions [14], clinical trials [30], and judicial decisions [56,57]. Due to the complexity of argumentation in law, some works focus on mining argumentation from the logical or rhetorical background [58,63], while others emphasise the argumentation from the perspective of legal professionals [14]. ...
... Regarding the key aspect of argumentation in social and political science [54], AM has been particularly prominent at the intersection of Artificial Intelligence (AI) and Law [7,14,42,56]. Specifically, different types of legal documents have been explored as AM materials including court decisions [14], clinical trials [30], and judicial decisions [56,57]. Due to the complexity of argumentation in law, some works focus on mining argumentation from the logical or rhetorical background [58,63], while others emphasise the argumentation from the perspective of legal professionals [14]. ...
... Argument mining is a field of study dedicated to the identification and analysis of argumentative structures within a text. It has garnered significant attention recently due to its potential applications in automated essay scoring (Ke et al., 2018), legal decision support (Walker et al., 2018), healthcare applications (Mayer et al., 2020), etc. AM is often divided into four key tasks: (i) Argument Component Identification (ACI), identifying argumentative text spans; (ii) Argument Component Classification (ACC), categorizing these spans into AC types (e.g., claims, premises); (iii) Argumentative Relation Identification (ARI), detecting relationships between the spans; and (iv) Argumentative Relation Classification (ARC), classifying the types of 1 Our code is available here. Figure 1: Examples of Related Key Phrases between related AC pairs, highlighted in green. ...
... The identification of human values behind arguments is an important aspect of argument mining. Some work have been researched in this area, which aims to extract natural language arguments and their relations from text (Cabrio and Villata, 2018), there are a lot of use cases like (Passon et al., 2018) predicting the usefulness of online reviews based solely on the amount of argumentative text that they contain, or finding relevant evidence (on argument premises) in the study of adjudication decisions about veteran´s claims for disability (Walker et al., 2018) 3 System Overview The task of classifying textual arguments based on human values categories is challenging due to the subjective nature of human values. However, the system was able to address this challenge by using advanced deep learning algorithms such as BERT and RoBERTa. ...
... Work on argument mining has so far focused on processing legal text (Moens et al. 2007;Wyner et al. 2010;Walker et al. 2018), persuasive student essays (Persing and Ng 2016; Stab and Gurevych 2017), and Oxford-style debates (Orbach et al. 2020;Slonim et al. 2021). Although persuasive in nature, propagandistic articles (i.e., articles that aim to influence public opinion via disseminating biased and/or misleading information) have received relatively little attention in Natural Language Processing (NLP). ...
Propaganda campaigns have long been used to influence public opinion via disseminating biased and/or misleading information. Despite the increasing prevalence of propaganda content on the Internet, few attempts have been made by AI researchers to analyze such content. We introduce the task of multimodal propaganda processing, where the goal is to automatically analyze propaganda content. We believe that this task presents a long-term challenge to AI researchers and that successful processing of propaganda could bring machine understanding one important step closer to human understanding. We discuss the technical challenges associated with this task and outline the steps that need to be taken to address it.
... Work on argument mining has so far focused on processing legal text (Moens et al. 2007;Wyner et al. 2010;Walker et al. 2018), persuasive student essays (Persing and Ng 2016; Stab and Gurevych 2017), and Oxford-style debates (Orbach et al. 2020;Slonim et al. 2021). Although persuasive in nature, propagandistic articles (i.e., articles that aim to influence public opinion via disseminating biased and/or misleading information) have received relatively little attention in Natural Language Processing (NLP). ...
Propaganda campaigns have long been used to influence public opinion via disseminating biased and/or misleading information. Despite the increasing prevalence of propaganda content on the Internet, few attempts have been made by AI researchers to analyze such content. We introduce the task of multimodal propaganda processing, where the goal is to automatically analyze propaganda content. We believe that this task presents a long-term challenge to AI researchers and that successful processing of propaganda could bring machine understanding one important step closer to human understanding. We discuss the technical challenges associated with this task and outline the steps that need to be taken to address it.
... This work focuses on English texts. Legal texts in other languages [23] are also worth exploring in the future study of Legal AM. [18] PSI2020 CA, AD 42 doc Niculae et al. [15] NPC2017 CDCP CA, AD 731 rec, 3,800 set Park and Cardie [17] PC2018 CA 731 rec, 3,800 set Galassi et al. [7] GLT2021 AD Walker et al. [26] WCDL2011 VICP CA 30 doc Grabmair et al. [8] GACS2015 AD Walker et al. [27] WHNY2017 BVA CA 20doc, 5,674 set Walker et al. [24] WFPR2018 CA 30doc, 8,149 set Walker et al. [28] WPDL2019 CA, AD 50doc, 6,153 set Westermann et al. [31] WSWA2019 AD Walker et al. [29] WSW2020 CA, AD 75 doc, 623 set Xu et al. [32] XSA2020 CanLII CA, AD 683 doc, 30,374 set Xu et al. [34] XSA2021a CA, AD 1,148 doc, 127,330 set Xu et al. [33] XSA2021b CA, AD 2,098 doc, 226,576 set [12] provided the initial study on computational argumentation in legal text. In MM2011, they produced a corpus including 47 English-language cases (judgments and decisions) from the HUDOC 3 open-source database of the European Court of Human Rights (ECHR), a common resource for legal text processing research. ...
The growing research field of argumentation mining (AM) in the past ten years has made it a popular topic in Natural Language Processing. However, there are still limited studies focusing on AM in the context of legal text (Legal AM), despite the fact that legal text analysis more generally has received much attention as an interdisciplinary field of traditional humanities and data science. The goal of this work is to provide a critical data-driven analysis of the current situation in Legal AM. After outlining the background of this topic, we explore the availability of annotated datasets and the mechanisms by which these are created. This includes a discussion of how arguments and their relationships can be modelled, as well as a number of different approaches to divide the overall Legal AM task into constituent sub-tasks. Finally we review the dominant approaches that have been applied to this task in the past decade, and outline some future directions for Legal AM research.KeywordsArgumentation miningLegal textText analysis
... Evidence can be categorized into many different types, such as expert opinion, anecdote, or study data [171], or, with slightly different wording, study, expert or anecdotal [3]. Walker et al. [199] distinguish lay testimony, medical records, performance evaluations, other service records, other expert opinions, other records. Niculae et al. [125] include references such as URLs or citations as pointers to evidence. ...
Analyzing statements of facts and claims in online discourse is subject of a multitude of research areas. Methods from natural language processing and computational linguistics help investigate issues such as the spread of biased narratives and falsehoods on the Web. Related tasks include fact-checking, stance detection and argumentation mining. Knowledge-based approaches, in particular works in knowledge base construction and augmentation, are concerned with mining, verifying and representing factual knowledge. While all these fields are concerned with strongly related notions, such as claims, facts and evidence, terminology and conceptualisations used across and within communities vary heavily, making it hard to assess commonalities and relations of related works and how research in one field may contribute to address problems in another. We survey the state-of-the-art from a range of fields in this interdisciplinary area across a range of research tasks. We assess varying definitions and propose a conceptual model – Open Claims – for claims and related notions that takes into consideration their inherent complexity, distinguishing between their meaning, linguistic representation and context. We also introduce an implementation of this model by using established vocabularies and discuss applications across various tasks related to online discourse analysis.
... Argumentation mining (AM) aims to identify the argumentation structures in text, which has received widespread attention in recent years (Lawrence and Reed, 2019). It has been shown beneficial in a broad range of fields, such as information retrieval (Carstens and Toni, 2015;Stab et al., 2018), automated essay scoring (Wachsmuth et al., 2016;Ke et al., 2018), and legal decision support (Palau and Moens, 2009;Walker et al., 2018). Given a piece of paragraph-level argumentative text, an AM system first detects argument components (ACs), which are segments of text with argumentative meaning, and then extracts the argumentative relations (ARs) between ACs to obtain an argumentation graph, where the nodes and edges represent ACs and ARs, * Equal Contribution † Corresponding Author Figure 1: An example of argumentation mining from the CDCP dataset (Park and Cardie, 2018). ...
... In their view, the legal AR system is the next stage in the evolution of legal IR since lawyers are primarily interested in retrieving arguments and not just documents. In subsequent work they provide examples of how an AR system could be constructed for the narrow U.S. legal domains of vaccine injury compensation (Grabmair 2016;Walker et al. 2015;Grabmair et al. 2015;Walker et al. 2014) and claims by military veterans for disability compensation (Zhong et al. 2019;Walker et al. 2017Walker et al. , 2018Walker et al. , 2019. These efforts are similar to our approach here in that they employ legal argument mining to improve legal IR. ...
In this work we study, design, and evaluate computational methods to support interpretation of statutory terms. We propose a novel task of discovering sentences for argumentation about the meaning of statutory terms. The task models the analysis of past treatment of statutory terms, an exercise lawyers routinely perform using a combination of manual and computational approaches. We treat the discovery of sentences as a special case of ad hoc document retrieval. The specifics include retrieval of short texts (sentences), specialized document types (legal case texts), and, above all, the unique definition of document relevance provided in detailed annotation guidelines. To support our experiments we assembled a data set comprising 42 queries (26,959 sentences) which we plan to release to the public in the near future in order to support further research. Most importantly, we investigate the feasibility of developing a system that responds to a query with a list of sentences that mention the term in a way that is useful for understanding and elaborating its meaning. This is accomplished by a systematic assessment of different features that model the sentences’ usefulness for interpretation. We combine features into a compound measure that accounts for multiple aspects. The definition of the task, the assembly of the data set, and the detailed task analysis provide a solid foundation for employing a learning-to-rank approach.
... Although these properties are limited in what they tell us about the overall argumentative structure, they provide valuable information about the role that a particular text span is playing in the argument as a whole. For example, knowing that a claim is verifiable suggests a link to a piece of evidence in the text supporting this claim (Park and Cardie 2014); knowing that a clause is increasing the author's ethos suggests that it is supporting a specific claim that they are making (Duthie, Budzynska, and Reed 2016); and knowing the type of evidence provided can be used to assign different weights to statements in clinical trials (Mayer, Cabrio, and Villata 2018), or help understand rulings in disability benefits claims (Walker et al. 2018). ...
Argument mining is the automatic identification and extraction of the structure of inference and reasoning expressed as arguments presented in natural language. Understanding argumentative structure makes it possible to determine not only what positions people are adopting, but also why they hold the opinions they do, providing valuable insights in domains as diverse as financial market prediction and public relations. This survey explores the techniques that establish the foundations for argument mining, provides a review of recent advances in argument mining techniques, and discusses the challenges faced in automatically extracting a deeper understanding of reasoning expressed in language in general.
Computational Argumentation studies how human argumentative reasoning can be approached from a computational viewpoint. Human argumentation is a complex process that has been studied from different perspectives (e.g., philosophical or linguistic) and that involves many different aspects beyond pure reasoning, such as the role of emotions, values, social contexts, and practical constraints, which are often overlooked in computational approaches to argumentation. The heterogeneity of human argumentation is present in Computational Argumentation research, in the form of various tasks that approach the main phases of argumentation individually. With the increasing interest of researchers in Artificial Intelligence, we consider that it is of great importance to provide guidance on the Computational Argumentation research area. Thus, in this paper, we present a general overview of Computational Argumentation, from the perspective of how humans argue. For that purpose, the following contributions are produced: (i) a consistent structure for Computational Argumentation research mapped with the human argumentation process; (ii) a collective understanding of the tasks approached by Computational Argumentation and their synergies; (iii) a thorough review of important advances in each of these tasks; and (iv) an analysis and a classification of the future trends in Computational Argumentation research and relevant open challenges in the area.
Argument pair extraction (APE) is a new task in the field of argument mining, aiming at mining interactive argument pairs from peer review and rebuttal. Previous work framed APE as a combination of a sequence labeling task and a sentence pair classification task via multi-task learning. However, this method lacks explicit modeling of the relations between arguments, and the two subtasks may not cooperate well. Towards these issues, we propose a hierarchical sequence labeling model that can efficiently represent arguments and explicitly capture the correlations between them. Our method matches argument pairs from the perspective of review and rebuttal respectively, and merges the results, enabling a more comprehensive extraction of argument pairs. Also, we propose a series of improved models and fuse them using weighted voting. Our method achieved first place in the NLPCC 2021 Shared Task on APE, proving the effectiveness of our method.
We report results of an effort to enable computers to segment US adjudicatory decisions into sentences. We created a data set of 80 court decisions from four different domains. We show that legal decisions are more challenging for existing sentence boundary detection systems than for non-legal texts. Existing sentence boundary detection systems are based on a number of assumptions that do not hold for legal texts, hence their performance is impaired. We show that a general statistical sequence labeling model is capable of learning the definition more efficiently. We have trained a number of conditional random fields models that outperform the traditional sentence boundary detection systems when applied to adjudicatory decisions. © 2017 Association pour le traitement automatique des langues. All rights reserved.
This paper focuses on the automated extraction of argument components from user content in the German online participation project Tempelhofer Feld. We adapt existing argumentation models into a new model for decision-oriented online participation. Our model consists of three categories: major positions, claims, and premises. We create a new German corpus for argument mining by annotating our dataset with our model. Afterwards, we focus on the two classification tasks of identifying argumentative sentences and predicting argument components in sentences. We achieve macro-averaged F1 measures of 69.77% and 68.5%, respectively.
In this paper, we present a novel ap-proach for identifying argumentative dis-course structures in persuasive essays. The structure of argumentation consists of sev-eral components (i.e. claims and premises) that are connected with argumentative re-lations. We consider this task in two consecutive steps. First, we identify the components of arguments using multiclass classification. Second, we classify a pair of argument components as either support or non-support for identifying the struc-ture of argumentative discourse. For both tasks, we evaluate several classifiers and propose novel feature sets including struc-tural, lexical, syntactic and contextual fea-tures. In our experiments, we obtain a macro F1-score of 0.726 for identifying argument components and 0.722 for argu-mentative relations.
Since the time of the ancient Greek philosophers and rhetoricians, argumentation theorists have searched for the requirements
that make an argument correct, by some appropriate standard of proof, by examining the errors of reasoning we make when we
try to use arguments. These errors have long been called fallacies, and the logic textbooks have for over 2000 years tried
to help students to identify these fallacies, and to deal with them when they are encountered. The problem was that deductive
logic did not seem to be much use for this purpose, and there seemed to be no other obvious formal structure that could usefully
be applied to them.
Argumentation mining aims to automatically detect, classify and structure argumentation in text. Therefore, argumentation
mining is an important part of a complete argumentation analyisis, i.e. understanding the content of serial arguments, their
linguistic structure, the relationship between the preceding and following arguments, recognizing the underlying conceptual
beliefs, and understanding within the comprehensive coherence of the specific topic. We present different methods to aid argumentation
mining, starting with plain argumentation detection and moving forward to a more structural analysis of the detected argumentation.
Different state-of-the-art techniques on machine learning and context free grammars are applied to solve the challenges of
argumentation mining. We also highlight fundamental questions found during our research and analyse different issues for future
research on argumentation mining.
KeywordsArgumentation–Information extraction–Text classification–Document structure
We describe research carried out as part of a text summarisation project for the legal domain for which we use a new XML corpus of judgments of the UK House of Lords. These judgments represent a particularly important part of public discourse due to the role that precedents play in English law. We present experimental results using a range of features and machine learning techniques for the task of predicting the rhetorical status of sentences and for the task of selecting the most summary-worthy sentences from a document. Results for these components are encouraging as they achieve state-of-the-art accuracy using robust, automatically generated cue phrase information. Sample output from the system illustrates the potential of summarisation technology for legal information management systems and highlights the utility of our rhetorical annotation scheme as a model of legal discourse, which provides a clear means for structuring summaries and tailoring them to different types of users.
Argumentation is the process by which arguments are con- structed and handled. Argumentation constitutes a major component of human intelligence. The ability to engage in argumentation is essential for humans to understand new problems, to perform scientific reasoning, to express, to clar- ify and to defend their opinions in their daily lives. Argu- mentation mining aims to detect the arguments presented in a text document, the relations between them and the in- ternal structure of each individual argument. In this paper we analyse the main research questions when dealing with argumentation mining and the dierent methods we have studied and developed in order to successfully confront the challenges of argumentation mining in legal texts.
Work on a computer program called SMILE + IBP (SMart Index Learner Plus Issue-Based Prediction) bridges case-based reasoning
and extracting information from texts. The program addresses a technologically challenging task that is also very relevant
from a legal viewpoint: to extract information from textual descriptions of the facts of decided cases and apply that information
to predict the outcomes of new cases. The program attempts to automatically classify textual descriptions of the facts of
legal problems in terms of Factors, a set of classification concepts that capture stereotypical fact patterns that effect
the strength of a legal claim, here trade secret misappropriation. Using these classifications, the program can evaluate and
explain predictions about a problem’s outcome given a database of previously classified cases. This paper provides an extended
example illustrating both functions, prediction by IBP and text classification by SMILE, and reports empirical evaluations
of each. While IBP’s results are quite strong, and SMILE’s much weaker, SMILE + IBP still has some success predicting and
explaining the outcomes of case scenarios input as texts. It marks the first time to our knowledge that a program can reason
automatically about legal case texts.
This paper announces the creation and public availability of a dataset of annotated decisions adjudicating claims by military veterans for disability compensation in the United States. This is intended to initiate a collaborative, transparent approach to semantic analysis for argument mining from legal documents. The dataset is being used in the LUIMA argument-mining project. We address two major sub-tasks for making legal reasoning computable. First, we report the semantic types of propositional connective we use to extract information about legal rules from sentences in statutes, regulations, and appellate court decisions, and to represent those rules as integrated systems. Second, we report the semantic types of sentence role we use to extract and represent the fact-finding reasoning found in adjudicatory decisions, with the goal of identifying successful and unsuccessful patterns of evidentiary argument. For each type system, we provide explanations and examples. Thus, we hope to stimulate a shared effort to create diverse datasets in law, to empirically evolve optimal sets of semantic types for argument mining, and to refine protocols for accurately applying those types to texts.
Nearly twenty years ago, Congress for the first time created a system for judicial review of decisions denying veterans benefits. Specifically, Congress created an Article I Court: the United States Court of Appeals for Veterans Claims. Veterans dissatisfied with actions of the Department of Veterans Affairs regarding benefits could appeal to the Veterans Court. The United States Court of Appeals for the Federal Circuit provided appellate oversight of the Veterans Court. There simply is nothing like the Veterans Court elsewhere in American law. Yet, despite its uniqueness, there has been little scholarly attention to this institution. This Article begins to fill the gap in the literature through a focused consideration of the decisions of the Veterans Court and the Federal Circuit from 2004 to 2006. It has three principal parts. First, it describes the current structure of judicial review in the area and provides a statistical analysis of its operation during the relevant period. Second, the Article explores the substantive development of veterans law from January 2004 through March 2006. Finally, based on that substantive law, the Article draws conclusions about the operations of both the Veterans Court and the Federal Circuit.
Unlike research in linguistics and artificial intelligence, legal research has not used advances in logical theory very effectively. This article uses default logic to develop a paradigm for analyzing all aspects of legal reasoning, including factfinding. The article provides a formal model that integrates legal rules and policies with the evaluation of both expert and non-expert evidence - whether the reasoning occurs in courts or administrative agencies, and whether in domestic, foreign, or international legal systems. This paradigm can standardize the representation of legal reasoning, guide empirical research into the dynamics of such reasoning, and put the representations and research results to immediate use through artificial intelligence software. This new model therefore has the potential to transform legal practice and legal education, as well as legal theory.
We describe an information extraction and retrieval system, called History Assistant, which extracts rulings from court opinions and retrieves relevant prior cases from a citator database. The technology employed is similar to that adopted in the Message Understanding Conferences, but attempts a fuller parse in order to distinguish current rulings from previous rulings reported in a case. In addition, we employ a combination of information retrieval and machine learning techniques to link each new case to related documents that it may impact. We present experimental results, in terms of precision and recall, for all tasks performed by the extraction and linking programs. Part of the finished system has been deemed worthy of further development into a computer-assisted database update tool to help editors assimilate historical relationships between cases into a concordance of court decisions, called a citator.
Legal judgments are complex in nature and hence a brief summary of the judgment, known as a headnote, is generated by experts to enable quick perusal. Headnote generation is a time consuming process and there have been attempts
made at automating the process. The difficulty in interpreting such automatically generated summaries is that they are not
coherent and do not convey the relative relevance of the various components of the judgment. A legal judgment can be segmented
into coherent chunks based on the rhetorical roles played by the sentences. In this paper, a comprehensive system is proposed for labeling sentences with their rhetorical roles
and extracting structured head notes automatically from legal judgments. An annotated data set was created with the help of
legal experts and used as training data. A machine learning technique, Conditional Random Field, is applied to perform document
segmentation by identifying the rhetorical roles. The present work also describes the application of probabilistic models
for the extraction of key sentences and composing the relevant chunks in the form of a headnote. The understanding of basic
structures and distinct segments is shown to improve the final presentation of the summary. Moreover, by adding simple additional
features the system can be extended to other legal sub-domains. The proposed system has been empirically evaluated and found
to be highly effective on both the segmentation and summarization tasks. The final summary generated with underlying rhetorical
roles improves the readability and efficiency of the system.
Statement Types in Legal Argument
- Latifa Al-Abdulkarim
- Katie Atkinson
- Trevor Bench-Capon
Latifa Al-Abdulkarim, Katie Atkinson, and Trevor
Bench-Capon. 2016. Statement Types in Legal
Argument. In Floris Bex and Serena Villata,
editors, Legal Knowledge and Information Systems
(JURIX 2016), pages 3-12. IOS Press.
The Downfall of Auer Deference: Veterans Law at the Federal Circuit in 2014
- Moshiashwili Victoria Hadfield
Victoria Hadfield Moshiashwili. 2015. The Downfall
of Auer Deference: Veterans Law at the Federal
Circuit in 2014. American University Law Review
64: 1007-1087. American University.
Argumentation Mining from Judicial Decisions: The Attribution Problem and the Need for Legal Discourse Models. Paper at the First Workshop on Automated Detection, Extraction and Analysis of Semantic Information in Legal Texts (ASAIL 2015)
- Vern R Walker
- Parisa Bagheri
- Andrew J Lauria
Vern R. Walker, Parisa Bagheri and Andrew J. Lauria.
2015. Argumentation Mining from Judicial
Decisions: The Attribution Problem and the Need
for Legal Discourse Models. Paper at the First
Workshop on Automated Detection, Extraction and
Analysis of Semantic Information in Legal Texts
(ASAIL 2015), San Diego, California, USA. URL:
https://people.hofstra.edu/vern_r_walker/WalkerEt
Al-AttributionAndLegalDiscourseModels-ASAIL2015.pdf.
Semantic Types for Decomposing Evidence Assessment in Decisions on Veterans' Disability Claims for PTSD
- Vern R Walker
- Ashtyn Hemendinger
- Nneka Okpara
- Tauseef Ahmed
Vern R. Walker, Ashtyn Hemendinger, Nneka Okpara
and Tauseef Ahmed. 2017b. Semantic Types for
Decomposing Evidence Assessment in Decisions
on Veterans' Disability Claims for PTSD. In
Proceedings of the Second Workshop on Automatic
Semantic Analysis of Semantic Information in Legal
Text (ASAIL 2017), London, UK.