No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Rule-Based Approach for Party-Based Sentiment Analysis in Legal Opinion Texts
... Many NLP Researches have been carried out in the legal domain in areas such as: domain specific embedding [5][6][7], ontologies [7][8][9], sentiment analysis [10][11][12][13][14], and discourse analysis [15][16][17] to address aforementioned inefficiencies. Party identification [18][19][20], which is the recognition of legal parties as introduced in Section I-A, and party based sentiment analysis [12][13][14], which is the identification of the positive or negative impact of a sentence for party members, are important related researches. ...
... Many NLP Researches have been carried out in the legal domain in areas such as: domain specific embedding [5][6][7], ontologies [7][8][9], sentiment analysis [10][11][12][13][14], and discourse analysis [15][16][17] to address aforementioned inefficiencies. Party identification [18][19][20], which is the recognition of legal parties as introduced in Section I-A, and party based sentiment analysis [12][13][14], which is the identification of the positive or negative impact of a sentence for party members, are important related researches. ...
... In the study by Rajapaksha et al. [12], the sentiment for each party is calculated using sentence level sentiment. To allocate the sentiment to each party they have used a simple convention. ...
The rapid growth of text corpora across various domains has emerged a need and an opportunity to leverage Natural Language Processing to automate and efficiently streamline tedious manual tasks. Legal domain is one such text rich domain which suffers a rapid growth of text corpora and requirement for natural language processing applications. In the pursuit of automating the prediction of the winning party of a court case among other usages, analysing sentiment in a party wise manner is beneficial for legal professionals. The two main sub-tasks in this process is to identify parties in a court case and afterwards analysing the respective sentiment towards each party. In this study we discuss the unification of two such models capable of doing the two task into a single pipeline to perform party based sentiment analysis efficiently
... As the number of legal cases increases, legal professionals typically endure heavy workloads on a daily basis, and they may become overwhelmed and as a result of that, be unable to obtain quality analysis. In this analysis process, identifying advantageous and disadvantageous statements relevant to legal parties [1][2][3][4] can be considered a critical and time consuming task. By automating this task, legal officers will be able to reduce their workload significantly. ...
... Extracting opinions with respect to each legal party cannot be performed only by using document-level, sentencelevel, or phrase-level sentiment analysis. Aspect-based sentiment analysis (ABSA) is the most appropriate and fine-grained solution to perform Party-Based Sentiment Analysis (PBSA) in the legal domain [1]. In aspect-based sentiment analysis, we can identify there processing steps such as "identification, classification, and aggregation" [6]. ...
... Generally, in ABSA aspects are extracted from a given text and then each aspect is allocated a sentiment level (positive, negative, or neutral) [7]. The members of legal parties in a court case are considered as aspects and therefore performing ABSA in the legal opinion texts can also be termed as Party-Based Sentiment Analysis (PBSA) [1]. ...
When lawyers and legal officers are working on a new legal case, they are supposed have properly studied prior cases similar to the current case, as the prior cases can provide valuable information which can have a direct impact on the outcomes of the current court case. Therefore, developing methodologies which are capable of automatically extracting information from legal opinion texts related to previous court cases can be considered as an important tool when it comes to the legal technology ecosystem. In this study, we focus on finding advantageous and disadvantageous facts or arguments in court cases, which is one of the most critical and time-consuming tasks in court case analysis. The Aspect-based Sentiment Analysis concept is used as the base of this study to perform legal information extraction. In this paper, we introduce a solution to predict sentiment value of sentences in legal documents in relation to its legal parties. The proposed approach employs a fine-grained sentiment analysis (Aspect-Based Sentiment Analysis) technique to achieve this task. Sigmalaw PBSA is a novel deep learning-based model for ABSA which is specifically designed for legal opinion texts. We evaluate the Sigmalaw PBSA model and existing ABSA models on the SigmaLaw-ABSA dataset which consists of 2000 legal opinion texts fetched from a public online data base. Experiments show that our model outperforms the state-of-the-art models. We also conduct an ablation study to identify which methods are most effective for legal texts.
... As the number of legal cases increases, legal professionals typically endure heavy workloads on a daily basis, and they may become overwhelmed and as a result of that, be unable to obtain quality analysis. In this analysis process, identifying advantageous and disadvantageous statements relevant to legal parties [1][2][3][4] can be considered a critical and time consuming task. By automating this task, legal officers will be able to reduce their workload significantly. ...
... Extracting opinions with respect to each legal party cannot be performed only by using document-level, sentencelevel, or phrase-level sentiment analysis. Aspect-based sentiment analysis (ABSA) is the most appropriate and fine-grained solution to perform Party-Based Sentiment Analysis (PBSA) in the legal domain [1]. In aspect-based sentiment analysis, we can identify there processing steps such as "identification, classification, and aggregation" [6]. ...
... Generally, in ABSA aspects are extracted from a given text and then each aspect is allocated a sentiment level (positive, negative, or neutral) [7]. The members of legal parties in a court case are considered as aspects and therefore performing ABSA in the legal opinion texts can also be termed as Party-Based Sentiment Analysis (PBSA) [1]. ...
Legal information retrieval holds a significant importance to lawyers and legal professionals. Its significance has grown as a result of the vast and rapidly increasing amount of legal documents available via electronic means. Legal documents, which can be considered flat file databases, contain information that can be used in a variety of ways, including arguments, counter-arguments, justifications, and evidence. As a result, developing automated mechanisms for extracting important information from legal opinion texts can be regarded as an important step toward introducing artificial intelligence into the legal domain. Identifying advantageous or disadvantageous statements within these texts in relation to legal parties can be considered as a critical and time consuming task. This task is further complicated by the relevance of context in automatic legal information extraction. In this paper, we introduce a solution to predict sentiment value of sentences in legal documents in relation to its legal parties. The Proposed approach employs a fine-grained sentiment analysis (Aspect-Based Sentiment Analysis) technique to achieve this task. Sigmalaw PBSA is a novel deep learning-based model for ABSA which is specifically designed for legal opinion texts. We evaluate the Sigmalaw PBSA model and existing ABSA models on the SigmaLaw-ABSA dataset which consists of 2000 legal opinion texts fetched from a public online data base. Experiments show that our model outperforms the state-of-the-art models. We also conduct an ablation study to identify which methods are most effective for legal texts.
... However, conversely, this has added a de facto expectation on businesses to understand the user sentiments expressed in these texts if they intend to make informed decisions and enhance customer satisfaction. Aspect-based sentiment analysis (ABSA) has emerged as a valuable technique in Natural Language Processing (NLP) to analyze opinions at a finer granular level by identifying sentiment towards specific aspects or features within a given domain [1][2][3][4][5][6]. In this paper, we focus on domain-specific ABSA [2][3][4], particularly focusing on its application in analyzing customer reviews, which provides valuable feedback for businesses to improve their products or services. ...
... Aspect-based sentiment analysis (ABSA) has emerged as a valuable technique in Natural Language Processing (NLP) to analyze opinions at a finer granular level by identifying sentiment towards specific aspects or features within a given domain [1][2][3][4][5][6]. In this paper, we focus on domain-specific ABSA [2][3][4], particularly focusing on its application in analyzing customer reviews, which provides valuable feedback for businesses to improve their products or services. ...
Since the dawn of the digitalisation era, customer feedback and online reviews are unequivocally major sources of insights for businesses. Consequently, conducting comparative analyses of such sources has become the de facto modus operandi of any business that wishes to give itself a competitive edge over its peers and improve customer loyalty. Sentiment analysis is one such method instrumental in gauging public interest, exposing market trends, and analysing competitors. While traditional sentiment analysis focuses on overall sentiment, as the needs advance with time, it has become important to explore public opinions and sentiments on various specific subjects, products and services mentioned in the reviews on a finer-granular level. To this end, Aspect-based Sentiment Analysis (ABSA), supported by advances in Artificial Intelligence (AI) techniques which have contributed to a paradigm shift from simple word-level analysis to tone and context-aware analyses, focuses on identifying specific aspects within the text and determining the sentiment associated with each aspect. In this study, we compare several deep-NN methods for ABSA on two benchmark datasets (Restaurant14 and Laptop-14) and found that FAST LSA obtains the best overall results of 87.6% and 82.6% accuracy but does not pass LSA+DeBERTa which reports 90.33% and 86.21% accuracy respectively.
... The legal domain is such a domain in which research interests have come up over the recent past, and many research have been carried out considering its different aspects. Party based sentiment analysis [1][2][3], extracting parties of a legal case [4][5][6], detecting critical sentences of a court case and predicting the outcome of a court case are such important aspects. ...
... 1) Petitioner lose and has negative impact towards petitioner (Petitioner lose and negative) 2) Petitioner lose and has positive impact towards petitioner (Petitioner lose and positive) 1 SigmaLaw ABSA Dataset: https://osf.io/37gkh/ 3) Petitioner win and has negative impact towards petitioner (Petitioner win and negative) 4) Petitioner win and has positive impact towards petitioner (Petitioner win and positive) For simplicity, we will refer the above categories in a shorter form as mentioned in the brackets. ...
The advancement of Natural Language Processing (NLP) is spreading through various domains in forms of practical applications and academic interests. Inherently, the legal domain contains a vast amount of data in text format. Therefore it requires the application of NLP to cater to the analytically demanding needs of the domain. Identifying important sentences, facts and arguments in a legal case is such a tedious task for legal professionals. In this research we explore the usage of sentence embeddings for multi-class classification to identify important sentences in a legal case, in the perspective of the main parties present in the case. In addition, a task-specific loss function is defined in order to improve the accuracy restricted by the straightforward use of categorical cross entropy loss.
... Sentiment analysis, the process of identifying and extracting subjective information from text, has become increasingly important in fields such as social media monitoring, customer service, and political discourse analysis. Traditional approaches to sentiment analysis often relied on rule-based systems or machine learning models trained on limited datasets [4]. However, the emergence of LLMs has brought about a paradigm shift, allowing for more accurate and scalable sentiment analysis [6] [5]. ...
The number of companies listed on the NYSE has been growing exponentially, creating a significant challenge for market analysts, traders, and stockholders who must monitor and assess the performance and strategic shifts of a large number of companies regularly. There is an increasing need for a fast, cost-effective, and comprehensive method to evaluate the performance and detect and compare many companies' strategy changes efficiently. We propose a novel data-driven approach that leverages large language models (LLMs) to systematically analyze and rate the performance of companies based on their SEC 10-K filings. These filings, which provide detailed annual reports on a company's financial performance and strategic direction, serve as a rich source of data for evaluating various aspects of corporate health, including confidence, environmental sustainability, innovation, and workforce management. We also introduce an automated system for extracting and preprocessing 10-K filings. This system accurately identifies and segments the required sections as outlined by the SEC, while also isolating key textual content that contains critical information about the company. This curated data is then fed into Cohere's Command-R+ LLM to generate quantitative ratings across various performance metrics. These ratings are subsequently processed and visualized to provide actionable insights. The proposed scheme is then implemented on an interactive GUI as a no-code solution for running the data pipeline and creating the visualizations. The application showcases the rating results and provides year-on-year comparisons of company performance.
... The study realized by [3] investigates the application of aspectbased sentiment analysis in the legal domain to extract valuable information from legal opinion text. The authors introduce a rule-based approach for conducting aspect-based sentiment analysis, aiming to determine the sentiment expressed in a sentence regarding each legal party involved in a court case, considering these parties as the aspects of analysis. ...
Sentiment analysis on social networks has become a highly active area of research in recent years. With the explosion of social media and the massive amount of user-generated data, it has become crucial to understand the opinions and sentiments expressed online. Sentiment analysis is used to categorize expressed feelings in various ways, such as negative, positive, or neutral. The aim of this work is to enhance techniques for researching and extracting opinions. The main idea is to identify opinions within a set of documents or texts available online for exploitation by other systems. In this study, we present an approach based on an opinion detection system on social networks (Facebook) regarding the UEFA Champions League. The implementation of this solution was carried out using the GATE platform (General Architecture for Text Engineering). This work thus contributes to the field of sentiment and opinion analysis in social networks by employing Gazetteers and leveraging the JAPE rules (Java Annotation Patterns Engine).
... Rajapaksha et al. [18] proposed a rule-based approach to perform aspect-based based SA in legal opinion texts. The Stanford NLP library was used to analyze the grammar of the sentence. ...
... For example, recent studies have shown that the domain specificity significantly impacts vital NLP tasks such as measuring semantic similarity (Sugathadasa et al., 2017) and domain specific text generation (Lebret et al., 2016). Further, it has been shown that models developed using data from a generic domain do not seamlessly transfer to tasks in a specific domain (Rajapaksha et al., 2020). Fantasy domains could be considered an extreme case of domain specific data, as it is possible to observe the full spectrum of deviations from the non-domain specific (general domain) usage, both in the lexical and semantic perspectives. ...
This paper introduces the Forgotten Realms Wiki (FRW) data set and domain specific natural language generation using FRW along with related analyses. Forgotten Realms is the de-facto default setting of the popular open ended tabletop fantasy role playing game, Dungeons & Dragons. The data set was extracted from the Forgotten Realms Fandom wiki consisting of more than over 45,200 articles. The FRW data set is constituted of 11 sub-data sets in a number of formats: raw plain text, plain text annotated by article title, directed link graphs, wiki info-boxes annotated by the wiki article title, Poincaré embedding of first link graph, multiple Word2Vec and Doc2Vec models of the corpus. This is the first data set of this size for the Dungeons & Dragons domain. We then present a pairwise similarity comparison benchmark which utilizes similarity measures. In addition , we perform D&D domain specific natural language generation using the corpus and evaluate the named entity classification with respect to the lore of Forgotten Realms.
... Likewise, Rajapaksha et al. acknowledges the importance of extracting crucial information from legal text in a fast and efficient manner to aid legal parties. To address this, they developed a model that is capable of efficiently perform this task [58]. Transformers have also found applications in aiding the criminal justice system, as exemplified by Alzahrani's study that employed various BERT models to analyze and profile the characteristics of cyber-criminals [59]. ...
Sexual violence is a pervasive and complex issue that demands an immediate and comprehensive solution. The previous study titled “LAW-U: Legal Guidance Through Artificial Intelligence Chatbot for Sexual Violence Victims and Survivors” highlighted the crucial role of technology in addressing this problem. The current study aims to overcome limitations in the previous study by investigating the use of transformer-based models for multi-intent classification of Thai Supreme Court decision fragments related to Section 276 of the Thai Criminal Code. Utilizing various evaluation matrices, the study evaluates the effectiveness of transfer learning through transformer-based pre-trained language models against a Word2Vec-based support vector machine to detect criminal intents from decision fragments. The results demonstrate that transformer-based models, particularly XLM-RoBERTa
BASE
, outperform the Word2Vec-based support vector machine in multi-intent classification. The macro average F1-score of 0.77 and micro average F1-score of 0.78 achieved by the best-performing model indicates the effectiveness of pre-trained transformers with fine-tuning. The study also employs a t-SNE visualization to gain insights into the overlapping between criminal intents and the areas where misclassifications occur. The visualization reveals that misclassification occur between closely related or overlapping intents, especially when decision fragments have multiple intents. Overall, the study contributes to the field of legal technology by creating a model that can accurately classify criminal intents related to Section 276, which can be extrapolated to other sexual violence laws. The model will be used to train the updated version of LAW-U, specifically called LAW-U-RoBERTa, which will provide legal recommendations to sexual violence survivors and empower them to reaffirm their inherent rights through seeking justice. The study demonstrates the potential of artificial intelligence chatbots to support survivors of sexual violence and contribute to the fight against the pervasive problem of sexual violence.
... For example, recent studies have shown that the domain specificity significantly impacts vital NLP tasks such as measuring semantic similarity (Sugathadasa et al., 2017) and domain specific text generation (Lebret et al., 2016). Further, it has been shown that models developed using data from a generic domain do not seamlessly transfer to tasks in a specific domain (Rajapaksha et al., 2020). Fantasy domains could be considered an extreme case of domain specific data, as it is possible to observe the full spectrum of deviations from the non-domain specific (general domain) usage, both in the lexical and semantic perspectives. ...
This paper introduces the Forgotten Realms Wiki (FRW) data set and domain specific natural language generation using FRW along with related analyses. Forgotten Realms is the de-facto default setting of the popular open ended tabletop fantasy role playing game, Dungeons & Dragons. The data set was extracted from the Forgotten Realms Fandom wiki consisting of more than over 45,200 articles. The FRW data set is constituted of 11 sub-data sets in a number of formats: raw plain text, plain text annotated by article title, directed link graphs, wiki info-boxes annotated by the wiki article title, Poincar\'e embedding of first link graph, multiple Word2Vec and Doc2Vec models of the corpus. This is the first data set of this size for the Dungeons & Dragons domain. We then present a pairwise similarity comparison benchmark which utilizes similarity measures. In addition, we perform D&D domain specific natural language generation using the corpus and evaluate the named entity classification with respect to the lore of Forgotten Realms.
... Specific traits in the language used in this field have challenged the researchers and developers working on NLP tools to tackle the existing defects in those tools and has constantly pushed them to significantly improve their work. Ontology population [14][15][16], discourse [26][27][28], semantic similarity [33,34], and sentiment analysis [11,[22][23][24][25]29] are some areas where extensive work has been carried out concerning the legal domain. ...
Since the advent of deep learning based Natural Language Processing (NLP), diverse domains of human society have benefited form automation and the resultant increment in efficiency. Law and order are, undoubtedly, crucial for the proper functioning of society; for without law there would be chaos, failing to offer equality to everyone. The legal domain being such a vital field, the incorporation of NLP into its workings has drawn attention in many research studies. This study attempts to leverage NLP into the task of extracting legal parties from legal opinion text documents. This task is of high importance given the significance of existing legal cases on contemporary cases under the legal practice, \textit{case law}. This study proposes a novel deep learning methodology which can be effectively used to resolve the problem of identifying legal party members in legal documents. We present two models here, where the first is a BRNN model to detect whether an entity is a legal party or not, and a second, a modification of the same neural network, to classify the thus identified entities into petitioner and defendant classes. Furthermore, in this study, we introduce a novel data set which is annotated with legal party information by an expert in the legal domain. With the use of the said dataset, we have trained and evaluated our models where the experiments carried out support satisfactory performance of our solution. The deep learning model we hereby propose, provides a benchmark for the legal party identification task on this data set. Evaluations for the solution presented in the paper show that our system has 90.89\% precision and 91.69\% recall for legal party extraction from an unseen paragraph from a legal document. As for the classification of petitioners and defendants, we show that GRU-512 obtains the highest F1 score.
This study analyses the feelings and opinions expressed in product reviews by utilizing NLTK (Natural Language Toolkit) and deep learning models to forecast customer sentiments and evaluate the probability of product purchases based on the review information available. The study examines the sentiment analysis applied to the Flipkart product reviews using a sentiment dataset. The NLTK, VADER, and RoBERTa models are evaluated for their effectiveness in predicting the sentiment of the customers. The analysis shows that the VADER, as a rule-based model is efficient in handling short and simple reviews but faces challenges with more complex sentiments. Meanwhile, RoBERTa outperforms VADER, with a Mean Absolute Error (MAE) of 0.12 and an R2 value of 0.85. The comparative study shows the ability of RoBERTa to capture subtle emotions in customer reviews and accurately understand customer feedback, proving to be valuable in e-commerce for optimizing product recommendations and customer satisfaction.
Information that are available in court case transcripts which describes the proceedings of previous legal cases are of significant importance to legal officials. Therefore, automatic information extraction from court case transcripts can be considered as a task of huge importance when it comes to facilitating the processes related to legal domain. A sentence can be considered as a fundamental textual unit of any document which is made up of text. Therefore, analyzing the properties of sentences can be of immense value when it comes to information extraction from machine readable text. This paper demonstrate how the properties of sentences can be used to extract valuable information from court case transcripts. As the first task, the sentence pairs were classified based on the relationship type which can be observed between the two sentences. There, we defined relationship types that can be observed between sentences in court case transcripts. A system combining a machine learning model and a rule-based approach was used to classify pairs of sentences according to the relationship type. The next classification task was performed based on whether a given sentence provides a legal argument or not. The results obtained through the proposed methodologies were evaluated using human judges. To the best of our knowledge, this is the first study where discourse relationships between sentences have been used to determine relationships among sentences in legal court case transcripts. Similarly, this study provides novel and effective approaches to identify argumentative sentences in a court case transcripts.
Selecting a representative vector for a set of vectors is a very common requirement in many algorithmic tasks. Traditionally, the mean or median vector is selected. Ontology classes are sets of homogeneous instance objects that can be converted to a vector space by word vector embeddings. This study proposes a methodology to derive a representative vector for ontology classes whose instances were converted to the vector space. We start by deriving five candidate vectors which are then used to train a machine learning model that would calculate a representative vector for the class. We show that our methodology out-performs the traditional mean and median vector representations.
Selecting a representative vector for a set of vectors is a very common requirement in many algorithmic tasks. Traditionally, the mean or median vector is selected. Ontology classes are sets of homogeneous instance objects that can be converted to a vector space by word vector embeddings. This study proposes a methodology to derive a representative vector for ontology classes whose instances were converted to the vector space. We start by deriving five candidate vectors which are then used to train a machine learning model that would calculate a representative vector for the class. We show that our methodology out-performs the traditional mean and median vector representations.
We describe the design and use of the Stanford CoreNLP toolkit, an extensible pipeline that provides core natural language analysis. This toolkit is quite widely used, both in the research NLP community and also among commercial and government users of open source NLP technology. We suggest that this follows from a simple, approachable design, straight-forward interfaces, the inclusion of robust and good quality analysis components, and not requiring use of a large amount of associated baggage.
Semantic similarity measures are an important part in Natural Language Processing tasks. However Semantic similarity measures built for general use do not perform well within specific domains. Therefore in this study we introduce a domain specific semantic similarity measure that was created by the synergistic union of word2vec, a word embedding method that is used for semantic similarity calculation and lexicon based (lexical) semantic similarity methods. We prove that this proposed methodology out performs word embedding methods trained on generic corpus and methods trained on domain specific corpus but do not use lexical semantic similarity methods to augment the results. Further, we prove that text lemmatization can improve the performance of word embedding methods.
A long-standing challenge in coreference resolution has been the incorporation of entity-level information - features defined over clusters of mentions instead of mention pairs. We present a neural network based coreference system that produces high-dimensional vector representations for pairs of coreference clusters. Using these representations, our system learns when combining clusters is desirable. We train the system with a learning to search algorithm that teaches it which local decisions (cluster merges) will lead to a high-scoring final coreference partition. The system substantially outperforms the current state-of-the-art on the English and Chinese portions of the CoNLL 2012 Shared Task dataset despite using few hand-engineered features.
Semantic word spaces have been very useful but cannot express the meaning of longer phrases in a principled way. Further progress towards understanding compositionality in tasks such as sentiment detection requires richer supervised training and evaluation resources and more powerful models of composition. To remedy this, we introduce a Sentiment Treebank. It includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment composition-ality. To address them, we introduce the Recursive Neural Tensor Network. When trained on the new treebank, this model outperforms all previous methods on several metrics. It pushes the state of the art in single sentence positive/negative classification from 80% up to 85.4%. The accuracy of predicting fine-grained sentiment labels for all phrases reaches 80.7%, an improvement of 9.7% over bag of features baselines. Lastly, it is the only model that can accurately capture the effects of negation and its scope at various tree levels for both positive and negative phrases.
In this article, a method for automatic sentiment analysis of movie reviews is proposed, implemented and evaluated. In contrast to most studies that focus on determining only sentiment orientation (positive versus negative), the proposed method performs fine-grained analysis to determine both the sentiment orientation and sentiment strength of the reviewer towards various aspects of a movie. Sentences in review documents contain independent clauses that express different sentiments toward different aspects of a movie. The method adopts a linguistic approach of computing the sentiment of a clause from the prior sentiment scores assigned to individual words, taking into consideration the grammatical dependency structure of the clause. The prior sentiment scores of about 32,000 individual words are derived from SentiWordNet with the help of a subjectivity lexicon. Negation is delicately handled. The output sentiment scores can be used to identify the most positive and negative clauses or sentences with respect to particular movie aspects.
Various approaches to aspect-based sentiment analysis
- bhoi
Fast approach to build an automatic sentiment annotator for legal domain using transfer learning
- gamage
Shift-of-perspective identification within legal cases
- ratnayaka