Conference Paper

Semantic Frame Identification with Distributed Word Representations

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

We present a novel technique for semantic frame identification using distributed representations of predicates and their syntactic context; this technique leverages automatic syntactic parses and a generic set of word embeddings. Given labeled data annotated with frame-semantic parses, we learn a model that projects the set of word representations for the syntactic context around a predicate to a low dimensional representation. The latter is used for semantic frame identification; with a standard argument identification method inspired by prior work, we achieve state-ofthe- art results on FrameNet-style framesemantic analysis. Additionally, we report strong results on PropBank-style semantic role labeling in comparison to prior work.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... It encompasses three subtasks: TI, FC and FSRL. Most of the previous studies [10,11,12,13,14,15] focus on FI or FSRL subtask individually, whereas others [16,17,18,19] simultaneously handle FC and FSRL subtasks, assuming the availability of all targets in advance. Limited work focus on TI and end-to-end FSP. ...
... Early studies [10,11,12] employ feature-engineering based methods construct models. Features usually include morphology (such as POS, lemma of the target), syntax (such as dependencies, child words, parent word of the target) and semantics (such as synonyms and collocations of the target) features. ...
... As for FI subtask, the previous studies can be divided into two categories according to whether frame knowledge is utilized, such as frame identification, frame relations. The methods without using frame knowledge [10,11,28] usually model FI as a span-based classification problem, while the methods using frame knowledge [29,30,31] usually model FI as a semantic similarity problem. ...
... Frame Identification (FI) aims to find the exact frame evoked by a target word in a given sentence. A frame represents an event scenario, and possesses frame elements (or semantic roles) that participate in the event (Hermann et al., 2014), which is described in the FrameNet knowledge base (Baker et al., 1998;Ruppenhofer et al., 2016) grounded on the theory of Frame Semantics (Fillmore et al., 2002). The theory asserts that people understand the meaning of words largely by virtue of the frames which they evoke. ...
... The recent studies of FI use distributed representations of target words and their syntactic context to construct features, and construct classification models with deep neural network (Hartmann et al., 2017;Kabbach et al., 2018). These methods usually transform frame labels into one-hot representations (Hermann et al., 2014;Täckström et al., 2015), and then learn the embeddings of target words and frames simultaneously. However, the abundant semantic information and structure knowledge of frames contained in FrameNet are still neglected. ...
... For instance, Hermann-14 (Hermann et al., 2014) implemented a model that jointly maps possible frame labels and the syntax context of target words into the same latent space using the WSABIE algorithm, and the syntax context was initialized with concatenating their word embeddings. SimpleFrameId (Hartmann et al., 2017) useed the concatenation of SentBOW (the average of embeddings of all the words in the sentence) to represent the context and then learns the common embedding space of context and frame labels following the line of (Hermann et al., 2014). The other trend is to construct the classifier model using deep neural network and regard discrete frame labels as supervision signals, which is similar to those earlier work. ...
... The word embedding approach not only reduces the need for expensive feature engineering of semantic structures but also leads to improved accuracy due to better generalization [20]. One of the earliest studies that leveraged distributed representations for frame identification was carried out by [21]. Their technique extracted the syntactic contexts of targets from data and generated an initial high-dimensional vector space using word embeddings. ...
... However, for the generation of the initial high-dimensional vectors, this technique still involved a great deal of feature engineering to extract features via syntactic parsing. In the same vein, a study conducted by [11] experimented with pre-trained word embeddings to represent targets and their contexts as inputs for learning two classification models: a two-layer neural network and a WSABIE model as proposed by [21]. Although their models did not outperform that of [21] for Das's benchmark test set [15], the results they obtained on out-of-domain data seemed to suggest that their models generalized better in other domains. ...
... In the same vein, a study conducted by [11] experimented with pre-trained word embeddings to represent targets and their contexts as inputs for learning two classification models: a two-layer neural network and a WSABIE model as proposed by [21]. Although their models did not outperform that of [21] for Das's benchmark test set [15], the results they obtained on out-of-domain data seemed to suggest that their models generalized better in other domains. ...
Preprint
Semantic parsing is the task of transforming sentences from natural language into formal representations of predicate-argument structures. Under this research area, frame-semantic parsing has attracted much interest. This parsing approach leverages the lexical information defined in FrameNet to associate marked predicates or targets with semantic frames, thereby assigning semantic roles to sentence components based on pre-specified frame elements in FrameNet. In this paper, a deep neural network architecture known as Positional Attention-based Frame Identification with BERT (PAFIBERT) is presented as a solution to the frame identification subtask in frame-semantic parsing. Although the importance of this subtask is well-established, prior research has yet to find a robust solution that works satisfactorily for both in-domain and out-of-domain data. This study thus set out to improve frame identification in light of recent advancements of language modeling and transfer learning in natural language processing. The proposed method is partially empowered by BERT, a pre-trained language model that excels at capturing contextual information in texts. By combining the language representation power of BERT with a position-based attention mechanism, PAFIBERT is able to attend to target-specific contexts in sentences for disambiguating targets and associating them with the most suitable semantic frames. Under various experimental settings, PAFIBERT outperformed existing solutions by a significant margin, achieving new state-of-the-art results for both in-domain and out-of-domain benchmark test sets.
... Specifically, we construct frame embeddings from the FrameNet graph structure and use them as input to a neural network. Although frame embeddings are often used in neural architectures for FrameID, prior work only learns these embeddings from frame-annotated corpus data (Hartmann et al., 2017;Hermann et al., 2014). To our knowledge, this is the first work to incorporate embeddings composed from the FrameNet graph structure itself, thus incorporating all frame, frame element, and predicate relationships that could otherwise be missing from corpus data. ...
... The first work to use embeddings for FrameID used the WSABIE algorithm to project frames and predicate contexts into the same shared space (Hermann et al., 2014). The authors then apply a pairwise loss to minimize the distance between the frames and their predicate instances. ...
... Our model underperforms compared to other embedding frameworks from Hermann et al. (2014) and Botschen et al. (2018), which can be explained through an examination of the input representation methods used by the different models, as well as their disambiguation strategies. The model by Hermann et al. (2014) constructs an input representation that encodes the syntactic dependency relations found within the predicate context by concatenating the embeddings for the arguments and learning a mapping to a lowerdimensional space. In this way it is similar to our recurrent neural network, but instead of learning the syntactic information implicitly, it feeds it directly, which potentially gives it an advantage. ...
Preprint
Full-text available
Lexical resources such as WordNet (Miller, 1995) and FrameNet (Baker et al., 1998) are organized as graphs, where relationships between words are made explicit via the structure of the resource. This work explores how structural information from these lexical resources can lead to gains in a downstream task, namely frame identification. While much of the current work in frame identification uses various neural architectures to predict frames, those neural architectures only use representations of frames based on annotated corpus data. We demonstrate how incorporating knowledge directly from the FrameNet graph structure improves the performance of a neural network-based frame identification system. Specifically, we construct a bidirectional LSTM with a loss function that incorporates various graph-and corpus-based frame embed-dings for learning and ultimately achieves strong performance gains with the graph-based embeddings over corpus-based embeddings alone.
... One is joint learning using distributed representation. Hartmann et al. [7] achieves higher scores for out-of-domain frame identification than previous systems via SimpleFrameId, which utilized SentBow (i.e., averaging the embeddings of all words in a sentence) after pointing out the sparsity of the feature space of Hermann's approach [18]. Hermann's system produced more interpretable output by taking the dependency information around the target word into a low-dimensional feature space and combining FI and FSRL with the WSABIE [19] algorithm. ...
... messages of lexical units and fuse such extra information with the target word using BERT. Hermann et al. [18] use syntactic information of a target word in the multitask structure. Botschen et al. [20] map the picture and text information into the same feature space. ...
Article
Full-text available
Frame identification, which is finding the exact evoked frame for a target word in a given sentence, is a fundamental and crucial prerequisite for frame semantic parsing. It is generally seen as a classification task for target words, whose contextual representations are usually obtained using a neural network like BERT as an encoder, and enriched with a joint learning model or the knowledge of FrameNet. However, the distinction at a fine-grained level, such as the delicate differences in the information of syntax and PropBank roles caused by different parts-of-speech (POS) of target words, is neglected. We propose a Multiple POS Dependency-aware Mixture of Experts(MPDaMoE) network that integrates five types of information, consisting of the syntactic information of target words whose POS are nominal, adjectival, adverbial, or prepositional, and the PropBank role information of target words whose POS are only verbal. To better learn such information, a Mixture of Experts network is employed, in which every expert is a Graph Convolutional Network, to incorporate the different dependency information of target words. Our model outperforms state-of-the-art models in experiments on two benchmark datasets, which shows its effectiveness.
... tween subtasks, they either focus on one or two subtasks (Hermann et al., 2014;FitzGerald et al., 2015;Marcheggiani and Titov, 2020) of frame semantic parsing or treat all subtasks independently Peng et al., 2018). Furthermore, in argument identification and role classification, previous approaches process each argument separately with sequence labeling strategy (Yang and Mitchell, 2017;Bastianelli et al., 2020) or spanbased graphical models Peng et al., 2018). ...
... A lot of previous systems neglect interactions between subtasks and relations between arguments. They either focus on one or two subtasks (Hermann et al., 2014;FitzGerald et al., 2015;Marcheggiani and Titov, 2020) of frame semantic parsing or treat all subtasks independently Peng et al., 2018). propose an efficient global graphical model, so they can enumerate all possible argument spans and treat the assignment as the Integer Linear Programming problem. ...
Preprint
Full-text available
Frame semantic parsing is a fundamental NLP task, which consists of three subtasks: frame identification, argument identification and role classification. Most previous studies tend to neglect relations between different subtasks and arguments and pay little attention to ontological frame knowledge defined in FrameNet. In this paper, we propose a Knowledge-guided Incremental semantic parser with Double-graph (KID). We first introduce Frame Knowledge Graph (FKG), a heterogeneous graph containing both frames and FEs (Frame Elements) built on the frame knowledge so that we can derive knowledge-enhanced representations for frames and FEs. Besides, we propose Frame Semantic Graph (FSG) to represent frame semantic structures extracted from the text with graph structures. In this way, we can transform frame semantic parsing into an incremental graph construction problem to strengthen interactions between subtasks and relations between arguments. Our experiments show that KID outperforms the previous state-of-the-art method by up to 1.7 F1-score on two FrameNet datasets. Our code is availavle at https://github.com/PKUnlp-icler/KID.
... Early work (Hermann et al., 2014;FitzGerald et al., 2015;Hartmann et al., 2017) on frame seman- * Corresponding author tic parsing adopts pipeline strategy. Their models apply independent models to handle different subtasks which ignore the interactions among subtasks. ...
... With the popularity of neural network and representation learning, neural network models are introduced to model frame semantic parsing problem. Hermann et al. (2014) uses distributed representations in frame identification and embedded both frames and the contextual representations of words into a shared low-dimension vector space. FitzGerald et al. (2015) uses a neural network to learn embeddings of both arguments and semantic roles, which adopts fine-grained similarity between roles to overcome the sparsity of some labeled data. ...
... Early neural approaches have focused in this regard on the integration of features extracted from dependency trees, both for the FI and SRL tasks (Hermann et al., 2014;Kshirsagar et al., 2015;Swayamdipta et al., 2017), with positive results. Amongst all, SRL is the task that has received more attention when investigating methods for injecting syntax into neural models, mostly due to the strict correlation between syntax and argument structures (Punyakanok et al., 2008). ...
... Syntax-aware neural models for Framesemantic parsing Identifying solutions to model syntax in neural models to improve Frame-semantic parsing related tasks has been a main concern in the community in the last five years. These have ranged from manual feature engineering (Hermann et al., 2014;Kshirsagar et al., 2015;Swayamdipta et al., 2017), to learning dependency path embeddings (Roth and Lapata, 2016), to encoding syntactic dependency structures via GCNs . Learning syntactic (Strubell et al., 2018) or semantic (Peng et al., 2018) dependency parsing as auxiliary tasks in a multi-task learning settings has found successful application, especially when the sub-task outputs were directly injected as features. ...
Preprint
Full-text available
We study the problem of integrating syntactic information from constituency trees into a neural model in Frame-semantic parsing sub-tasks, namely Target Identification (TI), FrameIdentification (FI), and Semantic Role Labeling (SRL). We use a Graph Convolutional Network to learn specific representations of constituents, such that each constituent is profiled as the production grammar rule it corresponds to. We leverage these representations to build syntactic features for each word in a sentence, computed as the sum of all the constituents on the path between a word and a task-specific node in the tree, e.g. the target predicate for SRL. Our approach improves state-of-the-art results on the TI and SRL of ~1%and~3.5% points, respectively (+2.5% additional points are gained with BERT as input), when tested on FrameNet 1.5, while yielding comparable results on the CoNLL05 dataset to other syntax-aware systems.
... Furthermore, text mining is looking also for modality in the text, it's the processing of analyzing the text to extraction the information that is useful for nominated purposes, compared with the kind of data we have been taking about. In this paper, the motivation of this aim paper for trying to extract the vector of text mining [1]. In addition, the NLP is the aim of the techniques in AI is to give an income to computers the capacity to realize natural language processing NLP. ...
... In computational linguistics and artificial intelligence, semantic representation deals with how word-meaning can extracted from the lexical source database or corpus. Therefore, the most extensive model which implements distinct semantic representation is WordNet as a lexical source [1], [2] it uses a semantic representation to compute the similarity of two words, a network model for representing huge number with nouns, and verbs in English. In WordNet 3.1, "Both nouns and verbs have their own semantic relations as well as their own organization based on the role they play in constructing linguistic messages. ...
Conference Paper
Full-text available
The textual analysis has become most important task due to the rapid increase of the number of texts that have been continuously generated in several forms such as posts and chats in social media, emails, articles, and news. The management of these texts requires efficient and effective methods, which can handle the linguistic issues that come from the complexity of natural languages. In recent years, the exploitation of semantic features from the lexical sources has been widely investigated by researchers to deal with the issues of "synonymy and ambiguity" in the tasks involved in the Social Media like document clustering. The main challenges of exploiting the lexical knowledge sources such as 1 WordNet 3.1 in these tasks are how to integrate the various types of semantic relations for capturing additional semantic evidence, and how to settle the high dimensionality of current semantic representing approaches. In this paper, the proposed weighting of features for a new semantic feature-based method as which combined four things as which is "Synonymy, Hypernym, non-taxonomy, and Glosses". Therefore, this research proposes a new knowledge-based semantic representation approach for text mining, which can handle the linguistic issues as well as the high dimensionality issue. Thus, the proposed approach consists of two main components: a feature-based method for incorporating the relations in the lexical sources, and a topic-based reduction method to overcome the high dimensionality issue. The proposed method approach will evaluated using WordNet 3.1 in the text clustering and text classification.
... Series: Materials Science and Engineering 769 (2020) 012002 IOP Publishing doi: 10.1088/1757-899X/769/1/012002 2 compared with the kind of data we have been taking about. In this paper, the motivation of this aim paper for trying to extract the vector of text mining [1]. In addition, the NLP is the aim of the techniques in AI is to give an income to computers the capacity to realize natural language processing NLP. ...
... In computational linguistics and artificial intelligence, semantic representation deals with how word-meaning can extracted from the lexical source database or corpus. Therefore, the most extensive model which implements distinct semantic representation is WordNet as a lexical source [1], [2] it uses a semantic representation to compute the similarity of two words, a network model for representing huge number with nouns, and verbs in English. In WordNet 3.1, "Both nouns and verbs have their own semantic relations as well as their own organization based on the role they play in constructing linguistic messages. ...
Article
Full-text available
The textual analysis has become the most important task due to the rapid increase of the number of texts that have been continuously generated in several forms such as posts and chats in social media, emails, articles, and news. The management of these texts requires efficient and effective methods, which can handle the linguistic issues that come from the complexity of natural languages. In recent years, the exploitation of semantic features from the lexical sources has been widely investigated by researchers to deal with the issues of “synonymy and ambiguity” in the tasks involved in the Social Media like document clustering. The main challenges of exploiting the lexical knowledge sources such as 1WordNet 3.1 in these tasks are how to integrate the various types of semantic relations for capturing additional semantic evidence, and how to settle the high dimensionality of current semantic representing approaches. In this paper, the proposed weighting of features for a new semantic feature-based method as which combined four things as which is “Synonymy, Hypernym, non-taxonomy, and Glosses”. Therefore, this research proposes a new knowledge-based semantic representation approach for text mining, which can handle the linguistic issues as well as the high dimensionality issue. Thus, the proposed approach consists of two main components: a feature-based method for incorporating the relations in the lexical sources, and a topic-based reduction method to overcome the high dimensionality issue. The proposed method approach will evaluated using WordNet 3.1 in the text clustering and text classification.
... Non-connectionist works that utilize distributional data in semantic analysis include Pennacchiotti et al. (2008) and Hermann et al. (2014). Pennacchiotti et al. (2008) employ distributional word representations for lexical unit induction, i.e. for extending FrameNet's scope by covering more frame-evoking lexical units. ...
... The underlying technique is wordsimilarity measurement through the comparison of distributional features. Hermann et al. (2014) also rely on distributional semantics in their proposal of a two-stage process of framesemantic parsing. In the first stage, frame identification and disambiguation are carried out. ...
Article
Full-text available
[Journal: ARGUMENTUM 14, 400-414, 2018. http://argumentum.unideb.hu/2018-anyagok/tothagoston.pdf] This paper reports the results of a series of experiments into recognizing semantic frames and frame elements using neural networks and measuring the added benefit of embedding large-scale co-occurrence information about words during the process. Frame recognition is carried out using Elman-type recurrent neural networks to give the system short-term memory of previous words within the sentence. Long-term memory is implemented in the system of weighted links between neurons. We test 9 word-representation methods including predict- and count-type distributional representations. We show that distributional word representations, which provide the frame recognizer with access to unlabelled co-occurrence information about every word, perform noticeably better than non-distributional techniques. Frame recognition F-score increased from 0.76 to 0.89, and frame element recognition – a considerably more difficult task – also benefited from the added information: we see an F-score increase from 0.46 to 0.53. We also show that this task is less sensitive to the particularities of collecting word distribution information than the known benchmark experiments.
... In these studies, frame-labeled corpora are used to learn embeddings for the frames as a side product of representation learning with different objective functions. Hermann et al. (2014) learned embeddings jointly for frames and the sentential contexts in which they were evoked. The current state-of-the-art in frame identification performs full-fledged semantic role labeling, i.e., it jointly assigns frames as well as frame elements, using a bi-directional LSTM architecture (Peng et al., 2018). ...
... The general evaluation metric for frame identification is accuracy: the relative frequency of correct assignments to predicates. Since the task of frame identification is moot for single-frame lexical units, frame identification systems standardly Peng et al., 2018;Hermann et al., 2014) report accuracy on two different subsets of the data: (1) all instances from the test set, called "Full Lexicon", because it includes lexical units that are unambiguous; and (2) only instances of predicates from the test set that can evoke multiple frames, called "Ambiguous". In the data set we use, the test partition contains 2,029 ambiguous predicates out of a total of 4,458 predicate instances. ...
Conference Paper
Full-text available
Categorization is a central capability of human cognition, and a number of theories have been developed to account for properties of categorization. Despite the fact that many semantic tasks involve categorization, theories of categorization do not play a major role in contemporary research in computational linguistics. This paper follows the idea that embedding-based models of semantics lend themselves well to being formulated in terms of classical categorization theories. The benefit is a group of models that enables (a) the formulation of hypotheses about the impact of major design decisions, and (b) a transparent assessment of these decisions. We instantiate this idea on the frame-semantic frame identification task. We define four models that cross two design variables: (a) the choice of prototype vs. exemplar categorization, corresponding to different degrees of generalization applied to the input, and (b) the presence vs. absence of a fine-tuning step, corresponding to generic vs. task-adaptive categorization. We find that for frame identification, generalization and task-adaptive categorization both yield substantial benefits. Our prototype-based, fine-tuned model, which combines the best choices over these variables, establishes a new state-of-the-art in frame identification.
... In the domain of supervised embeddings, SSI (Bai et al. 2009) and WSABIE (Weston, Bengio, and Usunier 2011) are early approaches that showed promise in NLP and information retrieval tasks ( , (Hermann et al. 2014)). Several more recent works including (Tang, Qin, and Liu 2015), (Zhang and LeCun 2015), (Conneau et al. 2016), TagSpace (Weston, Chopra, and Adams 2014) and fastText (Joulin et al. 2016) have yielded good results on classification tasks such as sentiment analysis or hashtag prediction. ...
Preprint
We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification, ranking tasks such as information retrieval/web search, collaborative filtering-based or content-based recommendation, embedding of multi-relational graphs, and learning word, sentence or document level embeddings. In each case the model works by embedding those entities comprised of discrete features and comparing them against each other -- learning similarities dependent on the task. Empirical results on a number of tasks show that StarSpace is highly competitive with existing methods, whilst also being generally applicable to new cases where those methods are not.
... For QAS there are various NLP models which are used. These models are symbol matching in which make use of linguistic annotation, structured world knowledge and semantic parsing [2] .There are various types of Symbolic matching models like frame-semantic parsing method, word distance benchmark method [3].These methods are not effective as they are only able to answer simple questions. The other method is using neural network. ...
Preprint
Question Answering System (QAS) is used for information retrieval and natural language processing (NLP) to reduce human effort. There are numerous QAS based on the user documents present today, but they all are limited to providing objective answers and process simple questions only. Complex questions cannot be answered by the existing QAS, as they require interpretation of the current and old data as well as the question asked by the user. The above limitations can be overcome by using deep cases and neural network. Hence we propose a modified QAS in which we create a deep artificial neural network with associative memory from text documents. The modified QAS processes the contents of the text document provided to it and find the answer to even complex questions in the documents.
... The use of embeddings for automatic frame identification in the monolingual case is not new. While early work used feature-based classification (Erk 2005), it was found that representations of frames through the embeddings of their FEEs matched and outperformed feature-based approaches (Hermann et al. 2014). However, the family of methods for creating embeddings available at that time was mostly monolingual. ...
Chapter
Full-text available
Frame semantics (Fillmore 1976) is a framework for the analysis of natural language semantics in terms of conceptual structures which can provide a semantic basis to constructions. Its cross-lingual applicability is subject to debate. Our study leverages recent developments in computational linguistics to assess the cross-lingual applicability of frames as defined in the FrameNet framework. Using a pre-trained multilingual model fine-tuned on frame identification, we test if frames have comparable representations in different languages and if we can identify frames cross-lingually using only English annotated examples. We show, using data from German and Korean, that frames are generally cross-lingually stable and identifiable; cross-lingual differences in lexicalization patterns are the biggest obstacle to cross-lingual application.
... Given that conceptual frames are formed by complex argument structures, their creation requires both linguistic and domain expertise, as well as tools for performing corpus-based searches , Hermann et al., 2014. When building entries of frame-based resources, lexicographers need to access specialized corpora, so that they can perform complex searches, involving verbs and their arguments. ...
Article
This paper focuses on Frame-based Terminology (FBT), a theoretical framework based on cognitive premises that underpin the representation of specialized objects and events. It demonstrates that specialized semantic frames can be inferred from lexical schemas and semantic annotation. This research used corpus techniques to collect data for the representation of multilingual semantic frames. The research focuses on deforestation within the domain of Environmental Science. The methodology involves the encapsulation of corpus queries and semantic annotation to guide the creation of specialized frames associated with the deforestation event. Although the creation of frame-based terminological resources can be time-consuming, our results show that it is facilitated by the use, of e-tools that extract the arguments of specialized predicates, which are often immersed in complex syntactic constructions.
... Some works focus only on a single parsing step, e.g. frame identification (Hermann et al., 2014;Hartmann et al., 2017); Sikos & Padó 2019, argument labeling with frame identification (Swayamdipta et al., 2017;Yang & Mitchell, 2017), or just argument labeling (Kshirsagar et al., 2015;Roth & Lapata, 2015;Swayamdipta et al., 2018), which can be considered as very similar to PropBank-style (Palmer et al., 2005) semantic role labeling, albeit more challenging because of the high granularity of semantic roles for frames. FrameNet-like resources are available only for very few languages and cover only a few domains. ...
Article
Full-text available
Semantic frames are formal structures describing situations, actions or events, e.g., Commerce buy, Kidnapping, or Exchange. Each frame provides a set of frame elements or semantic roles corresponding to participants of the situation and lexical units (LUs)—words and phrases that can evoke this particular frame in texts. For example, for the frame Kidnapping, two key roles are Perpetrator and the Victim, and this frame can be evoked with lexical units abduct, kidnap, or snatcher. While formally sound, the scarce availability of semantic frame resources and their limited lexical coverage hinders the wider adoption of frame semantics across languages and domains. To tackle this problem, firstly, we propose a method that takes as input a few frame-annotated sentences and generates alternative lexical realizations of lexical units and semantic roles matching the original frame definition. Secondly, we show that the obtained synthetically generated semantic frame annotated examples help to improve the quality of frame-semantic parsing. To evaluate our proposed approach, we decompose our work into two parts. In the first part of text augmentation for LUs and roles, we experiment with various types of models such as distributional thesauri, non-contextualized word embeddings (word2vec, fastText, GloVe), and Transformer-based contextualized models, such as BERT or XLNet. We perform the intrinsic evaluation of these induced lexical substitutes using FrameNet gold annotations. Models based on Transformers show overall superior performance, however, they do not always outperform simpler models (based on static embeddings) unless information about the target word is suitably injected. However, we observe that non-contextualized models also show comparable performance on the task of LU expansion. We also show that combining substitutes of individual models can significantly improve the quality of final substitutes. Because intrinsic evaluation scores are highly dependent on the gold dataset and the frame preservation, and cannot be ensured by an automatic evaluation mechanism because of the incompleteness of gold datasets, we also carried out experiments with manual evaluation on sample datasets to further analyze the usefulness of our approach. The results show that the manual evaluation framework significantly outperforms automatic evaluation for lexical substitution. For extrinsic evaluation, the second part of this work assesses the utility of these lexical substitutes for the improvement of frame-semantic parsing. We took a small set of frame-annotated sentences and augmented them by replacing corresponding target words with their closest substitutes, obtained from best-performing models. Our extensive experiments on the original and augmented set of annotations with two semantic parsers show that our method is effective for improving the downstream parsing task by training set augmentation, as well as for quickly building FrameNet-like resources for new languages or subject domains.
... For this work, we only use the dataset of fulltext annotations which provides 78 documents for FrameNet-1.5 and 108 documents for FrameNet-1.7. To generate data splits for both versions, we use 23 documents to extract the test set following the previous work Peng et al., 2018) and 16 documents are used as development set (Hermann et al., 2014), whereas the remaining documents are used as training set. Each frame is associated with one or more frame evoking elements commonly referred to as lexical-units. ...
Conference Paper
Full-text available
We investigate the semantic retrieval potential of pre-trained contextualized word embeddings (CWEs) such as BERT, in combination with explicit linguistic information, for various NLP tasks in an information retrieval setup. In this paper, we compare different strategies to aggregate contextualized word embeddings along lexical, syntactic, or grammatical dimensions to perform semantic retrieval for various natural language tasks. We apply this for fine-grained named entities, word senses, short texts, verb frames, and semantic relations, and show that incorporating certain linguistic knowledge improves the retrieval performance over various baselines. In a simulation study, we demonstrate the practical applicability of our findings to speed up the linguistic annotation of datasets. We also show that nearest neighbor classification , which implicitly uses the retrieval setup, works well with only small amounts of training data.
... Word embeddings can serve as input for supervised classification or regression models for specific tasks, whose training of course requires task-specific annotation ("fine tuning"). For frame identification, relatively straightfor-ward embedding-based classification was quickly able to match and outperform traditional feature-based models (Hermann et al., 2014) ...
Article
Full-text available
A recent research direction in computational linguistics involves efforts to make the field, which used to focus primarily on English, more multilingual and inclusive. However, resource creation often remains a bottleneck for many languages, in particular at the semantic level. In this article, we consider the case of frame-semantic annotation. We investigate how to perform frame selection for annotation in a target language by taking advantage of existing annotations in different, supplementary languages, with the goal of reducing the required annotation effort in the target language. We measure success by training and testing frame identification models for the target language. We base our selection methods on measuring frame transferability in the supplementary language, where we estimate which frames will transfer poorly, and therefore should receive more annotation, in the target language. We apply our approach to English, German, and French – three languages which have annotations that are similar in size as well as frames with overlapping lexicographic definitions. We find that transferability is indeed a useful indicator and supports a setup where a limited amount of target language data is sufficient to train frame identification systems.
... Each frame in the lexicon also enumerates several roles corresponding to facets of the scenario represented by the frame, where some roles can be omitted or nullinstantiated and left underspecified for the listener to infer (Ruppenhofer and Michaelis 2014). The problem of interpreting denominal verb usages can therefore be considered as inferring (the concepts evoked by) latent lexical unit(s) of the underlying semantic frame, which is itself related to the tasks of semantic frame identification (Hermann et al. 2014) and semantic role labeling (Gildea and Jurafsky 2002). Given the limited available resources for labeled or fully annotated data, many existing studies have considered a generative and semi-supervised learning approach to combine annotated lexical databases such as FrameNet (Baker, Fillmore, and Lowe 1998) and PropBank (Kingsbury and Palmer 2002) with other unannotated linguistic corpora. ...
Article
Full-text available
Humans can flexibly extend word usages across different grammatical classes, a phenomenon known as word class conversion. Noun-to-verb conversion, or denominal verb (e.g., to Google a cheap flight), is one of the most prevalent forms of word class conversion. However, existing natural language processing systems are impoverished in interpreting and generating novel denominal verb usages. Previous work has suggested that novel denominal verb usages are comprehensible if the listener can compute the intended meaning based on shared knowledge with the speaker. Here we explore a computational formalism for this proposal couched in frame semantics. We present a formal framework, Noun2Verb, that simulates the production and comprehension of novel denominal verb usages by modeling shared knowledge of speaker and listener in semantic frames. We evaluate an incremental set of probabilistic models that learn to interpret and generate novel denominal verb usages via paraphrasing. We show that a model where the speaker and listener cooperatively learn the joint distribution over semantic frame elements better explains the empirical denominal verb usages than state-of-the-art language models, evaluated against data from (1) contemporary English in both adult and child speech, (2) contemporary Mandarin Chinese, and (3) the historical development of English. Our work grounds word class conversion in probabilistic frame semantics and bridges the gap between natural language processing systems and humans in lexical creativity.
... Each frame in the lexicon also enumerates several roles corresponding to facets of the scenario represented by the frame, where some roles can be omitted or nullinstantiated and left under-specified for the listener to infer (Ruppenhofer and Michaelis 2014). The problem of interpreting denominal verb usages can therefore be considered as inferring (the concepts evoked by) latent lexical unit(s) of the underlying semantic frame, which is itself related to the tasks of semantic frame identification (Hermann et al. 2014) and semantic role labeling (Gildea and Jurafsky 2002). Given the limited available resources for labelled or fully annotated data, many existing studies have considered a generative and semi-supervised learning approach to combine annotated lexical databases such as FrameNet (Baker, Fillmore, and Lowe 1998) and PropBank (Kingsbury and Palmer 2002) with other unannotated linguistic corpora. ...
Preprint
Full-text available
Humans can flexibly extend word usages across different grammatical classes, a phenomenon known as word class conversion. Noun-to-verb conversion, or denominal verb (e.g., to Google a cheap flight), is one of the most prevalent forms of word class conversion. However, existing natural language processing systems are impoverished in interpreting and generating novel denominal verb usages. Previous work has suggested that novel denominal verb usages are comprehensible if the listener can compute the intended meaning based on shared knowledge with the speaker. Here we explore a computational formalism for this proposal couched in frame semantics. We present a formal framework, Noun2Verb, that simulates the production and comprehension of novel denominal verb usages by modeling shared knowledge of speaker and listener in semantic frames. We evaluate an incremental set of probabilistic models that learn to interpret and generate novel denominal verb usages via paraphrasing. We show that a model where the speaker and listener cooperatively learn the joint distribution over semantic frame elements better explains the empirical denominal verb usages than state-of-the-art language models, evaluated against data from 1) contemporary English in both adult and child speech, 2) contemporary Mandarin Chinese, and 3) the historical development of English. Our work grounds word class conversion in probabilistic frame semantics and bridges the gap between natural language processing systems and humans in lexical creativity.
... Previous work has shown the success of using feature engineering with linear classification models (Johansson and Nugues, 2007) and discriminative probabilistic models (Das et al., 2010), which were later improved by applying distributed word representations and deep neural network models (Hermann et al., 2014). Syntactic information, typically dependency paths, has consistently played an important role in frame identification Peng et al., 2018). ...
... In this section, we use the expanded FrameNet resource from §6.2 to improve model robustness on the task of Frame ID, a key subtask in FrameNet SRL (Das et al., 2010;Hermann et al., 2014). It is often prohibitively expensive to annotate entire documents under protocols such as FrameNet, and full-document annotation may not provide full coverage of the ontology due to the rarity of some ontological types. ...
Article
Full-text available
We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment. Our approach allows for the large-scale expansion of existing datasets or the rapid creation of new datasets using a small, manually produced seed corpus. We demonstrate our approach with experiments on the Berkeley FrameNet Project, a large-scale language understanding effort spanning more than two decades of human labor. With four days of training data collection for a span alignment model and one day of parallel compute, we automatically generate and release to the community 495,300 unique (Frame,Trigger) pairs in diverse sentential contexts, a roughly 50-fold expansion atop FrameNet v1.7. The resulting dataset is intrinsically and extrinsically evaluated in detail, showing positive results on a downstream task.
... The model presented in [4] uses latent variables and semi-supervised learning to improve frame disambiguation for targets unseen at training time. On the other hand, the work shown in [32] consists of a frame identification that is coupled into an argument parsing method to perform FSP. Sling, [33], is a framework for frame-semantic parsing that performs neural-network parsing with bidirectional LSTM input encoding and a transition based recurrent unit. ...
... The closely related task of frame identification has been addressed by [19]: in this work distributed representations of predicates and their syntactic context were exploited, paired with a general purpose set of word embeddings. Our work differs from the mentioned approach, in that we do not make use of syntactic information (since our input is very noisy, which would completely undermine parsing accuracy and reliability). ...
Article
Full-text available
Background Emergency room reports pose specific challenges to natural language processing techniques. In this setting, violence episodes on women, elderly and children are often under-reported. Categorizing textual descriptions as containing violence-related injuries (V) vs . non-violence-related injuries (NV) is thus a relevant task to the ends of devising alerting mechanisms to track (and prevent) violence episodes. Methods We present ViDeS (so dubbed after Violence Detection System ), a system to detect episodes of violence from narrative texts in emergency room reports. It employs a deep neural network for categorizing textual ER reports data, and complements such output by making explicit which elements corroborate the interpretation of the record as reporting about violence-related injuries. To these ends we designed a novel hybrid technique for filling semantic frames that employs distributed representations of terms herein, along with syntactic and semantic information. The system has been validated on real data annotated with two sorts of information: about the presence vs. absence of violence-related injuries, and about some semantic roles that can be interpreted as major cues for violent episodes, such as the agent that committed violence, the victim, the body district involved, etc.. The employed dataset contains over 150K records annotated with class (V,NV) information, and 200 records with finer-grained information on the aforementioned semantic roles. Results We used data coming from an Italian branch of the EU-Injury Database (EU-IDB) project, compiled by hospital staff. Categorization figures approach full precision and recall for negative cases and.97 precision and.94 recall on positive cases. As regards as the recognition of semantic roles, we recorded an accuracy varying from.28 to.90 according to the semantic roles involved. Moreover, the system allowed unveiling annotation errors committed by hospital staff. Conclusions Explaining systems’ results, so to make their output more comprehensible and convincing, is today necessary for AI systems. Our proposal is to combine distributed and symbolic (frame-like) representations as a possible answer to such pressing request for interpretability. Although presently focused on the medical domain, the proposed methodology is general and, in principle, it can be extended to further application areas and categorization tasks.
... resources but have not shown how the resulting data is useful for downstream tasks, other than as a means to guide future lexicographical additions. The dataset generated in §6.2 naturally lends itself to several downstream tasks such as word sense disambiguation (Das et al., 2010b) or Frame Identification, a major subtask (Das et al., 2010a;Hermann et al., 2014) of FrameNet semantic role labeling (SRL). In this section, we show how paraphrastic augmentation can improve Frame ID model robustness in low-resource settings. ...
Preprint
We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment. Our approach allows for the large-scale expansion of existing resources, or the rapid creation of new resources from a small, manually-produced seed corpus. We illustrate our framework on the Berkeley FrameNet Project, a large-scale language understanding effort spanning more than two decades of human labor. Based on roughly four days of collecting training data for the alignment model and approximately one day of parallel compute, we automatically generate 495,300 unique (Frame, Trigger) combinations annotated in context, a roughly 50x expansion atop FrameNet v1.7.
... The closely related task of frame identification has been addressed by [17]: in this work distributed representations of predicates and their syntactic context were exploited, paired with a general purpose set of word embeddings. Our work differs from the mentioned approach, in that we do not make use of syntactic information (since our input is very noisy, which would completely undermine parsing accuracy and reliability). ...
Preprint
Full-text available
Background Emergency room reports are a specific kind of text, posing specific challenges to natural language processing techniques. In this setting, violence episodes on women, elderly and children are often under-reported. Categorizing textual descriptions as containing violence-related injuries vs. non-violence-related injuries, is thus a relevant task, to the ends of devising alerting mechanisms to track violence episodes. Methods We present a system to detect episodes of violence from the textual descriptions contained in emergency room reports. It employs a deep neural network for categorizing textual ER reports data. Additionally, the system complements such output by making explicit which elements corroborate the interpretation of the record as reporting about violence-related injuries. To these ends we designed a novel hybrid technique for filling semantic frames that employs distributed representations of the terms herein, along with syntactic and semantic information. Results We tested our system on a set of real data of emergency room reports, coming from an Italian branch of the EU-Injury Database (EU-IDB) project, annotated by hospital staff. Our experimentation shows that the system produces accurate categorization (of violent vs. non violent records), paired with interesting results on the explanation of such output. At times, it also allowed unveiling annotation errors committed by hospital staff. Conclusions In the last few years deep architectures and word embeddings have been compared to a tsunami hitting AI and the area concerned with natural language processing. Only at a later time we have been realizing that the stunning output of deep networks needed to be explained: our proposal, combining distributed and symbolic (frame-like) representations are a possible answer to this pressing request for interpretability. Although the present application is focused on the medical domain, the proposed methodology is general and, in principle, it can be extended to further application areas and categorization tasks.
... localized) FrameNets, for instance. 62 In Hermann et al. (2014), the term Semantic Role Labelling is applied to PropBank-style (semantic) predicate-argument analysis and semantic parsing is a more general term that covers FrameNet-style labelling, too. In this model, frame identification needs input parsed with a dependency parser. ...
... They show that the main problem in domain adaptation for frame semantic parsing is the frame identification step and propose a more robust classifier using predicate and context embeddings to perform frame identification. This approach is suitable for cascade systems such as SEMAFOR , (Hermann et al., 2014) and (Yang and Mitchell, 2017). In this paper we propose to study the generalization issue within the framework of a sequence tagging semantic frame parser that performs frame selection and argument classification in one step. ...
Preprint
This paper addresses the issue of generalization for Semantic Parsing in an adversarial framework. Building models that are more robust to inter-document variability is crucial for the integration of Semantic Parsing technologies in real applications. The underlying question throughout this study is whether adversarial learning can be used to train models on a higher level of abstraction in order to increase their robustness to lexical and stylistic variations.We propose to perform Semantic Parsing with a domain classification adversarial task without explicit knowledge of the domain. The strategy is first evaluated on a French corpus of encyclopedic documents, annotated with FrameNet, in an information retrieval perspective, then on PropBank Semantic Role Labeling task on the CoNLL-2005 benchmark. We show that adversarial learning increases all models generalization capabilities both on in and out-of-domain data.
... They show that the main bottleneck in domain adaptation is at the Frame Identification step and propose a more robust classifier for this task, using predicate and context embeddings to perform Frame Identification. This approach is suitable for cascade systems such as SEMAFOR [19], [20]. In this paper we study the generalization issue within the framework of a sequence tagging semantic frame parser that performs frame selection and argument classification in one step. ...
Preprint
This paper presents a new semantic frame parsing model, based on Berkeley FrameNet, adapted to process spoken documents in order to perform information extraction from broadcast contents. Building upon previous work that had shown the effectiveness of adversarial learning for domain generalization in the context of semantic parsing of encyclopedic written documents, we propose to extend this approach to elocutionary style generalization. The underlying question throughout this study is whether adversarial learning can be used to combine data from different sources and train models on a higher level of abstraction in order to increase their robustness to lexical and stylistic variations as well as automatic speech recognition errors. The proposed strategy is evaluated on a French corpus of encyclopedic written documents and a smaller corpus of radio podcast transcriptions, both annotated with a FrameNet paradigm. We show that adversarial learning increases all models generalization capabilities both on manual and automatic speech transcription as well as on encyclopedic data.
... They show that the main bottleneck in domain adaptation is at the Frame Identification step and propose a more robust classifier for this task, using predicate and context embeddings to perform Frame Identification. This approach is suitable for cascade systems such as SEMAFOR [19], [20]. In this paper we study the generalization issue within the framework of a sequence tagging semantic frame parser that performs frame selection and argument classification in one step. ...
... Recent advances in the development of semantic role labeling (SRL) systems (e.g., Swayamdipta et al. 2018) offer the prospect of automating more of FrameNet's process (than at present), specifically the annotation of frame elements (i.e., semantic roles). Such SRL systems are based on existing annotated FN data, and exploit a range of different machine learning techniques , Hermann et al. 2014, Kshirsagar et al., 2015, Tckstrm et al., 2015. Not surprisingly, none of these systems attempt recognizing nullinstantiated frame elements, not least in part due to the difficulty of the task. ...
... These models do not have state-of-the-art performance (Hermann et al., 2014;FitzGerald et al., 2015), we picked them because they were accessible and allowed testing on a novel corpus. Finally, we evaluate the quality of the TC corpus, containing only the top frame picked by the crowd for every sentence. ...
... They show that the main problem in domain adaptation for frame semantic parsing is the frame identification step and propose a more robust classifier using predicate and context embeddings to perform frame identification. This approach is suitable for cascade systems such as SEMAFOR , (Hermann et al., 2014) and (Yang and Mitchell, 2017). In this paper we propose to study the generalization issue within the framework of a sequence tagging semantic frame parser that performs frame selection and argument classification in one step. ...
... We first formally define verb frames, which are used throughout this work to represent the semantic nature of verbs and objects, including missing information, in a structured way. Following the notation in the frame semantic parsing [32], [33] and thematic fit evaluation [19] literature, we define a verb frame as f = (v, r 1 , a 1 , . . . , r n , a n ), where v denotes the predicate, r i and a i denotes the i-th role and its argument respectively. ...
Preprint
Enabling robots to understand instructions provided via spoken natural language would facilitate interaction between robots and people in a variety of settings in homes and workplaces. However, natural language instructions are often missing information that would be obvious to a human based on environmental context and common sense, and hence does not need to be explicitly stated. In this paper, we introduce Language-Model-based Commonsense Reasoning (LMCR), a new method which enables a robot to listen to a natural language instruction from a human, observe the environment around it, and automatically fill in information missing from the instruction using environmental context and a new commonsense reasoning approach. Our approach first converts an instruction provided as unconstrained natural language into a form that a robot can understand by parsing it into verb frames. Our approach then fills in missing information in the instruction by observing objects in its vicinity and leveraging commonsense reasoning. To learn commonsense reasoning automatically, our approach distills knowledge from large unstructured textual corpora by training a language model. Our results show the feasibility of a robot learning commonsense knowledge automatically from web-based textual corpora, and the power of learned commonsense reasoning models in enabling a robot to autonomously perform tasks based on incomplete natural language instructions.
... • Inducing new structure, for example semantic frame detection through lexical patterns (Basili, Croce, Cao, et al., 2009;Coppola, Gangemi, Gliozzo, et al., 2009;Hermann, Das, Weston, et al., 2014). • Instantiating a predefined structure with data (Cardie, 1997;E. ...
Article
Full-text available
Throughout an information search, a user needs to make sense of the information found to create an understanding. This requires cognitive effort that can be demanding. Building on prior sensemaking models and expanding them with ideas from learning and cognitive psychology, we examined the use of cognitive mechanisms during individual sensemaking. We conducted a qualitative user study of 15 students who searched for and made sense of information for business analysis and news writing tasks. Through the analysis of think‐aloud protocols, recordings of screen movements, intermediate work products of sensemaking, including notes and concept maps, and final reports, we observed the use of 17 data‐driven and structure‐driven mechanisms for processing new information, examining individual concepts and relationships, and detecting anomalies. These cognitive mechanisms, as the basic operators that move sensemaking forward, provide in‐depth understanding of how people process information to produce sense. Meaningful learning and sensemaking are closely related, so our findings apply to learning as well. Our results contribute to a better understanding of the sensemaking process—how people think—and this better understanding can inform the teaching of thinking skills and the design of improved sensemaking assistants and mind tools.
... These models do not have state-of-the-art performance ( Hermann et al., 2014;FitzGerald et al., 2015), we picked them because they were accessible and allowed testing on a novel corpus. Finally, we evaluate the quality of the TC corpus, containing only the top frame picked by the crowd for every sentence. ...
Preprint
Full-text available
We present a resource for the task of FrameNet semantic frame disambiguation of over 5,000 word-sentence pairs from the Wikipedia corpus. The annotations were collected using a novel crowdsourcing approach with multiple workers per sentence to capture inter-annotator disagreement. In contrast to the typical approach of attributing the best single frame to each word, we provide a list of frames with disagreement-based scores that express the confidence with which each frame applies to the word. This is based on the idea that inter-annotator disagreement is at least partly caused by ambiguity that is inherent to the text and frames. We have found many examples where the semantics of individual frames overlap sufficiently to make them acceptable alternatives for interpreting a sentence. We have argued that ignoring this ambiguity creates an overly arbitrary target for training and evaluating natural language processing systems - if humans cannot agree, why would we expect the correct answer from a machine to be any different? To process this data we also utilized an expanded lemma-set provided by the Framester system, which merges FN with WordNet to enhance coverage. Our dataset includes annotations of 1,000 sentence-word pairs whose lemmas are not part of FN. Finally we present metrics for evaluating frame disambiguation systems that account for ambiguity.
... Most recently, Hermann et al. (2014) focused on frame identification using distributed representations of predicates and their syntactic context. Their system used word embeddings as the input and identified the semantic frames. ...
Thesis
Natural language texts frequently contain predicates whose complete understanding re- quires access to other parts of the discourse. Human readers can retrieve such infor- mation across sentence boundaries and infer the implicit piece of information. This capability enables us to understand complicated texts without needing to repeat the same information in every single sentence. However, for computational systems, resolv- ing such information is problematic because computational approaches traditionally rely on sentence-level processing and rarely take into account the extra-sentential context. In this dissertation, we investigate this omission phenomena, called implicit semantic role labeling. Implicit semantic role labeling involves identification of predicate argu- ments that are not locally realized but are resolvable from the context. For example, in ”What’s the matter, Walters? asked Baynes sharply.”, the ADDRESSEE of the predicate ask, Walters, is not mentioned as one of its syntactic arguments, but can be recoverable from the previous sentence. In this thesis, we try to improve methods for the automatic processing of such predicate instances to improve natural language pro- cessing applications. Our main contribution is introducing approaches to solve the data sparseness problem of the task. We improve automatic identification of implicit roles by increasing the amount of training set without needing to annotate new instances. For this purpose, we propose two approaches. As the first one, we use crowdsourcing to annotate instances of implicit semantic roles and show that with an appropriate task de- sign, reliable annotation of implicit semantic roles can be obtained from the non-experts without the need to present precise and linguistic definition of the roles to them. As the second approach, we combine seemingly incompatible corpora to solve the problem of data sparseness of ISRL by applying a domain adaptation technique. We show that out of domain data from a different genre can be successfully used to improve a baseline implicit semantic role labeling model, when used with an appropriate domain adapta- tion technique. The results also show that the improvement occurs regardless of the predicate part of speech, that is, identification of implicit roles relies more on semantic features than syntactic ones. Therefore, annotating instances of nominal predicates, for instance, can help to improve identification of verbal predicates’ implicit roles, we well. Our findings also show that the variety of the additional data is more important than its size. That is, increasing a large amount of data does not necessarily lead to a better model.
Article
Frame identification is a crucial and challenging task in frame semantic parsing, where the objective is to determine the most appropriate frame for a given target within its context. Existing approaches typically classify each target individually, overlooking the potential interactions among different targets. To address these limitations, we propose a novel two-step prediction framework that encompasses both local and global perspectives for target identification. Additionally, we leverage the frame ontology graph to enhance the interactions among multiple targets by incorporating rich frame ontology knowledge. Moreover, we introduce a dynamic masking strategy during model training. This strategy encourages our model to adopt a global view during prediction, avoiding local optimization. Experimental results demonstrate the superiority of our model compared to previous approaches on FrameNet1.5, and it achieves competitive performance on FrameNet1.7. Furthermore, supplementary experiments and analyses provide additional evidence of the effectiveness of our proposed model.
Article
Chinese FrameNet (CFN) is a scenario commonsense knowledge base (CKB) that plays an important role in research on Chinese language understanding. It is based on the theory of frame semantics and English FrameNet (FN). The CFN knowledge base contains a wealth of scenario commonsense knowledge, including frames, frame elements, and frame relations, as well as annotated instances with rich scenario-related labels on Chinese sentences and discourses. In this paper, we conduct a comprehensive overview of CFN from a commonsense perspective, covering topics such as scenario commonsense representation, CFN resources, and its applications. We also summarize recent breakthroughs and identify future research directions. First, we introduce the concept of scenario commonsense, including its definitions, examples, and representation methods, with a focus on the relationship between scenario commonsense and the frame concept in CFN. In addition, we provide a comprehensive overview of CFN resources and their applications, highlighting the newly proposed frame-based discourse representation and a human-machine collaboration framework for expanding the CFN corpus. Furthermore, we explore emerging topics such as expanding the CFN resource, improving the interpretability of machine reading comprehension, and using scenario CKBs for text generation.
Article
Frame-semantic Parsing (FSP) is a challenging and critical task in Natural Language Processing (NLP). Most of the existing studies decompose the FSP task into frame identification (FI) and frame semantic role labeling (FSRL) subtasks, and adopt a pipeline model architecture that clearly causes error propagation problem. On the other hand, recent jointly learning models aim to address the above problem and generally treat FSP as a span-level structured prediction task, which unfortunately leads to cascading error propagation problem between roles and less efficient solutions due to huge search space of roles. To address these problems, we reformulate the FSRL task into a target-aware relation classification task , and propose a novel and lightweight jointly learning framework that simultaneously processes three subtasks of FSP, including frame identification, argument identification and role classification. The novel task formulation and jointly learning with interaction mechanisms among subtasks can help improve the overall system performance, and reduce the search space and time complexity, compared with existing methods. Extensive experimental results demonstrate that our proposed model significantly outperforms ten state-of-the-art models in terms of F1 score across two benchmark datasets.
Article
Full-text available
This article presents PFN-PT, a robust system for the automatic semantic annotation of Portuguese, consisting of a new, parsing-oriented framenet and a rule-based frame- and role-tagger. The framenet provides almost 13,000 valency frames covering 7,300 verb lemmas with 10,700 senses. Frame and role tagging is achieved by iterated matching of syntactic structures and semantic noun types with slot-filler conditions in the framenet. We discuss design principles and present frame and role statistics. In an evaluation run on news data, the system achieved an overall F-score of 92.2% for frame senses.
Book
Full-text available
Journal special issue on the computational processing of Brazilian Portuguese
Chapter
In this chapter we give an overview of different types of lexical knowledge bases that are used in natural language processing (NLP). We cover widely known expert-built Lexical Knowledge Bases (LKBs), and also collaborative LKBs, i.e., those created by a large community of layman collaborators. First we define our terminology, then we give a broad overview of various kinds of LKBs that play an important role in NLP. For particular resource-specific details, we refer the reader to the respective reference publications.
Preprint
Frame semantic parsing is a semantic analysis task based on FrameNet which has received great attention recently. The task usually involves three subtasks sequentially: (1) target identification, (2) frame classification and (3) semantic role labeling. The three subtasks are closely related while previous studies model them individually, which ignores their intern connections and meanwhile induces error propagation problem. In this work, we propose an end-to-end neural model to tackle the task jointly. Concretely, we exploit a graph-based method, regarding frame semantic parsing as a graph construction problem. All predicates and roles are treated as graph nodes, and their relations are taken as graph edges. Experiment results on two benchmark datasets of frame semantic parsing show that our method is highly competitive, resulting in better performance than pipeline models.
Preprint
We present LOME, a system for performing multilingual information extraction. Given a text document as input, our core system identifies spans of textual entity and event mentions with a FrameNet (Baker et al., 1998) parser. It subsequently performs coreference resolution, fine-grained entity typing, and temporal relation prediction between events. By doing so, the system constructs an event and entity focused knowledge graph. We can further apply third-party modules for other types of annotation, like relation extraction. Our (multilingual) first-party modules either outperform or are competitive with the (monolingual) state-of-the-art. We achieve this through the use of multilingual encoders like XLM-R (Conneau et al., 2020) and leveraging multilingual training data. LOME is available as a Docker container on Docker Hub. In addition, a lightweight version of the system is accessible as a web demo.
Article
Full-text available
Frame semantics is a linguistic theory that has been instantiated for English in the FrameNet lexicon. We solve the problem of frame-semantic parsing using a two-stage statistical model that takes lexical targets (i.e., content words and phrases) in their sentential contexts and predicts frame-semantic structures. Given a target in context, the first stage disambiguates it to a semantic frame. This model uses latent variables and semi-supervised learning to improve frame disambiguation for targets unseen at training time. The second stage finds the target's locally expressed semantic arguments. At inference time, a fast exact dual decomposition algorithm collectively predicts all the arguments of a frame at once in order to respect declaratively stated linguistic constraints, resulting in qualitatively better structures than naïve local predictors. Both components are feature-based and discriminatively trained on a small set of annotated frame-semantic parses. On the SemEval 2007 benchmark data set, the approach, along with a heuristic identifier of frame-evoking targets, outperforms the prior state of the art by significant margins. Additionally, we present experiments on the much larger FrameNet 1.5 data set. We have released our frame-semantic parser as open-source software.
Article
Full-text available
This task consists of recognizing words and phrases that evoke semantic frames as defined in the FrameNet project (http: //framenet.icsi.berkeley.edu), and their semantic dependents, which are usually, but not always, their syntactic dependents (including subjects). The train-ing data was FN annotated sentences. In testing, participants automatically annotated three previously unseen texts to match gold standard (human) annotation, including pre-dicting previously unseen frames and roles. Precision and recall were measured both for matching of labels of frames and FEs and for matching of semantic dependency trees based on the annotation.
Article
Full-text available
We describe our contribution to the SemEval task on Frame-Semantic Structure Extrac-tion. Unlike most previous systems de-scribed in literature, ours is based on depen-dency syntax. We also describe a fully auto-matic method to add words to the FrameNet lexical database, which gives an improve-ment in the recall of frame detection.
Article
Full-text available
In this paper we describe the CoNLL-2005 shared task on Semantic Role La-beling. We introduce the specification and goals of the task, describe the data sets and evaluation methods, and present a general overview of the 19 systems that have con-tributed to the task, providing a compara-tive description and results.
Conference Paper
Full-text available
This paper takes a critical look at the features used in the semantic role tagging literature and show that the information in the input, gener- ally a syntactic parse tree, has yet to be fully exploited. We propose an additional set of fea- tures and our experiments show that these fea- tures lead to fairly signicant improvements in the tasks we performed. We further show that dierent features are needed for dierent sub- tasks. Finally, we show that by using a Maxi- mum Entropy classier and fewer features, we achieved results comparable with the best previ- ously reported results obtained with SVM mod- els. We believe this is a clear indication that developing features that capture the right kind of information is crucial to advancing the state- of-the-art in semantic analysis.
Conference Paper
Full-text available
Transition-based dependency parsers generally use heuristic decoding algorithms but can accommodate arbitrarily rich feature representations. In this paper, we show that we can improve the accuracy of such parsers by considering even richer feature sets than those employed in previous systems. In the standard Penn Treebank setup, our novel features improve attachment score form 91.4% to 92.9%, giving the best results so far for transition-based parsing and rivaling the best results overall. For the Chinese Treebank, they give a signficant improvement of the state of the art. An open source release of our parser is freely available.
Conference Paper
Full-text available
If we take an existing supervised NLP sys- tem, a simple and general way to improve accuracy is to use unsupervised word representations as extra word features. We evaluate Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeddings of words on both NER and chunking. We use near state-of-the-art supervised baselines, and find that each of the three word representations improves the accu- racy of these baselines. We find further improvements by combining di erent word representations. You can download our word features, for o -the-shelf use in existing NLP systems, as well as our code, here: http://metaoptimize. com/projects/wordreprs/
Article
Full-text available
We present a general framework for semantic role labeling. The framework combines a machine-learning technique with an integer linear programming-based inference procedure, which incorporates linguistic and structural constraints into a global decision process. Within this framework, we study the role of syntactic parsing information in semantic role labeling. We show that full syntactic parsing information is, by far, most relevant in identifying the argument, especially, in the very first stage—the pruning stage. Surprisingly, the quality of the pruning stage cannot be solely determined based on its recall and precision. Instead, it depends on the characteristics of the output candidates that determine the difficulty of the downstream problems. Motivated by this observation, we propose an effective and simple approach of combining different semantic role labeling systems through joint inference, which significantly improves its performance. Our system has been evaluated in the CoNLL-2005 shared task on semantic role labeling, and achieves the highest F1 score among 19 participants.
Article
Full-text available
Semantic role labeling, the computational identification and labeling of arguments in text, has become a leading task in computational linguistics today. Although the issues for this task have been studied for decades, the availability of large resources and the development of statistical machine learning methods have heightened the amount of effort in this field. This special issue presents selected and representative work in the field. This overview describes linguistic background of the problem, the movement from linguistic theories to computational practice, the major resources that are being used, an overview of steps taken in computational systems, and a description of the key issues and results in semantic role labeling (as revealed in several international evaluations). We assess weaknesses in semantic role labeling and identify important challenges facing the field. Overall, the opportunities and the potential for useful further research in semantic role labeling are considerable.
Article
Full-text available
The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.
Article
Full-text available
The Proposition Bank project takes a practical approach to semantic representation, adding a layer of predicate-argument information, or semantic role labels, to the syntactic structures of the Penn Treebank. The resulting resource can be thought of as shallow, in that it does not represent coreference, quantification, and many other higher-order phenomena, but also broad, in that it covers every instance of every verb in the corpus and allows representative statistics to be calculated. We discuss the criteria used to define the sets of semantic roles used in the annotation process and to analyze the frequency of syntactic/semantic alternations in the corpus. We describe an automatic system for semantic role tagging trained on the corpus and discuss the effect on its performance of various types of information, including a comparison of full syntactic parsing with a flat representation and the contribution of the empty “trace” categories of the treebank.
Conference Paper
In this paper we describe the CoNLL-2005 shared task on Semantic Role Labeling. We introduce the specification and goals of the task, describe the data sets and evaluation methods, and present a general overview of the 19 systems that have contributed to the task, providing a comparative description and results.
Conference Paper
Distributed representations of words have proven extremely useful in numerous natural lan-guage processing tasks. Their appeal is that they can help alleviate data sparsity problems common to supervised learning. Methods for inducing these representations require only unlabeled language data, which are plentiful for many natural languages. In this work, we induce distributed representations for a pair of languages jointly. We treat it as a multitask learning problem where each task corresponds to a single word, and task relatedness is derived from co-occurrence statistics in bilingual parallel data. These representations can be used for a number of crosslingual learning tasks, where a learner can be trained on annotations present in one language and applied to test data in another. We show that our representations are informative by using them for crosslingual document classification, where classifiers trained on these representations substantially outperform strong baselines (e.g. machine translation) when applied to a new language.
Article
In this paper we describe the CoNLL-2004 shared task: semantic role labeling. We intro- duce the specification and goal of the task, de- scribe the data sets and evaluation methods, and present a general overview of the systems that have contributed to the task, providing compar- ative description.
Article
This article presents general background information about the FrameNet project, including an introduction to its basic assumptions and goals, a description of its precursors, and information about its evolution during the six years of the project. The companion articles in this special issue of IJL describe various aspects of the project in greater detail.
Chapter
This paper locates the linguistic-semantic use of frames within a merging of the traditions of using the word 'frames' in general cognitive science, where its connection to language is missing or incidental, and in linguistics and artificial intelligence where it was explicitly used for identifying the combinatory possibilities of lexical units, both syntactically (as, e.g., subcategorization frames) or semantically (as case frames). Examples are offered of the possible kinds of empirical and cognitive studies of English words and texts, and a sketch is provided of a large research effort devoted to the creation of a corpus-supported frame-based lexicon of English.
Article
This paper describes NomBank, a project that will provide argument structure for instances of common nouns in the Penn Treebank II corpus. NomBank is part of a larger effort to add ad-ditional layers of annotation to the Penn Tree-bank II corpus. The University of Pennsylva-nia's PropBank, NomBank and other annota-tion projects taken together should lead to the creation of better tools for the automatic analy-sis of text. This paper describes the NomBank project in detail including its specifications and the process involved in creating the resource.
Conference Paper
We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Article
We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.
Chapter
A central goal of statistical language modeling is to learn the joint probability function of sequences of words in a language. This is intrinsically difficult because of the curse of dimensionality: a word sequence on which the model will be tested is likely to be different from all the word sequences seen during training. Traditional but very successful approaches based on n-grams obtain generalization by concatenating very short overlapping sequences seen in the training set. We propose to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. Generalization is obtained because a sequence of words that has never been seen before gets high probability if it is made of words that are similar (in the sense of having a nearby representation) to words forming an already seen sentence. Training such large models (with millions of parameters) within a reasonable time is itself a significant challenge. We report on several methods to speed-up both training and probability computation, as well as comparative experiments to evaluate the improvements brought by these techniques. We finally describe the incorporation of this new language model into a state-of-the-art speech recognizer of conversational speech.
Conference Paper
We describe a single convolutional neural net- work architecture that, given a sentence, out- puts a host of language processing predic- tions: part-of-speech tags, chunks, named en- tity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semanti- cally) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data ex- cept the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the general- ization of the shared tasks, resulting in state- of-the-art performance.
Conference Paper
In ranking with the pairwise classication ap- proach, the loss associated to a predicted ranked list is the mean of the pairwise classi- cation losses. This loss is inadequate for tasks like information retrieval where we prefer ranked lists with high precision on the top of the list. We propose to optimize a larger class of loss functions for ranking, based on an or- dered weighted average (OWA) (Yager, 1988) of the classication losses. Convex OWA ag- gregation operators range from the max to the mean depending on their weights, and can be used to focus on the top ranked ele- ments as they give more weight to the largest losses. When aggregating hinge losses, the optimization problem is similar to the SVM for interdependent output spaces. Moreover, we show that OWA aggregates of margin- based classication losses have good general- ization properties. Experiments on the Letor 3.0 benchmark dataset for information re- trieval validate our approach.
Conference Paper
We introduce a novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions. Our method learns vector space representations for multi-word phrases. In sentiment prediction tasks these representations outperform other state-of-the-art approaches on commonly used datasets, such as movie reviews, without using any pre-defined sentiment lexica or polarity shifting rules. We also evaluate the model's ability to predict sentiment distributions on a new dataset based on confessions from the experience project. The dataset consists of personal user stories annotated with multiple labels which, when aggregated, form a multinomial distribution that captures emotional reactions. Our algorithm can more accurately predict distributions over such labels compared to several competitive baselines.
Conference Paper
In this paper we propose a novel statistical language model to capture long-range se- mantic dependencies. Specifically, we ap- ply the concept of semantic composition to the problem of constructing predictive his- tory representations for upcoming words. We also examine the influence of the un- derlying semantic space on the composi- tion task by comparing spatial semantic representations against topic-based ones. The composition models yield reductions in perplexity when combined with a stan- dard n-gram language model over the n-gram model alone. We also obtain per- plexity reductions when integrating our models with a structured language model.
Conference Paper
This paper contributes a formalization of frame-semantic parsing as a structure prediction problem and describes an implemented parser that transforms an English sentence into a frame-semantic representation. It finds words that evoke FrameNet frames, selects frames for them, and locates the arguments for each frame. The system uses two feature-based, discriminative probabilistic (log-linear) models, one with latent variables to permit disambiguation of new predicate words. The parser is demonstrated to significantly outperform previously published results.
Conference Paper
Image annotation datasets are becoming larger and larger, with tens of millions of images and tens of thousands of possible annotations. We propose a strongly performing method that scales to such datasets by simultaneously learning to optimize precision at the top of the ranked list of annotations for a given image and learning a low-dimensional joint embedding space for both images and annotations. Our method, called WSABIE, both outperforms several baseline methods and is faster and consumes less memory.
Article
We present a system for identifying the semantic relationships, or semantic roles, filled by constituents of a sentence within a semantic frame. Given an input sentence and a target word and frame, the system labels constituents with either abstract semantic roles, such as Agent or Patient, or more domain-specific semantic roles, such as Speaker, Message, and Topic. The system is based on statistical classifiers trained on roughly 50,000 sentences that were hand-annotated with semantic roles by the FrameNet semantic labeling project. We then parsed each training sentence into a syntactic tree and extracted various lexical and syntactic features, including the phrase type of each constituent, its grammatical function, and its position in the sentence. These features were combined with knowledge of the predicate verb, noun, or adjective, as well as information such as the prior probabilities of various combinations of semantic roles. We used various lexical clustering algorithms to generalize across possible fillers of roles. Test sentences were parsed, were annotated with these features, and were then passed through the classifiers. Our system achieves 82% accuracy in identifying the semantic role of presegmented constituents. At the more difficult task of simultaneously segmenting constituents and identifying their semantic role, the system achieved 65% precision and 61% recall. Our study also allowed us to compare the usefulness of different features and feature combination methods in the semantic role labeling task. We also explore the integration of role labeling with statistical syntactic parsing and attempt to generalize to predicates unseen in the training data.
Article
This paper proposes a framework for repre- senting the meaning of phrases and sentences in vector space. Central to our approach is vector composition which we operationalize in terms of additive and multiplicative func- tions. Under this framework, we introduce a wide range of composition models which we evaluate empirically on a sentence similarity task. Experimental results demonstrate that the multiplicative models are superior to the additive alternatives when compared against human judgments.
Article
We present conditional random fields, a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Article
FrameNet is a three-year NSF-supported project in corpus-based computational lexicography, now in its second year (NSF IRI-9618838, "Tools for Lexicon Building"). The project's key features are (a) a commitment to corpus evidence for semantic and syntactic generalizations, and (b) the representation of the valences of its target words (mostly nouns, adjectives, and verbs) in which the semantic portion makes use of frame semantics. The resulting database will contain (a) descriptions of the semantic frames underlying the meanings of the words described, and (b) the valence representation (semantic and syntactic) of several thousand words and phrases, each accompanied by (c) a repre- sentative collection of annotated corpus attestations, which jointly exemplify the observed linkings between "frame elements" and their syntactic realizations (e.g. grammatical function, phrase type, and other syntactic traits). This report will present the project's goals and workflow, and information about the computational tools that have been adapted or created in-house for this work.
SemEval-2007 Task 19: Frame semantic structure extraction
  • C Baker
  • M Ellsworth
  • K Erk
C. Baker, M. Ellsworth, and K. Erk. 2007. SemEval-2007 Task 19: Frame semantic structure extraction. In Proceedings of SemEval.
A neural probabilistic language model
  • Y Bengio
  • R Ducharme
  • P Vincent
  • C Jauvin
Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155.
Introduction to the CoNLL-2004 shared task: Semantic role labeling
  • X Carreras
  • L Màrquez
X. Carreras and L. Màrquez. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of CoNLL.
The NomBank project: An interim report
  • A Meyers
  • R Reeves
  • C Macleod
  • R Szekely
  • V Zielinska
  • B Young
  • R Grishman
A. Meyers, R. Reeves, C. Macleod, R. Szekely, V. Zielinska, B. Young, and R. Grishman. 2004. The NomBank project: An interim report. In Proceedings of NAACL/HLT Workshop on Frontiers in Corpus Annotation.