Conference PaperPDF Available

InVeRo-XL: Making Cross-Lingual Semantic Role Labeling Accessible with Intelligible Verbs and Roles

Authors:

Abstract

Notwithstanding the growing interest in cross-lingual techniques for Natural Language Processing , there has been a surprisingly small number of efforts aimed at the development of easy-to-use tools for cross-lingual Semantic Role Labeling. In this paper, we fill this gap and present InVeRo-XL, an off-the-shelf state-of-the-art system capable of annotating text with predicate sense and semantic role labels from 7 predicate-argument structure inventories in more than 40 languages. We hope that our system-with its easy-to-use RESTful API and Web interface-will become a valuable tool for the research community , encouraging the integration of sentence-level semantics into cross-lingual downstream tasks. InVeRo-XL is available online at http://nlp.uniroma1.it/invero.
InVeRo-XL: Making Cross-Lingual Semantic Role Labeling
Accessible with Intelligible Verbs and Roles
Simone Conia
Sapienza University of Rome
conia@di.uniroma1.it
Riccardo Orlando
Babelscape, Italy
orlando@babelscape.com
Fabrizio Brignone
Babelscape, Italy
brignone@babelscape.com
Francesco Cecconi
Babelscape, Italy
cecconi@babelscape.com
Roberto Navigli
Sapienza University of Rome
navigli@diag.uniroma1.it
Abstract
Notwithstanding the growing interest in cross-
lingual techniques for Natural Language Pro-
cessing, there has been a surprisingly small
number of efforts aimed at the development
of easy-to-use tools for cross-lingual Seman-
tic Role Labeling. In this paper, we fill this
gap and present InVeRo-XL, an off-the-shelf
state-of-the-art system capable of annotating
text with predicate sense and semantic role la-
bels from 7 predicate-argument structure in-
ventories in more than 40 languages. We
hope that our system – with its easy-to-use
RESTful API and Web interface – will be-
come a valuable tool for the research commu-
nity, encouraging the integration of sentence-
level semantics into cross-lingual downstream
tasks. InVeRo-XL is available online at http:
//nlp.uniroma1.it/invero.
1 Introduction
Informally, Semantic Role Labeling (SRL) is of-
ten defined as the task of automatically answering
the question “Who did What, to Whom, Where,
When, and How?” (Màrquez et al.,2008). More
precisely, SRL aims at recovering the predicate-
argument structures within a sentence, providing
an explicit overlay that uncovers the underlying
semantics of text. For this reason, SRL is thought
to be key in enabling Natural Language Under-
standing (Navigli,2018). Today SRL is still an
open problem, with several research papers being
published each year at top-tier conferences, reveal-
ing novel insights and proposing better approaches.
Over the years, thanks to this active development,
SRL has been successfully exploited in a wide ar-
ray of downstream tasks that span across different
areas of Artificial Intelligence, from Natural Lan-
guage Processing (NLP) with Information Retrieval
(Christensen et al.,2010), Question Answering (He
et al.,2015), Machine Translation (Marcheggiani
et al.,2018) and Semantic Parsing (Banarescu et al.,
2013), to Computer Vision with Visual Semantic
Role Labeling (Gupta and Malik,2015) and Situa-
tion Recognition (Yatskar et al.,2016).
Recently, the growing interest in cross-lingual
NLP, supported by the increasingly wide availabil-
ity of pretrained multilingual language models such
as BERT (Devlin et al.,2019) and XLM-RoBERTa
(Conneau et al.,2020), has sparked renewed inter-
est in multilingual and cross-lingual SRL. In just a
few years, researchers have found ways to design
fully-neural end-to-end systems for SRL (Cai et al.,
2018), to take advantage of contextual word repre-
sentations (Peters et al.,2018;Li et al.,2019), to
achieve high performance on multiple languages
(He et al.,2019a;Conia and Navigli,2020), to
generate sense and role labels with sequence-to-
sequence models (Blloshmi et al.,2021) and to
perform SRL jointly across heterogeneous invento-
ries (Conia et al.,2021).
Since SRL is a task that involves complex lin-
guistic theories, inventories and techniques, there
have been efforts to develop easy-to-use tools that
offer automatic predicate sense and semantic role
annotations to users interested in the integration of
sentence-level semantics into downstream tasks.
Some notable examples include SENNA
1
(Col-
lobert et al.,2011), which uses an ensemble of
feature-based classifiers (Koomen et al.,2005), Al-
lenNLP’s SRL demo
2
, which provides a reimple-
mentation of a BERT-based model (Shi and Lin,
2019), and InVeRo (Conia et al.,2020), which of-
fers annotations according to two different linguis-
tic inventories, PropBank (Palmer et al.,2005) and
1https://ronan.collobert.com/senna
2https://demo.allennlp.org/semantic-role- labeling
VerbAtlas (Di Fabio et al.,2019). However, one
important drawback of the above-mentioned tools
is that they are able to perform SRL only in English,
which hinders the exploitation of their annotations
in multilingual and cross-lingual NLP.
In order to fill this gap, we build upon InVeRo
and propose its next major release, InVeRo-XL,
with the objective of making SRL accessible in
multiple languages. We rebuild InVeRo-XL from
the ground up to offer:
The first end-to-end system to tackle the
whole SRL pipeline in over 40 languages;
The first off-the-shelf system to provide SRL
annotations for 7 linguistic inventories;
A RESTful API service that can be queried
either online, so as not to install any software,
or offline, to maximize throughput;
A Web interface that provides a visualization
of the system output which can be useful for
teaching purposes, comparing linguistic theo-
ries, and prototyping new ideas.
We believe that InVeRo-XL can provide a stepping
stone for the integration of explicit sentence-level
semantics into cross-lingual tasks, attracting new
researchers to the field of SRL and its applications.
3
1.1 What’s New in InVeRo-XL
As previously mentioned, InVeRo-XL is the suc-
cessor of InVeRo. Although its main new feature is
the ability to provide predicate sense and semantic
role annotations in over 40 languages with 7 differ-
ent inventories, InVeRo-XL has been overhauled
to also improve several other important aspects. In
particular:
Preprocessing:
while its predecessor used
a very limited set of rules to preprocess En-
glish text, InVeRo-XL features a multilingual
preprocessing module based on spaCy and
Stanza;
SRL model:
the English-only model has been
replaced by a cross-lingual model that is able
to perform not only span-based SRL, but also
dependency-based SRL;
3
InVeRo-XL can be downloaded upon request at
http://
nlp.uniroma1.it/resources
. InVeRo-XL is licensed under
Creative Commons Attribution-NonCommercial-ShareAlike
4.0 International.
API:
the service is now able to handle batched
requests and documents of arbitrary length;
Offline usage:
InVeRo-XL is now available
for download, free for research purposes, al-
lowing users to host their own instance locally.
2 System Overview
In this Section, we provide an overview of the main
components of InVeRo-XL and how they interact,
describing in detail the preprocessing module (Sec-
tion 2.1) and the SRL model (Section 2.2).
2.1 Preprocessing
The previous version of InVeRo-XL preprocessed
an English sentence using a very limited and sim-
ple set of rules. In order to correctly support more
languages, InVeRo-XL now relies on both spaCy
(Honnibal et al.,2020) and Stanza (Qi et al.,2020)
to deal transparently with document splitting and to-
kenization. An automatic language detector based
on fastText
4
(Joulin et al.,2017) is used to dy-
namically choose between the two preprocessing
tools, depending on the language detected: spaCy
is faster for high-resource languages, e.g. English,
but also less reliable on lower-resource languages,
e.g. Catalan, for which our system falls back to
Stanza.
2.2 Model Architecture
In line with its predecessor, InVeRo-XL encapsu-
lates an SRL model that falls within the broad cat-
egory of end-to-end systems, tackling the whole
SRL pipeline – predicate identification, predicate
sense disambiguation, argument identification and
argument classification – in a single forward pass.
However, the design of the SRL model itself has
been completely revamped and now follows the ar-
chitecture recently proposed by Conia et al. (2021),
which is capable of performing cross-lingual SRL
with heterogeneous linguistic inventories. In the
following, we describe the main components of the
SRL model architecture provided by InVeRo-XL.
Multilingual word encoder.
The first compo-
nent of our model is a multilingual word en-
coder that takes advantage of a pretrained language
model, XLM-RoBERTa (Conneau et al.,2020), to
provide rich contextualized word representations.
4https://fasttext.cc
More formally, for each word
wi
in an input sen-
tence
w=hw1, w2, . . . , wni
of length
n
, it com-
putes an encoding
ei= Swish(Wwhi+bw)
as
a non-linear projection of the concatenation
hi
of
the corresponding hidden states of the four topmost
layers of the language model.
Universal word encoder.
The resulting se-
quence of multilingual word encodings
E=
he1,e2,...,eni
is then given to a “universal” word
encoder that computes a sequence of task-specific
timestep encodings
T=ht1,t2,...,tni
as fol-
lows:
tj
i=(eiif j= 0
tj1
iBiLSTMj
i(tj1)otherwise
T=htK
1,tK
2,...,tK
ni
where
BiLSTMj
i(·)
is the
i
-th timestep of the
j
-th
BiLSTM layer and
K
is the total number of BiL-
STM layers. The purpose of this encoder is to cre-
ate representations that are shared across languages
and inventories and are, therefore, “universal”.
Universal predicate-argument encoder.
Simi-
larly to the encoder above, the objective of the
universal predicate-argument encoder is to build
predicate-specific argument representations that lie
in a vector space shared across languages and inven-
tories. Assuming that
wp
is a predicate in the input
sentence
w
, this encoder builds a sequence
A
of
predicate-specific argument encodings as follows:
aj
i=(tptiif j= 0
aj1
iBiLSTMj
i(aj1)otherwise
A=haK0
1,aK0
2,...,aK0
ni
where
ti
is the
i
-th timestep encoding from the
universal sentence encoder,
tp
is the timestep of the
predicate
wp
,
aj
i
is the argument encoding for
wi
with respect to
wp
produced after the
j
-th BiLSTM
layer, and
K0
is the total number of BiLSTM layers.
Inventory-specific decoders.
Finally, the univer-
sal encodings are given to a set of classifiers in
order to obtain the desired output labels. More
specifically, for each inventory, we need three types
of output: i) whether a word
wi
is a predicate
wp
;
ii) the most appropriate sense
s
for a predicate
wp
;
iii) which semantic role
r
, possibly the null role,
exists between a word
wi
and a predicate
wp
. More
formally, our model features three classifiers for
each inventory Ias follows:
σp(wi|I) = Wp|ISwish(Wpti+bp) + bp|I
σs(wp|I) = Ws|ISwish(Wstp+bs) + bs|I
σr(wr|wp, I) = Wr|ISwish(Wrai+br) + br|I
where each
σ·(·)
provides a score distribution over
the possible output classes, i.e. two (true or false)
for predicate identification, the number of senses
of an inventory for predicate sense disambiguation,
and the number of semantic roles (including the
null role) of an inventory for argument labeling.
Miscellanea.
While we follow the architecture
proposed by Conia et al. (2021), the SRL model of
InVeRo-XL also comes with a small but significant
number of enhancements. One such enhancement
is that, while Conia et al. (2021) propose a model
for dependency-based SRL, our model is also able
to perform span-based SRL by treating spans as
sequences of BIO tags. In order to correctly de-
code valid spans at inference time, InVeRo-XL
makes use of a Viterbi decoder. Other improve-
ments include training the model with the RAdam
optimizer (Liu et al.,2020), ensuring that each
training batch features a balanced number of in-
stances for each language in the training set, and
searching randomly for better hyperparameter val-
ues.
2.3 Evaluation
Datasets.
We report the performance of InVeRo-
XL on two gold standard benchmarks for SRL:
CoNLL-2009 (Hajiˇ
c et al.,2009) for dependency-
based SRL and CoNLL-2012 (Pradhan et al.,2012)
for span-based SRL. To the best of our knowl-
edge, CoNLL-2009 is the largest benchmark for
multilingual SRL as it comprises six languages,
namely, Catalan, Chinese, Czech, English, German
and Spanish.
5
The main challenge of this bench-
mark is that each language was annotated with a
different predicate-argument structure inventory,
e.g. the English PropBank (Palmer et al.,2005)
for English, AnCora (Taulé et al.,2008) for Span-
ish/Catalan and PDT-Vallex (Hajic et al.,2003) for
Czech. While CoNLL-2009 is an ideal test bed for
evaluating the multilingual capabilities of an SRL
system, dependency-based annotations may look
unfamiliar to end users who are not used to the
5
Japanese is not available anymore from LDC due to li-
censing issues.
Catalan Czech German English Spanish Chinese
Span
AllenNLP’s SRL demo 86.5
InVeRo 86.2 –
InVeRo-XLspan-based 83.3 85.9 87.0 86.8 81.8 84.9
Dependency
Marcheggiani et al. (2017) — 86.0 87.7 80.3 81.2
Chen et al. (2019) 81.7 88.1 76.4 91.1 81.3 81.7
Cai and Lapata (2019b) 82.7 90.0 81.8 83.6
Cai and Lapata (2019a) — 83.8 91.2 82.9 85.0
Lyu et al. (2019) 80.9 87.5 75.8 90.1 80.5 83.3
He et al. (2019b) 86.0 89.7 81.1 90.9 85.2 86.9
Conia and Navigli (2020) 88.3 92.1 89.1 92.4 86.9 89.1
Conia et al. (2021) 88.0 91.5 88.0 91.8 86.3 87.7
InVeRo-XLdependency-based 88.7 92.1 89.9 92.1 87.2 89.1
Table 1: Comparison between InVeRo-XL and other recent systems for SRL. Top: F1scores on argument labeling
with pre-identified predicates using the official CoNLL-2005 scoring script on the CoNLL-2012 English test set
for span-based SRL and the CoNLL-2009 test sets converted from dependency-based to span-based as described
in Section 2.3.Bottom: F1scores on argument labeling and sense disambiguation with pre-identified predicates
using the official CoNLL-2009 scoring script on the test sets of the CoNLL-2009 shared task for dependency-based
multilingual SRL.
notion of syntactic/semantic heads. Therefore, dif-
ferently from Conia et al. (2021), we also adapt the
system to perform span-based SRL and evaluate
its effectiveness on the standard English datasets
of CoNLL-2012 and on CoNLL-2009, converting
dependency-based annotations to span-based anno-
tations. We convert an argument head to an argu-
ment span by considering all those words that fall
in the syntactic subtree whose root is the argument
head and discarding all those predicates for which
this conversion produces overlapping spans.
Experimental setup.
We train the dependency-
based SRL model on the standard training splits
of CoNLL-2009, making the model learn from all
six languages jointly. Instead, we train the span-
based SRL model on the union of the English train-
ing split of CoNLL-2012 and the Catalan, Chi-
nese, Czech, German and Spanish training sets
converted from dependency-based to span-based,
as explained above. Each model configuration is
trained for 30 epochs using the RAdam optimizer
with learning rates of 10
5
for the weights of XLM-
RoBERTa and 10
3
for the other weights. Follow-
ing standard practice, we select the model check-
point with highest F
1
score on the development
set.
Results.
Table 1(top) shows how InVeRo-XL
performs on the English test set of CoNLL-2012 for
span-based SRL compared to its previous release
(InVeRo) and the previously best-performing on-
line system for SRL (AllenNLP’s SRL demo). Not
only does InVeRo-XL achieve better results, but it
is also the only system that is capable of performing
span-based cross-lingual SRL, showing strong re-
sults on each of the non-English test sets of CoNLL-
2009 converted from dependency-based to span-
based as described in Section 2.3. Furthermore,
Table 1(bottom) shows that InVeRo-XL achieves
results that are comparable to or better than those
of current state-of-the-art models on 5 of the 6
languages of CoNLL-2009 for dependency-based
SRL, the key advantages being that our model is
part of a prepackaged tool with additional user-
friendly features (see Sections 3and 4).
3 The InVeRo-XL API
In order to facilitate the integration of predicate-
argument structure information into downstream
tasks, InVeRo-XL exposes its fully self-contained
end-to-end multilingual SRL pipeline through an
easy-to-use RESTful API. In the following, we pro-
vide an overview of the main functionalities of the
InVeRo-XL API, from its Resource API (Section
3.1) to its Model API (Section 3.2) and how to
host InVeRo-XL locally on a user’s own hardware
(Section 3.3). We refer users to the online documen-
tation for the complete list of supported languages
and inventories, together with other details.6
3.1 Resource API
The Resource API is a simple way for obtaining
semantic information about predicates using the
intelligible verb senses and semantic roles defined
by VerbAtlas, a large-scale predicate-argument
structure inventory which clusters WordNet synsets
(Miller,1992) that share similar semantic behav-
ior. The Resource API of InVeRo-XL builds upon
the functionalities provided by its predecessor with
the key difference that it now supports multiple
languages thanks to BabelNet 5.0
7
(Navigli and
Ponzetto,2012;Navigli et al.,2021), a multilin-
gual encyclopedic dictionary that provides uni-
fied access to several knowledge bases including
WordNet.
More specifically, the Resource API defines two
endpoints:
/api/verbatlas/predicate
: given a
predicate
p
, this endpoint retrieves the set of
VerbAtlas frames which include at least one
sense of p.
/api/verbatlas/frame
: given a Ver-
bAtlas frame
f
, this endpoint retrieves its
predicate-argument structure, i.e., the seman-
tic roles, and the WordNet/BabelNet synsets
that belong to f.
3.2 Model API
The Model API of InVeRo-XL has been updated
to not only take advantage of the new multilingual
SRL system but also to provide quality-of-life im-
provements. The Model API now accepts requests
in over 40 languages and returns semantic annota-
tions according to 7 linguistic inventories. On top
of this, the Model API is now able to process docu-
ments of arbitrary length and to handle batches of
documents in a single request.
More specifically, the Model API exposes an
endpoint named
/api/model
. This endpoint ac-
cepts
POST
requests with a
JSON
body containing
a list of input objects, one for each document the
user wishes to annotate. Each input object shall
specify the following fields:
text
: a mandatory field that contains the text
of the document.
6http://nlp.uniroma1.it/invero/api-documentation
7https://babelnet.org
[{
"tokens":[
...
{"index":2,"text":"volpe" },
{"index":3,"text":"salta" },
{"index":4,"text":"sul" },
...
],
"annotations":[
{
"tokenIndex":3,
"verbatlas":{
"sense":"GOFORWARD",
"arguments":[
{
"role":"Agent",
"score":1.0,
"span":[0,3]
},{
"role":"Destination",
"score":1.0,
"span":[4,6]
},
...
]
},
"englishPropbank":{...},
"chinesePropbank":{...},
"germanPropbank":{...},
"pdtVallex":{...},
"catalanAncora":{...},
"spanishAncora":{...}
}
...
]
}]
Figure 1: An example of a response from the Model
API for an input Italian sentence. The response con-
tains the tokenized input sentence and the automatic
SRL annotations according to 7 different linguistic in-
ventories.
lang
: an optional field that indicates the lan-
guage of the document. If omitted, InVeRo-
XL will use an automatic language detector
(see Section 2.1).
Each request to the Model API returns a
JSON
response containing a list of output objects, one
for each input document, containing the automatic
annotations according to each of the 7 linguistic
inventories, as shown in Figure 1.
Figure 2: The home page of the Web interface of InVeRo-XL. Users can search for predicate information (e.g. the
VerbAtlas frames a verb belongs to) and tag sentences in multiple languages with different linguistic inventories
(see Figure 3).
3.3 Offline Usage
One of the most requested features that is currently
missing from InVeRo is the possibility of running
an offline instance of the service so as to annotate
large quantities of text in a shorter time, indepen-
dently of the latency of the network and the volume
of requests being processed by our Web server. To
address this issue, InVeRo-XL is also distributed
as a Docker
8
image that can be deployed locally
on a user’s own hardware.
9
While network latency
is often a bottleneck for processing a request, an
offline instance of InVeRo-XL does not suffer from
such a constraint and can therefore benefit greatly
from running on better hardware, e.g. on GPU. We
distribute InVeRo-XL in two configurations:
invero-xl-span
is the configuration that
performs multilingual span-based SRL and is
the one used by InVeRo-XL’s Web server;
invero-xl-dependency
is an alterna-
tive configuration built to perform multilin-
gual dependency-based SRL.
Running a local instance of InVeRo-XL is also
simple. First, users are required to perform a one-
time setup to load one of the available images:
#!/bin/bash
docker load -i invero-xl-span_2.0.0.tar
8https://www.docker.com
9
Docker images for InVeRo-XL are freely available for
research purposes at http://nlp.uniroma1.it/resources.
After that, InVeRo-XL can be started with:
#!/bin/bash
PORT=12345
LANGUAGES="EN IT FR ZH"
docker run \
--name invero-xl-span-en \
-p $PORT:80 \
-e LANGUAGES=$LANGUAGES
invero-xl-span:2.0.0
Once started, users can forward their requests lo-
cally. We refer the reader to the online documenta-
tion for further details.10
4 Web Interface
Similarly to its predecessor, InVeRo-XL includes
a public-facing Web interface (Figures 2and 3)
that provides a visual environment for both the Re-
source API and the Model API, allowing users to
explore the main functionalities while also provid-
ing an intuitive overview of how an SRL system
annotates a sentence or a short document. Most
importantly, the Web interface of InVeRo-XL has
been updated to reflect the changes in the Model
API and the underlying SRL model; now users
can annotate text in 12 languages
11
and visualize
predicate senses and semantic roles in 7 linguistic
inventories on the fly, without having to write code.
Figure 3shows an example sentence in Italian
with its corresponding predicate senses and seman-
10http://nlp.uniroma1.it/invero/api- documentation
11
We limit the number of languages available on the Web
interface due to hardware constraints as Stanza and spaCy use
one preprocessing model for each language.
Figure 3: The beginning of the Divine Comedy by Dante Alighieri in Italian as tagged by InVeRo-XL with three
predicate-argument structure inventories – VerbAtlas, the Chinese PropBank and the Spanish AnCora. “Nel mezzo
del cammin di nostra vista mi ritrovai per una selva oscura, ché la diritta via era smarrita” translates into “Midway
upon the journey of our life I found myself within a forest dark, for the straightforward pathway had been lost”.
tic roles as provided by InVeRo-XL. Thanks to
a dropdown menu, users can immediately switch
from the labels of one inventory to those of an-
other, independently of the input language, without
reloading the Web page. We argue that this Web
interface should help teachers explain SRL to their
students, allow linguists to compare linguistic in-
ventories on particular case studies, attract new
researchers to the field, and inspire others to ex-
ploit SRL in downstream tasks or even real-world
scenarios.
5 Conclusion and Future Work
Over the years, the research community has greatly
advanced the field of SRL, proposing ever more
complex approaches to tackle the task more effec-
tively. However, despite the growing interest in
cross-lingual NLP, there have been very few efforts
to develop automatic tools to perform SRL in mul-
tiple languages. Our objective with InVeRo-XL is
to fill this gap and equip researchers with an easy-
to-use, high-performing system capable of provid-
ing predicate sense and semantic role annotations
in over 40 languages with 7 linguistic inventories.
Users can take advantage of our state-of-the-art sys-
tem for cross-lingual SRL through a RESTful API
that relieves them from the need to reimplement
complex neural models and/or to build an efficient
preprocessing/postprocessing pipeline.
Although InVeRo-XL is a major step forward
compared to its predecessor, we intend to further
improve our system by adopting future and more
advanced SRL model architectures and by includ-
ing new training datasets, such as UniteD-SRL
(Tripodi et al.,2021). We strongly believe that
InVeRo-XL will facilitate the integration of SRL
into downstream cross-lingual tasks, hopefully aid-
ing further advancements in cross-lingual Natural
Language Understanding.
Acknowledgments
The authors gratefully acknowledge
the support of the ERC Consolida-
tor Grant MOUSSE No. 726487
and the European Language Grid
project No. 825627 (Universal Se-
mantic Annotator, USeA) under the
European Union’s Horizon 2020 re-
search and innovation programme.
References
Laura Banarescu, Claire Bonial, Shu Cai, Madalina
Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin
Knight, Philipp Koehn, Martha Palmer, and Nathan
Schneider. 2013. Abstract Meaning Representation
for sembanking. In Proceedings of the 7th Linguis-
tic Annotation Workshop and Interoperability with
Discourse, pages 178–186, Sofia, Bulgaria.
Rexhina Blloshmi, Simone Conia, Rocco Tripodi, and
Roberto Navigli. 2021. Generating Senses and
RoLes: An end-to-end model for dependency- and
span-based Semantic Role Labeling. In Proceedings
of the Thirtieth International Joint Conference on
Artificial Intelligence, IJCAI-21, pages 3786–3793.
Main Track.
Jiaxun Cai, Shexia He, Zuchao Li, and Hai Zhao. 2018.
A full end-to-end semantic role labeler, syntactic-
agnostic over syntactic-aware? In Proceedings of
the 27th International Conference on Computational
Linguistics, pages 2753–2765, Santa Fe, New Mex-
ico, USA.
Rui Cai and Mirella Lapata. 2019a. Semi-supervised
Semantic Role Labeling with cross-view training. In
Proceedings of the 2019 Conference on Empirical
Methods in Natural Language Processing and the
9th International Joint Conference on Natural Lan-
guage Processing (EMNLP-IJCNLP), pages 1018–
1027, Hong Kong, China.
Rui Cai and Mirella Lapata. 2019b. Syntax-aware Se-
mantic Role Labeling without parsing.Trans. Assoc.
Comput. Linguistics, 7:343–356.
Xinchi Chen, Chunchuan Lyu, and Ivan Titov. 2019.
Capturing argument interaction in Semantic Role La-
beling with capsule networks. In Proceedings of
the 2019 Conference on Empirical Methods in Nat-
ural Language Processing and the 9th International
Joint Conference on Natural Language Processing
(EMNLP-IJCNLP), pages 5415–5425, Hong Kong,
China.
Janara Christensen, Mausam, Stephen Soderland, and
Oren Etzioni. 2010. Semantic Role Labeling for
open information extraction. In Proceedings of the
NAACL HLT 2010 First International Workshop on
Formalisms and Methodology for Learning by Read-
ing, pages 52–60, Los Angeles, California.
Ronan Collobert, Jason Weston, Léon Bottou, Michael
Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa.
2011. Natural language processing (almost) from
scratch.J. Mach. Learn. Res., 12:2493–2537.
Simone Conia, Andrea Bacciu, and Roberto Navigli.
2021. Unifying cross-lingual Semantic Role Label-
ing with heterogeneous linguistic resources. In Pro-
ceedings of the 2021 Conference of the North Amer-
ican Chapter of the Association for Computational
Linguistics: Human Language Technologies, pages
338–351, Online.
Simone Conia, Fabrizio Brignone, Davide Zanfardino,
and Roberto Navigli. 2020. InVeRo: Making Se-
mantic Role Labeling accessible with intelligible
verbs and roles. In Proceedings of the 2020 Con-
ference on Empirical Methods in Natural Language
Processing: System Demonstrations, pages 77–84,
Online.
Simone Conia and Roberto Navigli. 2020. Bridg-
ing the gap in multilingual Semantic Role Label-
ing: A language-agnostic approach. In Proceed-
ings of the 28th International Conference on Com-
putational Linguistics, pages 1396–1410, Barcelona,
Spain (Online).
Alexis Conneau, Kartikay Khandelwal, Naman Goyal,
Vishrav Chaudhary, Guillaume Wenzek, Francisco
Guzmán, Edouard Grave, Myle Ott, Luke Zettle-
moyer, and Veselin Stoyanov. 2020. Unsupervised
cross-lingual representation learning at scale. In
Proceedings of the 58th Annual Meeting of the Asso-
ciation for Computational Linguistics, pages 8440–
8451, Online.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. BERT: Pre-training of
deep bidirectional transformers for language under-
standing. In Proceedings of the 2019 Conference of
the North American Chapter of the Association for
Computational Linguistics: Human Language Tech-
nologies, Volume 1 (Long and Short Papers), pages
4171–4186, Minneapolis, Minnesota.
Andrea Di Fabio, Simone Conia, and Roberto Navigli.
2019. VerbAtlas: A novel large-scale verbal seman-
tic resource and its application to Semantic Role La-
beling. In Proceedings of the 2019 Conference on
Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natu-
ral Language Processing (EMNLP-IJCNLP), pages
627–637, Hong Kong, China.
Saurabh Gupta and Jitendra Malik. 2015. Visual Se-
mantic Role Labeling.arXiv, abs/1505.04474.
Jan Hajiˇ
c, Massimiliano Ciaramita, Richard Johans-
son, Daisuke Kawahara, Maria Antònia Martí, Lluís
Màrquez, Adam Meyers, Joakim Nivre, Sebastian
Padó, Jan Štˇ
epánek, Pavel Straˇ
nák, Mihai Surdeanu,
Nianwen Xue, and Yi Zhang. 2009. The CoNLL-
2009 shared task: Syntactic and semantic depen-
dencies in multiple languages. In Proceedings of
the Thirteenth Conference on Computational Natu-
ral Language Learning (CoNLL 2009): Shared Task,
pages 1–18, Boulder, Colorado.
Jan Hajic, Jarmila Panevová, Zdenka Urešová,
Alevtina Bémová, Veronika Kolárová, and Petr Pa-
jas. 2003. PDT-VALLEX: Creating a large-coverage
valency lexicon for treebank annotation. In Proceed-
ings of the second workshop on treebanks and lin-
guistic theories, volume 9, pages 57–68.
Luheng He, Mike Lewis, and Luke Zettlemoyer. 2015.
Question-answer driven Semantic Role Labeling:
Using natural language to annotate natural language.
In Proceedings of the 2015 Conference on Empiri-
cal Methods in Natural Language Processing, pages
643–653, Lisbon, Portugal.
Shexia He, Zuchao Li, and Hai Zhao. 2019a. Syntax-
aware multilingual Semantic Role Labeling. In
Proceedings of the 2019 Conference on Empirical
Methods in Natural Language Processing and the
9th International Joint Conference on Natural Lan-
guage Processing (EMNLP-IJCNLP), pages 5350–
5359, Hong Kong, China.
Shexia He, Zuchao Li, and Hai Zhao. 2019b. Syntax-
aware multilingual Semantic Role Labeling. In
Proceedings of the 2019 Conference on Empirical
Methods in Natural Language Processing and the
9th International Joint Conference on Natural Lan-
guage Processing, EMNLP-IJCNLP 2019, Hong
Kong, China, November 3-7, 2019, pages 5349–
5358.
Matthew Honnibal, Ines Montani, Sofie Van Lan-
deghem, and Adriane Boyd. 2020. spaCy:
Industrial-strength Natural Language Processing in
Python.
Armand Joulin, Edouard Grave, Piotr Bojanowski, and
Tomas Mikolov. 2017. Bag of tricks for efficient text
classification. In Proceedings of the 15th Confer-
ence of the European Chapter of the Association for
Computational Linguistics: Volume 2, Short Papers,
pages 427–431, Valencia, Spain.
Peter Koomen, Vasin Punyakanok, Dan Roth, and Wen-
tau Yih. 2005. Generalized inference with multiple
Semantic Role Labeling systems. In Proceedings
of the Ninth Conference on Computational Natural
Language Learning (CoNLL-2005), pages 181–184,
Ann Arbor, Michigan.
Zuchao Li, Shexia He, Hai Zhao, Yiqing Zhang, Zhu-
osheng Zhang, Xi Zhou, and Xiang Zhou. 2019.
Dependency or span, end-to-end uniform Seman-
tic Role Labeling. In The Thirty-Third AAAI Con-
ference on Artificial Intelligence, AAAI 2019, The
Thirty-First Innovative Applications of Artificial In-
telligence Conference, IAAI 2019, The Ninth AAAI
Symposium on Educational Advances in Artificial In-
telligence, EAAI 2019, Honolulu, Hawaii, USA, Jan-
uary 27 - February 1, 2019, pages 6730–6737.
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu
Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han.
2020. On the variance of the adaptive learning rate
and beyond. In 8th International Conference on
Learning Representations, ICLR 2020, Addis Ababa,
Ethiopia, April 26-30, 2020.
Chunchuan Lyu, Shay B. Cohen, and Ivan Titov. 2019.
Semantic Role Labeling with iterative structure re-
finement. In Proceedings of the 2019 Conference on
Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natu-
ral Language Processing (EMNLP-IJCNLP), pages
1071–1082, Hong Kong, China.
Diego Marcheggiani, Jasmijn Bastings, and Ivan Titov.
2018. Exploiting semantics in Neural Machine
Translation with graph convolutional networks. In
Proceedings of the 2018 Conference of the North
American Chapter of the Association for Computa-
tional Linguistics: Human Language Technologies,
Volume 2 (Short Papers), pages 486–492, New Or-
leans, Louisiana.
Diego Marcheggiani, Anton Frolov, and Ivan Titov.
2017. A simple and accurate syntax-agnostic neu-
ral model for dependency-based Semantic Role La-
beling. In Proceedings of the 21st Conference on
Computational Natural Language Learning (CoNLL
2017), pages 411–420, Vancouver, Canada.
Lluís Màrquez, Xavier Carreras, Kenneth C. Litkowski,
and Suzanne Stevenson. 2008. Special issue in-
troduction: Semantic Role Labeling: An introduc-
tion to the special issue.Computational Linguistics,
34(2):145–159.
George A. Miller. 1992. WordNet: A lexical database
for English. In Speech and Natural Language: Pro-
ceedings of a Workshop Held at Harriman, New
York, February 23-26, 1992.
Roberto Navigli. 2018. Natural Language Understand-
ing: Instructions for (present and future) use. In Pro-
ceedings of the Twenty-Seventh International Joint
Conference on Artificial Intelligence, IJCAI 2018,
July 13-19, 2018, Stockholm, Sweden, pages 5697–
5702.
Roberto Navigli, Michele Bevilacqua, Simone Conia,
Dario Montagnini, and Francesco Cecconi. 2021.
Ten years of BabelNet: A survey. In Proceedings
of the Thirtieth International Joint Conference on
Artificial Intelligence, IJCAI-21, pages 4559–4567.
Survey Track.
Roberto Navigli and Simone Paolo Ponzetto. 2012.
BabelNet: The automatic construction, evaluation
and application of a wide-coverage multilingual se-
mantic network.Artif. Intell., 193:217–250.
Martha Palmer, Daniel Gildea, and Paul Kingsbury.
2005. The Proposition Bank: An annotated cor-
pus of semantic roles.Computational Linguistics,
31(1):71–106.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt
Gardner, Christopher Clark, Kenton Lee, and Luke
Zettlemoyer. 2018. Deep contextualized word repre-
sentations. In Proceedings of the 2018 Conference
of the North American Chapter of the Association
for Computational Linguistics: Human Language
Technologies, Volume 1 (Long Papers), pages 2227–
2237, New Orleans, Louisiana.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue,
Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-
2012 shared task: Modeling multilingual unre-
stricted coreference in OntoNotes. In Joint Confer-
ence on EMNLP and CoNLL - Shared Task, pages
1–40, Jeju Island, Korea.
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton,
and Christopher D. Manning. 2020. Stanza: A
Python Natural Language Processing toolkit for
many human languages. In Proceedings of the 58th
Annual Meeting of the Association for Computa-
tional Linguistics: System Demonstrations, pages
101–108, Online.
Peng Shi and Jimmy Lin. 2019. Simple BERT mod-
els for relation extraction and semantic role labeling.
CoRR, abs/1904.05255.
Mariona Taulé, M. Antònia Martí, and Marta Recasens.
2008. AnCora: Multilevel annotated corpora for
Catalan and Spanish. In Proceedings of the Sixth In-
ternational Conference on Language Resources and
Evaluation (LREC’08), Marrakech, Morocco.
Rocco Tripodi, Simone Conia, and Roberto Navigli.
2021. UniteD-SRL: A unified dataset for span- and
dependency-based multilingual and cross-lingual Se-
mantic Role Labeling. In Proceedings of the 2021
Conference on Empirical Methods in Natural Lan-
guage Processing (EMNLP).
Mark Yatskar, Luke S. Zettlemoyer, and Ali Farhadi.
2016. Situation Recognition: Visual Semantic Role
Labeling for image understanding. In 2016 IEEE
Conference on Computer Vision and Pattern Recog-
nition, CVPR 2016, Las Vegas, NV, USA, June 27-30,
2016, pages 5534–5542.
... For USeA, we develop and encapsulate an SRL model that falls within the broad category of end-to-end systems, tackling the whole SRL pipeline -predicate identification, predicate sense disambiguation, argument identification and argument classification -in a single forward pass. Differently from other prepackaged SRL systems, such as InVeRo and Al-lenNLP's SRL demo, USeA is based on a multilingual neural model (Conia et al., 2021a;Conia et al., 2021b) which is able to perform state-of-the-art SRL not only across-languages, but also using different and heterogeneous linguistic inventories, namely, the English PropBank (Palmer et al., 2005), the Chinese Prop-Bank (Xue, 2008), AnCora (Taulé et al., 2008), and VerbAtlas (Di Fabio et al., 2019), inter alia. ...
Conference Paper
Full-text available
In this paper, we present the Universal Semantic Annotator (USeA), which offers the first unified API for high-quality automatic annotations of texts in 100 languages through state-of-the-art systems for Word Sense Disambiguation, Semantic Role Labeling and Semantic Parsing. Together, such annotations can be used to provide users with rich and diverse semantic information, help second-language learners, and allow researchers to integrate explicit semantic knowledge into downstream tasks and real-world applications.
Conference Paper
Full-text available
Multilingual and cross-lingual Semantic Role Labeling (SRL) have recently garnered increasing attention as multilingual text representation techniques have become more effective and widely available. While recent work has attained growing success, results on gold multilingual benchmarks are still not easily comparable across languages, making it difficult to grasp where we stand. For example, in CoNLL-2009, the standard benchmark for multilingual SRL, language-to-language comparisons are affected by the fact that each language has its own dataset which differs from the others in size, domains, sets of labels and annotation guidelines. In this paper, we address this issue and propose UNITED-SRL, a new benchmark for multilingual and cross-lingual, span-and dependency-based SRL. UNITED-SRL provides expert-curated parallel annotations using a common predicate-argument structure inventory, allowing direct comparisons across languages and encouraging studies on cross-lingual transfer in SRL. We release UNITED-SRL v1.0 at https://github.com/SapienzaNLP/united-srl.
Conference Paper
Full-text available
Multilingual and cross-lingual Semantic Role Labeling (SRL) have recently garnered increasing attention as multilingual text representation techniques have become more effective and widely available. While recent work has attained growing success, results on gold multilingual benchmarks are still not easily comparable across languages, making it difficult to grasp where we stand. For example, in CoNLL-2009, the standard benchmark for multilingual SRL, language-to-language comparisons are affected by the fact that each language has its own dataset which differs from the others in size, domains, sets of labels and annotation guidelines. In this paper, we address this issue and propose UNITED-SRL, a new benchmark for multilingual and cross-lingual, span-and dependency-based SRL. UNITED-SRL provides expert-curated parallel annotations using a common predicate-argument structure inventory, allowing direct comparisons across languages and encouraging studies on cross-lingual transfer in SRL. We release UNITED-SRL v1.0 at https://github.com/SapienzaNLP/united-srl.
Conference Paper
Full-text available
Despite the recent great success of the sequence-to-sequence paradigm in Natural Language Processing, the majority of current studies in Semantic Role Labeling (SRL) still frame the problem as a sequence labeling task. In this paper we go against the flow and propose GSRL (Generating Senses and RoLes), the first sequence-to-sequence model for end-to-end SRL. Our approach benefits from recently-proposed decoder-side pretraining techniques to generate both sense and role labels for all the predicates in an input sentence at once, in an end-to-end fashion. Evaluated on standard gold benchmarks, GSRL achieves state-of-the-art results in both dependency- and span-based English SRL, proving empirically that our simple generation-based model can learn to produce complex predicate-argument structures. Finally, we propose a framework for evaluating the robustness of an SRL model in a variety of synthetic low-resource scenarios which can aid human annotators in the creation of better, more diverse, and more challenging gold datasets. We release GSRL at github.com/SapienzaNLP/gsrl.
Conference Paper
Full-text available
While cross-lingual techniques are finding increasing success in a wide range of Natural Language Processing tasks, their application to Semantic Role Labeling (SRL) has been strongly limited by the fact that each language adopts its own linguistic formalism, from PropBank for English to AnCora for Spanish and PDT-Vallex for Czech, inter alia. In this work, we address this issue and present a unified model to perform cross-lingual SRL over heterogeneous linguistic resources. Our model implicitly learns a high-quality mapping for different formalisms across diverse languages without resorting to word alignment and/or translation techniques. We find that, not only is our cross-lingual system competitive with the current state of the art but that it is also robust to low-data scenarios. Most interestingly, our unified model is able to annotate a sentence in a single forward pass with all the inventories it was trained with, providing a tool for the analysis and comparison of linguistic theories across different languages. We release our code and model at https://github.com/SapienzaNLP/unify-srl.
Conference Paper
Full-text available
Recent research indicates that taking advantage of complex syntactic features leads to favorable results in Semantic Role Labeling. Nonetheless, an analysis of the latest state-of-the-art multilingual systems reveals the difficulty of bridging the wide gap in performance between high-resource (e.g., English) and low-resource (e.g., German) settings. To overcome this issue, we propose a fully language-agnostic model that does away with morphological and syntactic features to achieve robustness across languages. Our approach outperforms the state of the art in all the languages of the CoNLL-2009 benchmark dataset, especially whenever a scarce amount of training data is available. Our objective is not to reject approaches that rely on syntax, rather to set a strong and consistent language-independent baseline for future innovations in Semantic Role Labeling. We release our model code and checkpoints at https://github.com/SapienzaNLP/multi-srl.
Conference Paper
Full-text available
Semantic Role Labeling (SRL) is deeply dependent on complex linguistic resources and sophisticated neural models, which makes the task difficult to approach for non-experts. To address this issue we present a new platform named Intelligible Verbs and Roles (InVeRo). This platform provides access to a new verb resource, VerbAtlas, and a state-of-the-art pre-trained implementation of a neural, span-based architecture for SRL. Both the resource and the system provide human-readable verb sense and semantic role information, with an easy to use Web interface and RESTful APIs available at http://nlp.uniroma1.it/invero.
Conference Paper
Full-text available
We present VerbAtlas, a new, hand-crafted lexical-semantic resource whose goal is to bring together all verbal synsets from WordNet into semantically-coherent frames. The frames define a common, prototypical argument structure while at the same time providing new concept-specific information. In contrast to PropBank, which defines enumerative semantic roles, VerbAtlas comes with an explicit, cross-frame set of semantic roles linked to selectional preferences expressed in terms of WordNet synsets, and is the first resource enriched with semantic information about implicit, shadow, and default arguments. We demonstrate the effectiveness of VerbAtlas in the task of dependency-based Semantic Role Labeling and show how its integration into a high-performance system leads to improvements on both the in-domain and out-of-domain test sets of CoNLL-2009. VerbAtlas is available at http://verbatlas.org.
Conference Paper
The intelligent manipulation of symbolic knowledge has been a long-sought goal of AI. However, when it comes to Natural Language Processing (NLP), symbols have to be mapped to words and phrases, which are not only ambiguous but also language-specific: multilinguality is indeed a desirable property for NLP systems, and one which enables the generalization of tasks where multiple languages need to be dealt with, without translating text. In this paper we survey BabelNet, a popular wide-coverage lexical-semantic knowledge resource obtained by merging heterogeneous sources into a unified semantic network that helps to scale tasks and applications to hundreds of languages. Over its ten years of existence, thanks to its promise to interconnect languages and resources in structured form, BabelNet has been employed in countless ways and directions. We first introduce the BabelNet model, its components and statistics, and then overview its successful use in a wide range of tasks in NLP as well as in other fields of AI.