Paul Kingsbury’s research while affiliated with University of Pennsylvania and other places

The Proposition Bank project takes a practical approach to semantic representation, adding a layer of predicate-argument information, or semantic role labels, to the syntactic structures of the Penn Treebank. The resulting resource can be thought of as shallow, in that it does not represent coreference, quantification, and many other higher-order phenomena, but also broad, in that it covers every instance of every verb in the corpus and allows representative statistics to be calculated. We discuss the criteria used to define the sets of semantic roles used in the annotation process and to analyze the frequency of syntactic/semantic alternations in the corpus. We describe an automatic system for semantic role tagging trained on the corpus and discuss the effect on its performance of various types of information, including a comparison of full syntactic parsing with a flat representation and the contribution of the empty “trace” categories of the treebank.

This paper presents a methodology for using the argument structure of sentences, as encoded by the PropBank project, to develop clusters of verbs with similar meaning and usage. These clusters can be favorably compared to the classes developed by the VerbNet project. The most interesting cases are those where the clustering methodology suggests new members for VerbNet classes which will then be associated with the semantic predicates for that class.

This paper presents our basic approach to creating Proposition Bank, which involves adding a layer of semantic annotation to the Penn English TreeBank. Without attempting to confirm or disconfirm any particular semantic theory, our goal is to provide consistent argument labeling that will facilitate the automatic extraction of relational data. An argument such as the window in John broke the window and in The window broke would receive the same label in both sentences. In order to ensure reliable human annotation, we provide our annotators with explicit guidelines for labeling all of the syntactic and semantic frames of each particular verb. We give several examples of these guidelines and discuss the inter-annotator agreement figures. We also discuss our current experiments on the automatic expansion of our verb guidelines based on verb class membership. Our current rate of progress and our consistency of annotation demonstrate the feasibility of the task.

This paper describes our approach to the development of a Proposition Bank, which involves the addition of semantic information to the Penn English Treebank. Our primary goal is the labeling of syntactic nodes with specific argument labels that preserve the similarity of roles such as the window in John broke the window and the window broke. After motivating the need for explicit predicate argument structure labels, we briefly discuss the theoretical considerations of predicate argument structure and the need to maintain consistency across syntactic alternations. The issues of consistency of argument structure across both polysemous and synonymous verbs are also discussed and we present our actual guidelines for these types of phenomena, along with numerous examples of tagged sentences and verb frames. Metaframes are introduced as a technique for handling similar frames among nearsynonymous verbs. We conclude with a summary of the current status of annotation process.

The success of interlingual annotation depends crucially on agreement as to the entities to be annotated, both in terms of the categories of entities and in terms of the names of the entities. Two independent annotation efforts at Penn resulted in annotations which had a comfortable amount of overlap but which still disagreed substantially on even the categories to be marked. The English annotation took a very rich approach to annotation, marking a great many relationships which were not explicitly mentioned in the conference guidelines but which seemed logical and necessary to avoid losing important information. The Chinese annotation, in contrast, took a conservative approach, more in line with the conference guidelines, therefore showing many fewer marked entities. This mismatch can be attributed to the lack of specific guidelines for annotation. In contrast, the PropBank annotation scheme, designed for predicate-argument structure, specifies the same set of markable entities in each language, while still capturing most of the information desired by the conference guidelines. This allows for a considerable savings in time and annotator effort, largely due to the large existing dictionary of predicted argument structure.

This paper presents our basic approach to creating Proposition Bank, which involves adding a layer of semantic annotation to the Penn English TreeBank. Without attempting to confirm or disconfirm any particular semantic theory, our goal is to provide consistent argument labeling that will facilitate the automatic extraction of relational data. An argument such as the window in John broke the window and in The window broke would receive the same label in both sentences. In order to ensure reliable human annotation, we provide our annotators with explicit guidelines for labeling all of the syntactic and semantic frames of each particular verb. We give several examples of these guidelines and discuss the inter-annotator agreement figures. We also discuss our current experiments on the automatic expansion of our verb guidelines based on verb class membership. Our current rate of progress and our consistency of annotation demonstrate the feasibility of the task.

We compare three projects that annotate semantic roles: PropBank, FrameNet, and SALSA. The first part of our analysis is a comparison of the different word sense distinction criteria underlying the annotation. Then, we study the effects of these criteria at the level of actual phenomena that require annotation. In particular, we discuss metaphor, support constructions, words with multiple meaning aspects, phrases realizing more than one semantic role, and nonlocal semantic roles.