Charles B. Callaway

• Ph.D. Computer Science, NCSU
• Senior Researcher at Fondazione Bruno Kessler

Wearables offer an attractive platform for interacting intelligently with our environment and ourselves. Commercially available wearables are not aimed at the academic/research environment. They have proprietary protocols, do not willingly share recorded data or information on how it was processed and filtered, and do not have the right combination...

Small groups can have a better museum visit when that visit is both a social and an educational occasion. The unmediated discussion that often ensues during a shared cultural experience, especially when it is with a small group whose members already know each other, has been shown by ethnographers to be important for a more enriching experience. We...

Well-being in a small group can be tied to how much its members interact. Small group tours are social occasions, and the discussion that ensues has been shown by ethnographers to be important for a more enriching experience. Increasing conversation can thus be seen as a way to improve social and psychological well-being.We present DramaTric, a mob...

Museum visits can be more enjoyable to small groups if they can be both social and educational experiences. One very rewarding aspect of a visit, especially those involving small groups such as families, is the unmediated group discussion that can ensue during a shared cultural experience. We present a situated, mobile museum system that delivers a...

Museum visits can be more enjoyable to small groups if they can be both social and educational experiences. One very rewarding aspect of a visit, especially those involving small groups such as families, is the unmediated group discussion that can ensue during a shared cultural experience. We present a museum mobile system that perceives and analyz...

Multilingual generation is becoming an increasingly important aspect of imple-mented systems that showcase the abil-ities of generation systems. Most such systems require multiple grammars, one for each language which must be de-ployed. Yet little is known about the development costs for additional lan-guages which are developed not from scratch, b...

We present Beetle II, a tutorial dialogue system designed to accept unrestricted language input and support experimentation with different tutorial planning and dialogue strategies. Our first system evaluation used two different tutoring policies and demonstrated that Beetle II can be successfully used as a platform to study the impact of different...

Most Natural Language Generation systems focus on the production of perfectly written text for visual presentation. But as NLG tries to move into other areas of language production, such as speech in di-alogue systems, it is not clear that approaches for traditional applications are best. Speech exhibits particular features like disfluency that are...

We present BEETLE II, a tutorial dialogue system designed to accept unrestricted language input and support experimentation with different tutorial planning and dialogue strategies. Our first system evaluation used two different tutorial policies and demonstrated that the system can be successfully used to study the impact of different approaches t...

Museum visits can be more useful to small groups if they can be the centerpiece of a social experience as well as an educational one. One of the most rewarding aspects of a visit, especially those involving families, is the unmediated group discussion that can ensue during a shared cultural experience. However, if current methods were to be applied...

We propose that a set of five dimensions forms a foundation underlying a number of prevalent theoretical perspectives on learning. We show how student contributions to instructional dialogue can be reliably annotated with these dimensions. Finally, we provide preliminary validation evidence for our coding scheme and illustrate the potential value o...

We describe an approach to dealing with interpretation errors in a tutorial dialogue system. Allowing students to provide ex- planations and generate contentful talk can be helpful for learning, but the language that can be understood by a computer sys- tem is limited by the current technology. Techniques for dealing with understanding problems hav...

Understanding answers to open-ended explanation questions is important in intelligent tutoring systems. Existing systems use natural language techniques in essay analysis, but revert to scripted interaction with short-answer questions during remediation, making adapting dialogue to individual students difficult. We describe a corpus study that show...

Abstract The lack of large amounts of readily available, explicitly represented knowledge,has long been recognized as a barrier to applications requir- ing semantic knowledge,such as machine,translation and question an- swering. This problem,is analogous to that facing machine,translation decades ago, where one proposed solution was to use human tr...

The study of intelligent user interfaces and user modeling and adaptation is well suited for augmenting educational visits to museums. We have defined a novel integrated framework for museum visits and claim that such a framework is essential in such a vast domain that inherently implies complex interactivity. We found that it requires a significan...

We present tutorial dialogue systems in two different domains that demonstrate the use of dialogue management and deep natural language processing techniques. Generation techniques are used to produce natural sounding feedback adapted to stu- dent performance and the dialogue his- tory, and context is used to interpret ten- tative answers phrased a...

We present an empirical approach to adaptively selecting a tutoring system's remediation strategy based on an annotated corpus of human-human tutorial dialogues. We are interested in the remediation selection problem, that of generating the best remediation strategy given a diagnosis for an incorrect answer and the current problem solving context....

Since the Web by far represents the largest public repository of natural language texts, recent experiments, methods, and tools in the area of corpus linguistics often use the Web as a corpus. For applications where high accuracy is crucial, the problem ...

We discuss how deep interpretation and generation can be integrated with a know-ledge representation designed for question answering to build a tutorial dialogue sys-tem. We use a knowledge representa-tion known to perform well in answering exam-type questions and show that to sup-port tutorial dialogue it needs additional features, in particular,...

We discuss a set of tools for annotating a complex hierarchical and linguistic structure of tutorial dialogue based on the NITE XML Toolkit (NXT) (Carletta et al., 2003). The NXT API supports multi-layered stand-off data annotation and synchronisation with timed and speech data. Using NXT, we built a set of extensible tools for detailed structure a...

This report represents the Student Model Dialogue deliverable (D32) and describes in detail the improvements that have been made to the dialogue component of the Open Learner Model (OLM) since its original description in D29. In this document, the term “dialogue” in the Open Learner Model should not be understood in the broad sense of open-ended co...

A personalized multimedia museum visitor’s guide system may be a valuable tool for improving user satisfaction in a museum visit. Personalization poses challenges to user modeling in the museum environment, especially when several different applications are supported by the same user model, where it is required to operate in a non-intrusive manner....

Automatically constructing a complete documentary or educational film from scattered pieces of images and knowledge is a significant challenge. Even when this information is provided in an annotated format, the problems of ordering, structuring and animating sequences of images, and producing natural language descriptions that correspond to those i...

Museum visitors can continue interacting with museum exhibits even after they have left the museum. We can help them do this by creating a report that includes a basic, personalized narration of their visit, the items and relationships they found most interesting, pointers to additional related online information, and suggestions for future visits...

The M-PIRO project targets the concept of personalized information objects ─ that is, entities capable of responding to requests for information by taking into account what the requester already knows, what they are most interested in, and how the related information is to be made available. M- PIRO's technology allows textual and spoken descriptio...

Recent empirical experiments on surface realizers have shown that grammars for generation can be effectively evaluated using large corpora. Evalu-ation metrics are usually reported as single aver-ages across all possible types of errors and syntac-tic forms. But the causes of these errors are diverse, and the extent to which the accuracy of generat...

Recent evaluation techniques applied to corpus-based systems have been introduced that can predict quantitatively how well surface realizers will generate unseen sentences in isolation. We introduce a similar method for determining the coverage on the Fuf/Surge symbolic surface realizer, report that its coverage and accuracy on the Penn TreeBank is...

Pipelined Natural Language Generation (NLG) systems have grown increasingly complex as architectural modules were added to support language functionalities such as referring expressions, lexical choice, and revision. This has given rise to discussions about the relative placement of these new modules in the overall architecture.

After many successes, statistical approaches that have been popular in the parsing community are now making headway into Natural Language Gen- eration (NLG). These systems are aimed mainly at surface realization, and promise the same advan- tages that make statistics valuable for parsing: ro- bustness, wide coverage and domain independence. A recen...

Pipelined Natural Language Generation (NLG) systems have grown increasingly complex as architectural modules were added to support language functionalities such as referring expressions, lexical choice, and revision. This has given rise to discussions about the relative placement of these new modules in the overall architecture. Recent work on anot...

Natural Language Generation has made great strides towards multilingual gen-eration from large-scale knowledge sources. Meanwhile, current research in revision has vastly improved the qual-ity of text that NLG systems produce. However, to-date there has been no at-tempt at combining revision and mul-tilingual NLG. This paper presents re-search in m...

Previous approaches to pronominalization have largely been theoretical rather than applied in nature. Frequently, such methods are based on Centering Theory, which deals with the resolution of anaphoric pronouns. But it is not clear that complex theoretical mechanisms, while having satisfying explanatory power, are necessary for the actual generati...

Natural language generation systems to date have concen- trated on the tasks of explanation generation, tutorial dialogue, auto- mated software documentation, and similar technical tasks. A largely unexplored area is narrative prose generation, or the production of texts used in stories such as novels, mysteries, and fairy tales. We present a featu...

Narrative generation has historically suffered from poor writing quality, stemming from a narrow focus on story grammars and plot design. Moreover, to-date natural language generation systems have not been capable of faithfully reproducing either the variety or complexity of naturally occurring narratives. In this article we first propose a model o...

We are witnessing the emergence of a new technology for dynamically creating stories tailored to the interests of particular readers. Narrative prose generators offer much promise for literacy education, but designing them for maximal effectiveness requires us to understand their effect on readers. This article describes the evaluation of STORYBOOK...

Document drafting is a key component of legal expertise. Effective legal document drafting requires knowledge both of legal domain knowledge and of the structure of legal discourse. Automating the task of legal document drafting therefore requires explicit representation of both these types of knowledge. This paper proposes an architecture that int...

The capacity for self-explanation can make computer-drafted documents more credible, assist in the retrieval and adaptation of archival documents, and permit comparison of documents at a deep level. We propose a knowledge-based model of documents that makes explicit the underlying goals that documents are intended to achieve and the stylistic conve...

Document drafting is a central judicial problem-solving activity. Development of automated systems to assist judicial document drafting has been impeded by the absence of an explicit model of (1) the connection between the document drafter's goals and the text intended to achieve those goals, and (2) the rhetorical constraints expressing the stylis...

This book describes research in all aspects of the design, implementation, and evaluation of embodied conversational agents as well as details of specific working systems. Embodied conversational agents are computer-generated cartoonlike characters that demonstrate many of the same properties as humans in face-to-face conversation, including the ab...

Intelligent multimedia systems hold great promise for knowledge-based learning environments. Because of recent advances in our understanding of how to dynamically generate multimodal explanations and the rapid growth in the performance of 3D graphics technologies, it is becoming feasible to create multimodal explanation generators that operate in r...

Because narrative plays such a central role in cognition and culture, narrative-centered curricula have been the subject of increasing attention. By taking advantage of the inherent structure of narrative, narrative-centered learning environments could provide engaging worlds in which students are actively involved in motivating storybuilding activ...

Document drafting is a key component of legal expertise. Effective legal document drafting requires knowledge both of legal domain knowledge and of the structure of legal discourse. Automating the task of legal document drafting therefore requires explicit representation of both these types of knowledge. This paper proposes an architecture that int...

Intelligent multimedia systems hold great promise for knowledge-based learning environments. Be- cause of recent advances in our understanding of how to dynamically generate multimodal explana- tions and the rapid growth in the performance of D graphics technologies, it is becoming feasible to create multimodal explanation generators that operate i...

It is crucial for knowledge-based learning environments to be able to provide students with dynamic, realtime explanations. Achieving this requires the integration of 3D animated explanations with speech. Unfortunately, planning the integrated creation of 3D animation and the types of linguistic utterances capable of reactively and dynamically expl...

Life-like animated interface agents for knowledge-based learning environments can provide timely, customized advice to support students' problem solving. Because of their strong visual presence, they hold significant promise for substantially increasing students' enjoyment of their learning experiences. A key problem posed by life-like agents that...

Lifelike animated agents for knowledge-based learning environments can provide timely, customized advice to support students' problem solving. Because of their strong visual presence, they hold significant promise for substantially increasing students' enjoyment of their learning experiences. A key problem posed by lifelike agents that inhabit arti...

Dynamically providing students with clear explanations of complex spatial concepts is critical for a broad range of knowledge-based educational and training systems. This calls for a realtime solution that can dynamically create 3D animated explanations that artfully integrate well-chosen speech with rich visualizations. Unfortunately, planning the...

Knowledge-based learning environments provide an ideal testbed for developing and evaluating computational models of mixed initiative interaction. In each problem-solving episode, learners incrementally develop solutions for problems posed by the environment. To maximize learning effectiveness and learning efficiency, we have been developing animat...

Document drafting is a central judicial problem-solving activity. Development of automated systems to assist judicial document drafting has been impeded by the absence of an explicit model of (1) the connection between the document drafter's goals and the text intended to achieve those goals, and (2) the rhetorical constraints expressing the stylis...

Interactive 3D worlds offer an intriguing testbed for the natural language generation community. To complement interactive 3D worlds' rich visualizations, they require significant linguistic flexibility and communicative power. We explore the major functionalities and architectural implications of natural language generation for three key classes o...

Life-like animated interface agents for knowledgebased learning environments can provide timely, customized advice to support students' problem solving. Because of their strong visual presence, they hold significant promise for substantially increasing students' enjoyment of their learning experiences. A key problem posed by life-like agents that i...

Recent years have witnessed rapid progress in explanation generation. Despite these ad-vances, the quality of prose produced by ex-planation generators warrants signi cant im-provement. Revision-based explanation gener-ation o ers a promising means for improving explanations at runtime. In contrast to single-draft explanation generation architectur...

In recent years, natural language generation research has begun to mature rapidly. It has witnessed both the appearance of sophisticated off-the-shelf surface realizations systems and the development in the knowledge representation community of knowledge bases in a variety of domains. However, knowledge bases typically employ a logic formalism whil...

In recent years, the natural language generation community has begun to mature rapidly and produce sophisticated off-the-shelf surface realizers. A parallel development in the knowledge representation community has been the emergence of large-scale knowledge bases that house tens of thousands of facts encoded in expressive representational language...

We describe a tutorial dialogue system under development that as-sists students in learning how to differentiate equations. The system uses deep natural language understanding and generation to both in-terpret students' utterances and automatically generate a response that is both mathematically correct and adapted pedagogically and linguistically...

Many modalities have been utilized for the presentation of route directions to users with mobile devices, such as 3D graphics, 2D maps, 2D diagrams, as well as written and spoken text. Most such direction giving systems either use localization technologies and can thus be interactive, or are non localized and as a consequence non-interactive. We de...

We present an analysis of student lan- guage input in a corpus of tutoring dia- logue in the domain of symbolic differ- entiation. Our focus on procedural tutor- ing makes the dialogue comparable to col- laborative problem-solving (CPS). Exist- ing CPS models describe the process of negotiating plans and goals, which also fits procedural tutoring....

User modeling data in dynamic, personal and adaptable sys- tems is usually collected immediately before system interaction with a questionnaire, or during application execution when users' choices are recorded and analyzed. This data is then typically used to intelligently adapt the system's output, hopefully improving the user's interaction in som...

We describe two tutorial dialogue systems that adapt techniques from task-oriented dialogue systems to tutorial dialogue. Both systems employ the same reusable deep natural language understanding and generation components to interpret students' written utterances and to automatically generate adaptive tutorial responses, with separate domain reason...

Tools and data that can be shared in the Natural Language Generation community require common standards for data col- lection, documentation, implementation and licensing as well as a central place to find such resources. We argue for open, free, well-documented and simply- structured resources, and introduce a free and open online repository of NL...