Jonathan Su’s research while affiliated with University of Washington and other places

In this paper, we present a study of how instructors draw diagrams in the process of delivering lectures. We are motivated by wanting to understand challenges and opportunities for automatically analyzing diagrams, and to use this to improve tools to support the delivery of presentations and the viewing of archived lectures. The study was conducted by analyzing a large group of examples of diagrams collected from real lectures that were delivered from a Tablet PC. The main result of the paper is the identification of three specific challenges in analyzing spontaneous instructor diagrams: separating the diagram from its annotations and other surrounding ink, identifying phases in discussion of a diagram, and constructing the active context in a diagram.

In this paper, we report on an empirical exploration of digital ink and speech usage in lecture presentation. We studied the video archives of five Master's level Computer Science courses to understand how instructors use ink and speech together while lecturing, and to evaluate techniques for analyzing digital ink. Our interest in understanding how ink and speech are used together is to inform the development of future tools for supporting classroom presentation, distance education, and viewing of archived lectures. We want to make it easier to interact with electronic materials and to extract information from them. We want to provide an empirical basis for addressing challenging problems such as automatically generating full text transcripts of lectures, matching speaker audio with slide content, and recognizing the meaning of the instructor's ink. Our results include an evaluation of handwritten word recognition in the lecture domain, an approach for associating attentional marks with content, an analysis of linkage between speech and ink, and an application of recognition techniques to infer speaker actions.

There has been much research on the benefits of active and collaborative learning and on its use in computer science courses. As classroom technology becomes more prevalent it is natural to develop systems that support the use of these techniques. We have developed such a system as an extension to Classroom Presenter [2], a Tablet PC-based presentation system. In our system students (or groups of students) are equipped with tablet computers and at various points during the lecture, are asked to solve a problem or respond to a question. Students respond by writing their solution on the tablet and submitting it wirelessly to the instructor. The instructor can view all student responses, select one or more to display to the class, and annotate responses with ink as they are being displayed. Student responses can be saved for review after lecture by the instructor or made available electronically to the students. In this paper we describe the system and our initial experiences using the system in two classes (CS1 and Computer Ethics) at the University of San Diego in Fall 2003.

We are interested in understanding how digital ink and speech are used together in presentation. Our long range goal is to develop tools to analyze the ink and speech channels of recorded lectures. As a first step in this process, we are making a detailed study of instructors' digital ink usage in real university lectures. This work is being done in the context of a Tablet-PC based pre- sentation system we have developed, but is applicable to other systems which record digital ink and speech. In this paper we concentrate on how instructors draw and use diagrams in the process of lecture delivery and identify phenomena which are important when automat- ically processing the diagrammatic ink.