David Kirsh

• D.Phil, Oxford
• Professor at University of California, San Diego

Transparency seems to represent a solution to many ethic issues generated by systems that collect implicit data from users to model the user themselves based on programmed criteria. However, making such systems transparent – besides being a major technical challenge - risks raising more issues than it solves, actually reducing the user’s ability to...

Study objective The use of a double check by 2 nurses has been advocated as a key error-prevention strategy. This study aims to determine how often a double check is used for high-alert medications and whether it increases error detection. Methods Emergency department and ICU nurses worked in pairs to care for a simulated patient. Nurses were rand...

Symbiotic systems are systems that gather personal data implicitly provided by the user, derive a profile/model of the user from such data and adjust their output/service according to their notion of what would be desirable to the user thus modeled. Because of these three characteristics, symbiotic systems represent a step forward towards facilitat...

Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation—external representations save internal memory and computation —is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structure...

To study how designers explore ideas when making physical models we ran an experiment in which architects and undergraduate students constructed a dream house made of blocks. We coded their interactions in terms of robotic pick and place actions: adding, subtracting, modifying and relocating blocks. Architects differed from students along three dim...

We undertook a detailed ethnographic study of the dance creation process of a noted choreographer and his distinguished troupe. All choreographer dancer interactions were video'ed, the choreographer and dancers were interviewed extensively each day, as well as other observations and tests performed. The choreographer used three main methods to prod...

Individual creativity is standardly treated as an ‘internalist’ process occurring solely in the head. An alternative, more interactionist view is presented here, where working with objects, media and other external things is seen as a fundamental component of creative thought. The value of chance interaction and chance cueing — practices widely use...

This workshop explores and identifies the cognitive issues fundamental to the design of gestural interactive systems. To achieve this, a dialogue will be facilitated among researchers in the cognitive science of gesture and gestural interaction within the HCI community. During the workshop we will discuss the different methodologies and results wit...

Video surveillance systems are of a great value to prevent threats and identify/investigate criminal activities. Manual analysis of a huge amount of video data from several cameras over a long period of time often becomes impracticable. The use of automatic detection methods can be challenging when the video contains many objects with complex motio...

The main section is devoted to the domain of new cognitive science: embodied cognition, enactivism, external representations, as well as ecological psychology in the context of robotics.

Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation- external representations save internal memory and com- putation-is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structur...

The theory of embodied cognition can provide HCI practitioners and theorists with new ideas about interac-tion and new principles for better designs. I support this claim with four ideas about cognition: (1) interacting with tools changes the way we think and perceive – tools, when manipulated, are soon absorbed into the body schema, and this absor...

To explore the question of physical thinking – using the body as an instrument of cognition – we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. A striking case of physical thinking is found in the phenomenon of marking. Mar-king refers to dancing a phrase in a...

We designed an experiment to explore the learning effectiveness of three different ways of practicing dance movements. To our surprise we found that partial modeling, called marking in the dance world, is a better method than practicing the complete phrase, called practicing full-out; and both marking and full-out are better methods than practicing...

Using the perspective of situated cognition we studied how people interact with a physical map to help them navigate through an unfamiliar environment. The study used a mixture of cognitive ethnography and traditional experimental methods. We found that the difference between high and low performing navigators showed up in the speed they completed...

Video data from three large captures of choreographic dance making was analyzed to determine if there is a difference between participant knowledge – the knowledge an agent acquires by being the cause of an action – and observer knowledge – the knowledge an observer acquires through close attention to someone else's performance. The idea that there...

To explore the question of physical thinking – using the body as an instrument of cognition – we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. A striking case of physical thinking is found in the phenomenon of marking. Mar-king refers to dancing a phrase in a...

To explore the question of physical thinking -- using the body as an instrument of cognition -- we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. We report here on two phenomena: 'marking' and 'riffing'. Marking refers to dancing a phrase in a less than complet...

Introduction : The use of wireless, electronic, medical records and communications in the prehospital and disaster field is increasing. Objective: This study examines the role of wireless, electronic, medical records and communications technologies on the quality of patient documentation by emergency field responders during a mass-casualty exercise...

There is growing interest in the use of technology to enhance the tracking and quality of clinical information available for patients in disaster settings. This paper describes the design and evaluation of the Wireless Internet Information System for Medical Response in Disasters (WIISARD). WIISARD combined advanced networking technology with elect...

I mean it is in between, it is somewhere between doing it and thinking it, itʼs just like thinking it with the body (professional dancer in interview about marking, 2009) In dance, there is a practice called 'marking'. When dancers mark, they execute a dance phrase in a simplified, schematic or abstracted form. Based on our interviews with professi...

A videographic study of origami is presented in which subjects were observed making four different origami objects under five modes of instruction: photos + captions, illustrations-only, illustrations with small captions, illustrations with large captions, and text-only as control. The objective of the study was to explore the gestures and other ac...

Contemporary choreography offers a window onto creative processes that rely on harnessing the power of sensory systems. Dancers use their body as a thing to think with and their sensory systems as engines to simulate ideas nonpropositionally. We report here on an initial analysis of data collected in a lengthy ethnographic study of the making of a...

Can virtual engagement enable the sort of interactive coupling with objects enjoyed by archaeologists who are physically present at a site? To explore this question I consider three points: 1) Tangible interaction: What role does encounter by muscle and sinew play in experiencing and understanding objects? 2) Thinking with things. What sorts of int...

We define, classify and discover new functions, aspects and types of marking as a body skill. Through a cognitive ethnography of a top class dance company we collected extensive video data on the complete rehearsal process as well as concise speech extracted from multiple interviews made to the dancers and the choreographer. We provide a detailed q...

Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation—external representations save internal memory and computation—is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structure o...

To explore the question of physical thinking – using the body as an instrument of cognition – we collected extensive video and interview data on the creative process of a noted choreographer and his company as they made a new dance. A striking case of physical thinking is found in the phenomenon of marking. Marking refers to dancing a phrase in a l...

When people make sense of situations, illustrations, instructions and problems they do more than just think with their heads. They gesture, talk, point, annotate, make notes and so on. What extra do they get from interacting with their environment in this way? To study this fundamental problem, I looked at how people project structure onto geometri...

Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation – external representations save internal memory and computation – is only part of the story. I discuss eight ways external representations enhance cognitive power: they provide a structure tha...

In the course of daily life we solve problems often enough that there is a special term to characterize the activity and the right to expect a scientific theory to explain its dynamics. The classical view in psychology is that to solve a problem a subject must frame it by creating an internal representation of the problem’s structure, usually calle...

Much of a culture’s history – its knowledge, capacity, style, and mode of material engagement – is encoded and transmitted in its artifacts. Artifacts crystallize practice; they are a type of meme reservoir that people interpret though interaction. So, in a sense, artifacts transmit cognition; they help to transmit practice across generations, shap...

The degree to which information is encoded explicitly in a representation is related to the computational cost of recovering or using the information. Knowledge that is implicit in a system need not be represented at all, even implicitly, if the cost of recovering it is prohibitive.

In audio-visual telecommunication, low video frame rates represent a popular method for saving on bandwidth requirements. When key frames displayed the extremes of lip movements we found that participants performed comparably to standard displays at 30 frames per second. Experiments were conducted to compare the effectiveness of a small number of a...

The Wireless Internet Information System for Medical Response in Disasters (WIISARD) explores the use of scalable wireless networks to facilitate medical care at the site of a disaster. The focus of the project is care of victims of industrial accidents or terrorist attacks with traumatic injuries complicated by chemical, biological or radiological...

Humans are closely coupled with their environments. They rely on being `embedded' to help coordinate the use of their internal cognitive resources with external tools and resources. Consequently, everyday cognition, even cognition in the absence of others, may be viewed as partially distributed. As cognitive scientists our job is to discover and ex...

We study how Metropolitan Medical Response System units conceptualize the physical space of a disaster and their organized response. Using a variety of ethnographic methods before, during, and after a disaster drill, we have developed an initial ontology for geospatial and context-aware technology. The conceptual map of first responders is far more...

Experiments were conducted to investigate the interdependency of frame rates (30, 15, 10 fps) and audio-visual skew (from +163 to -233 ms1). Noised nonsense words like 'abagava' were presented to 20 participants who were asked to identify the middle consonant. At low frame rates (10 fps) consonant perception was impaired when audio ran ahead of vid...

I argue that it is not possible to accurately represent our task settings as closed environments with a single well defined cost structure. Natural environments are places where many things are done, often at the same time, and often by many people. To appreciate the way such invariants of everyday life affect design I present a case study, a micro...

We study how Metropolitan Medical Response System units conceptualize the physical space of a disaster and their organized response. Using a variety of ethnographic methods before, during, and after a disaster drill, we have developed an initial ontology for geospatial and context-aware technology. The conceptual map of first responders is far more...

Metacognition is associated with planning, monitoring, evaluating and repairing performance Designers of elearning systems can improve the quality of their environments by explicitly structuring the visual and interactive display of learning contexts to facilitate metacognition. Typically page layout, navigational appearance, visual and interactivi...

The influence of imagery on perception depends on the content of the mental image. Sixty-three students responded to the location of the 2 hands of a clock while visualizing the correct or an incorrect clock. Reaction time was shorter with valid cueing. Could this have resulted from visual acquisition strategies such as planning visual saccades or...

Overall Objectives 1. Develop guidelines * how best to annotate videos and video annotate stills so as to improve shared understanding * simulate realistic planning contexts in order to determine how planners and analysts should annotate stills and videos to make better decisions 2. Experimentally discover media factors affecting: * Shared Understa...

The goal of three-dimensional visualization is to present information in such a way that the viewer suspends disbelief and uses the screen imagery the same way as he or she would use an identical, real 3D scene. To do this effectively, programmers employ a variety of 3D depth cues. Our own anecdotal experience says that shadows and stereopsis are t...

A small collection of illustrations is provided to show some of the diverse ways illustration may aid understanding. The display of parts and assemblies often relies on techniques such as explosions and canonical views to communicate the global structure and relations of a system that may have hidden pieces. Book illustrations exemplify specific vi...

The question of how to conceive and represent the context of work is explored from the theoretical perspective of distributed cognition. It is argued that to understand the office work context we need to go beyond tracking superficial physical attributes such as who or what is where and when and consider the state of digital resources, people’s con...

Finding one's way to sites of interest on the Web can be problematic, and this difficulty has been recently exacerbated by widespread development of 3-D Web content and virtual-world browser technology using the Virtual Reality Modeling Language (VRML). Whereas travelers can often navigate 2-D Web sites based on textual and 2-D thumbnail image repr...

To discover how to couple reflection with reaction we have been studying how people play the computer game Tetris. Our basic intuition is that the job of the reasoner is to monitor the environment and the agent's behavior over time to discover trends or deviations from the agent's normative policy, and tune the priorities of the attentional system...

Architecture is about to enter its first magical phase: a time when buildings actively co-operate with their inhabitants; when objects know what they are, where they are, what is near them; when social and physical space lose their type coupling; when wall and partitions change with mood and task. As engineers and scientists explore how to digitse...

We are quickly passing through the historical moment when people work in front of a single computer, dominated by a small CRT and focused on tasks involving only local information. Networked computers are becoming ubiquitous and are playing increasingly significant roles in our lives and in the basic infrastructures of science, business, and social...

We present data and argument to show that in Tetris---a real-time interactive video game---certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understoo...

em if they are ignored. J.J. Gibson was the first psychologist to emphasize this mutuality [Gibson, 1950, Gibson, 1966]. His central concern was how the constant activity of an organism created a dynamic stream of information that was richer in action-relevant data than static images. By looking for regularities between actions and the first or sec...

Réflexions sur la surcharge cognitive. Cet article aborde le problème de la saturation cognitive tel que les individus le vivent au quotidien sur leur lieu de travail. Il examine d'abord une série d'hypothèses sur les causes de ce phénomène : trop d'information en "push" et en "pull", le multi-tâche et les interruptions, un environnement de travail...

An experiment was performed to test the hypothesis that people sometimes take physical actions to make themselves more effective problem solvers. The task was to generate all possible words that could be formed from seven Scrabble letters.

In the Coqos project task performance measures and a corresponding framework are suggested and pursued as a novel and more suitable means for determining utility curves. TPM are intended to avoid limits inherent in traditional measures like mean opinion scores. MOS rely merely on subjective ratings rather than on more objective performance in relat...

The type of principles which cognitive engineers need to design better work environments are principles which explain interactivity and distributed cognition: how human agents interact with themselves and others, their work spaces, and the resources and constraints that populate those spaces. A first step in developing these principles is to clarif...

Mechanisms for QoS provisioning in communication networks range from flow-based resource reservation schemes, based on reservation aggregation techniques to adaptation of applications, compensating for incomplete reservation. Scalable, aggregation-based reservations can also be combined with adaptations for a more flexible and robust overall QoS pr...

This paper introduces the concept of Adaptive Rooms, which are virtual environments able to dynamically adapt to users’ needs, including ‘physical’ and cognitive workflow requirements, number of users, differing cognitive abilities and skills. Adaptive rooms are collections of virtual objects, many of them self-transforming objects, housed in an ar...

Dramatic advances in 3D Web technologies have recently led to widespread development of virtual world Web browsers and 3D content. A natural question is whether 3D thumbnails can be used to find one's way about such 3D content the way that text and 2D thumbnail images are used to navigate 2D Web content. We have conducted an empirical experiment th...

Virtual environment landmarks are essential in wayfinding: they anchor routes through a region and provide memorable destinations to return to later. Current virtual environment browsers provide user interface menus that characterize available travel destinations via landmark textual descriptions or thumbnail images. Such characterizations lack the...

Multimedia technology offers instructional designers an unprecedented opportunity to create richly interactive learning environments. With greater design freedom comes complexity. The standard answer to the problems of too much choice, disorientation, and complex navigation is thought to lie in the way we design the interactivity in a system. Unfor...

On most accounts of expertise, as agents increase their skill, they are assumed to make fewer mistakes and to take fewer redundant or backtracking actions. Contrary to such accounts, in this paper we present data collected from people learning to play the videogame Tetris which show that as skill increases,the proportion of game actions that are la...

This paper introduces the concept of Adaptive Rooms, which are virtual environments able to dynamically adapt to users’ needs, including ‘physical’ and cognitive workflow requirements, number of users, differing cognitive abilities and skills. Adaptive rooms are collections of virtual objects, many of them self-transforming objects, housed in an ar...

The objective of this essay is to provide the beginning of a principled classification of some of the ways space is intelligently used. Studies of planning have typically focused on the temporal ordering of action, leaving as unaddressed, questions of where to lay down instruments, ingredients, work-in-progress, and the like. But, in having a body,...

The objective of this essay is to provide the beginning of a principled classification of some of the ways space is intelligently used. Studies of planning have typically focused on the temporal ordering of action, leaving as unaddressed, questions of where to lay down instruments, ingredients, work-in-progress, and the like. But, in having a body,...

Keywords Introduction

We present data and argument to show that in Tetris—a real‐time, interactive video game—certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world than by performing computational actions in the head alone. We have found that some of the translations and rotations made by players of th...

Computation is a process of making explicit, information that was implicit. In computing 5 as the solution to ∛125, for example, we move from a description that is not explicitly about 5 to one that is. We are drawing out numerical consequences to the description ∛125. We are extracting information implicit in the problem statement. Can we precisel...

It is sometimes argued that if PDP networks can be trained to make correct judgements of grammaticality we have an existence proof that there is enough information in the stimulus to permit learning grammar by inductive means alone. This seems inconsistent superficially with Gold's theorem and at a deeper level with the fact that networks are desig...

Theories of intelligence can be of use to neuroscientists if they: 1. Provide illuminating suggestions about the functional architecture of neural systems; 2. Suggest specific models of processing that neural circuits might implement. The objective of our session was to stand back and consider the prospects for this interdisciplinary exchange.

We present data and argument to show that in Tetris---a real-time interactive video game---certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understoo...

A startling amount of intelligent activity can be controlled without reasoning or thought. By tuning the perceptual system to task relevant properties a creature can cope with relatively sophisticated environments without concepts. There is a limit, however, to how far a creature without concepts can go. Rod Brooks, like many ecologically oriented...

The objective of research in the foundations of AI is to explore such basic questions as: What is a theory in AI? What are the most abstract assumptions underlying the competing visions of intelligence? What are the basic arguments for and against each assumption? In this essay I discuss five foundational issues: (1) Core AI is the study of concept...

In this essay I shall consider a certain methodological claim gaining currency in connectionist quarters: The claim that variables are costly to implement in PDP systems and hence are not likely to be as important in cognitive processing as orthodox theories of cognition assume.

BLDSC reference no.: D53831/85. Thesis (D. Phil.)--University of Oxford, 1983. Includes bibliographical references.

In order to evaluate the impact WIISARD technology has on disaster response we have defined a set of quantitative measures which can be applied at drills and disasters. The measures were found through ethnographic study of responders, especially team leaders and incident commanders, and we tested their viability in simulation models. Our site metri...