Milgram’s Reality-Virtuality Continuum (Milgram and Kishino, 1994) 

Contexts in source publication

Context 1

... Reality Agent recorded in scientific literature and was years ahead of similar efforts. The Invisible Person (Psik et al., 2003) is based on the ALIVE system and employs a humanoid virtual character in an effort to engage visitors at the Vienna Museum of Technology in a game of Tic Tac Toe. The game board is digitally added onto the floor, and both user and character control the game via body postures and hand gestures. The character’s internal state is based on an emotional system that directs its actions, facial expressions and manner of interaction with the user. Feedback from visitors of the exhibition commend the lifelikeness of the agent. Storytelling engines also explicitly model individual agents’ goals and motivations in order to create dynamic narratives from the interplay of these goals and the users actions. One of the first projects to apply digital narrative to Mixed Reality is the cultural heritage application GEIST (Kretschmer et al., 2001). GEIST 1 immerses the user in a thrilling adventure involving events from the Thirty Years’ War. As the user roams the old town of Heidelberg, he can enter certain ‘hotspots’ in which ghosts from the past appear in the form of virtual characters. They plead for the user’s help in solving the mystery surrounding their death, creating a quest around the city in which the user has to learn about the history of places and events in order to succeed. In the physical domain, GEIST agents are limited to sensing the user’s position and orientation, while in the virtual, which is populated with spatially aligned models of buildings and other virtual objects, interaction is much more varied and versatile. However, due to the spirit nature of the GEIST agents, corporeal presence of the ghosts is inhibited, as they appear translucent and float in midair. Another prominent example of MR storytelling is the Mixed-Reality Interactive Storytelling (MRIS) project (Charles et al., 2004), which allows a user to immerse himself into a spy thriller story in the role of the villain. It does so by capturing the user’s image in real time, extracting it from the background, and inserting it into a virtual world populated by autonomous synthetic actors with which the user then interacts using natural language and gestures. The resulting image is projected onto a large screen facing the user, who sees his own image embedded in the virtual stage alongside the synthetic actors. Notably, when viewed from the perspective of Milgram’s continuum of MR displays (see Figure 2), MRIS is a rare example exemplifying the concept of Augmented Virtuality, i.e. a virtual world with added ‘real’ components. Finally, Virtual Gunslinger (Hartholt et al., 2009), is a more recent example of a Mixed Reality storytelling experience. In it, the user plays the character of a cowboy in a Wild West saloon who gets challenged to a duel. The user is placed in an environment featuring a real bar counter and a virtual bartender and outlaw, both of which are projected onto screens placed in the room. The user can interact with the agents using natural language dialogues and gestures, e.g. moving his arm as if to pull a gun when duelling with the outlaw. Common to all these strong agents is that the agent architecture facilitates the development of agents that exhibit realistic and lifelike behaviour. But strong agent system are also often used to realise highly complex and distributed systems that deal with ...

Context 2

... (1998, 1999) and Duffy (2000), who claim that the social environment includes the agent’s interactions with other agents or human users. Within the context of this research, embodiment is seen as the strong provision of environmental context (structural coupling) with a social element included. An agent is embodied if it is situated in a particular environment, has a body , and senses and interacts with that environment, and any other individuals located therein. This definition of embodiment coincides with that of Dourish (2001), who emphasised the importance of an embodied approach to Human-Computer Interaction, in light of new developments in Ubiquitous and Social Computing and proposed a number of design guidelines for the development of Embodied Interaction. Early Artificial Intelligence (AI) research focused upon reasoning based upon search of abstract symbol structures (Newell and Simon, 1976). But this unembodied approach, sometimes referred to as ‘Good Old Fashioned AI’ (Haugeland, 1985), had a number of flaws. As noted by Steels (2000) and Dautenhahn (1999), humans have a tendency to ‘animate’ the world and are unlikely to see an unembodied agent as intelligent. Therefore embodiment has, in recent years, come to be seen as an important requirement in the development of an intelligent system (Duffy et al., 2005). The move away from the unembodied approach was triggered by a series of papers by Brooks (1991a,b), who emphasised the situatedness and embodiment of an agent. Brooks’ popularisation of the reactive approach served as a catalyst for the creation of a more embodied approach to AI, where an agent must be structurally coupled with its environment if it is to be seen as intelligent. While robot agents are embodied in a physical form, using sensors and actuators to perceive and act upon the physical world, virtual agents can also be considered embodied in their simulated environment, at least to the extent to which the simulation manages to create a structural coupling between the agent and the simulated environment. Both strands are motivated by the desire to create agents that are capable of behaving and interacting in an intelligent manner with other agents. Crucially, both robotic and virtual agents can be considered synthetic characters, although differently embodied, i.e. in physical or digital form. Key to the definition of Mixed Reality Agents, as outlined within this paper, is the idea of a Mixed Reality environment. Milgram and Kishino (1994) define MR in terms of their Reality-Virtuality Continuum (Figure 2) whereby Mixed Reality is the space between a purely physical (or ‘real’, as they describe it) environment and a purely virtual environment. Each MR environment can be seen, to a greater or ...