ChapterPDF Available

The Role of Haptics in Games

Authors:
  • Leapshot Labs
11
The Role of Haptics in Games
Mauricio Orozco, Juan Silva, Abdulmotaleb El Saddik and Emil Petriu
University of Ottawa,
Canada
1. Introduction
Today, playing a video game is a different story than what it was twenty years ago in many
senses. Game consoles have steadily gained popularity not only among kids but also among
other age groups. For example, today, the Xbox 360 from Microsoft comes with Kinect
technology which is based on webcam built in system for users to control and interact with
the console without the need for conventional game controller to bring entertainment and
playing games to people of all ages. On the other hand electronic gadgets including smart
phones have changed the way game controllers, pads, buttons, joy sticks can be used with
the incorporation of haptic and sensing technologies. Additionally, impressive graphics and
3D rendering displays are creating virtual environments more realistic which have the
potential to capture the gamer’s attention all throughout the gaming process. As stated by
Jurgelionis and others: “The increasing number of broadband users, and a demand for
quality and diversity in entertainment services drives the development of new pervasive
entertainment systems” (Jurgelionis et al., 2007). Indeed, they also stated that such
entertainment systems should be accessible without any limits on time and location.
In the real world, people receive and disseminate information in three-dimensional space.
Computers, through graphical user interfaces, allow users to perceive an imitated three-
dimensional world that exists in the real world. Such a virtual world can be enhanced in a
more complete imitation of the real space by the introduction of an artificial support
technology called haptics. A haptic interface is a device that allows a user to interact with a
computer by receiving tactile and force feedback. The interaction can embrace the entire
body or only the tip of a finger, giving the user information about the nature of objects
inside the world. The introduction of haptics permits one to enhance a vast spectrum of
human tasks in a virtual environment.
Currently, haptic research and technology has been focused on designing and evaluating
prototypes of different features and capabilities for the use in virtual environments. The
evidence is that, some of these prototypes have become commercially available to the
market. In that sense, applications of this technology have been invaded rapidly from
devices that can interact with sophisticated graphical user interfaces (GUI’s), games,
multimedia publishing, scientific discovery and visualization, arts and creation, editing
sound and images, the vehicle industry, engineering, manufacturing, Tele-robotics and Tele-
operations, education and training, as well as medical simulation and rehabilitation.
Haptics Rendering and Applications
218
For the time being we could argue that haptic research related to home entertainment and
computer games has blossomed and impacted the development of technology during the
past few years.
It is well-know that the game experience comprises four aspects: physical, mental, social,
and emotional (El Saddik, 2007). It is on the physical aspects that, force feedback technology
(haptics) enhances the game experience by creating a more realistic physical feeling of
playing a game. This physical experience can be translated for example in improving the
physical skills of the players, and imitating the use of physical artefacts. By using existing,
well-developed game engine components—specifically, a scene graph library and physics
engine – and augmenting them with the design and implementation of haptic rendering
algorithms, it is possible to create a highly useful haptic game development environment.
This can be reflected in a rich environment which provides to players or users a higher sense
of immersion as well as new and interesting ways to interact with the game environment. In
addition this simulated world can be used to do research on applications such as physical
rehabilitation, driving training simulation and more.
Currently, a diverse spectrum of games available in the market take advantage of the force
feedback stimuli effects offered by mainstream haptic interfaces. Here we intend to give a
broad view of past and present developments aimed at enabling game environment with
haptic augmented interfaces, while discussing the strengths and weaknesses of such
developments.
2. Machine haptics
In order to fully understand the potential and capabilities of haptics in the gaming
environment, let us first look how the sense of touch has brought a more complex and
appealing dimension the human computer interaction. Haptics was introduced, at the
beginning of the 20th century by research work in the field of experimental psychology
aimed at understanding human touch perception and manipulation. Psychophysical
experiments provided the contextual clues involved in the haptic perception between the
human and the machine. The areas of psychology and physiology provided a refreshed
impulse into the study of haptics and it remained popular until the late eighties. Researchers
have found that the mechanism by which we feel and perceive the tactual qualities of our
environments are considerably more complex in structure than, for example, our visual
modality. However, they opened up a wealth of opportunities in academic research and
development to answer the many questions that the haptic cognition discipline has.
Therefore the discipline that put in practice many of the theories developed in haptic
perception domain were addressed by machine haptics.
Machine Haptics involve designing, constructing, and developing mechanical devices that
replace or augment human touch. These devices, also called haptic interfaces, are put into
physical contact with the human body for the purpose of exchanging (measuring and
displaying) information with the human nervous system. In general, haptic interfaces have
two basic functions; firstly, they measure the poses (positions and/or orientations) and/or
contact forces of any part of the human body, and secondly, they display the computed
reaction touch to a haptic scene that populates touchable virtual objects with haptic
properties such as stiffness, roughness, friction, etc. Haptic interfaces can be broadly divided
The Role of Haptics in Games
219
into two categories: force feedback devices and tactile devices. Force feedback devices
display force and/or torque and enable users to feel resistive force, friction, roughness, etc.
Tactile devices present vibration, temperature, pressure, etc. on the human skin and display
textures of a virtual object or provide information such as showing direction, reading text,
displaying distance, etc.
Turning to the robotics arena in the seventies and eighties, most researchers were
considering the systems aspect of controlling remote robotic vehicles to manipulate and
perceive their environments by touch. The main objective was to create devices with
dexterity inspired by human abilities. Robotic mechanical systems with a human being in
their control loop are referred to as Tele-manipulators, where an intelligent machine is
expected to perceive the environment, reason about the perceived information, make
decisions based on this perception, and act according to a plan specified at a very high level.
In time, the robotics community found interest in topics including but not limited to:
sensory design and processing, grasp control and manipulation, object modeling and haptic
information encoding.
Meanwhile, terms such as tele-operation, tele-presence, tele-robotics and supervisory were
used interchangeably by the robotics community until the mid-nineties. From those terms
two were especially important to develop haptics systems; Tele-operation and Tele-
presence. Tele-operation refers to the extension of a person’s sensing and manipulation
capabilities to a remote location. Tele-presence can be described as a realistic way that an
operator feels while physically being at a remote site. Motivated by these concepts, the Tele-
presence and Tele-operation research communities developed several projects in the several
related fields such as the nuclear industry, sub-sea, space and the military markets.
In the early nineties, the use of the word haptics, in the context of computer haptics was
introduced. Much like computer graphics, computer haptics is concerned with the
techniques and processes of generating and displaying haptic stimuli to the user. In fact,
computer haptics uses digital display technology as a medium for physically tangible
interaction where objects can be simulated in an interactive manner. This new modality
presents information to the user’s hand and/or other parts of the body by exerting
controlled forces through the haptic interface. These forces are delivered to the user
depending on the physical properties of the objects that can be perceived. The hardware
components of this interface play an important role in displaying these forces through the
response sensors to the user. Unlike computer graphics, the behaviour of haptic interaction
is bidirectional, due to energy and information flow in both directions from the user to the
haptic interface and vice versa.
Only recently, have haptic technologies become integrated with high-end workstations for
computer-aided design (CAD) and, at the lower end, on home PCs and consoles, to augment
the human-computer interaction (HCI). Effectively this implies the opening of a new
mechanical channel between human and computer so that data can be accessed and literally
manipulated through haptic interfaces.
Nowadays, computer haptic systems can display objects of sophisticated complexity and
behaviour; thanks to the availability of high-performance force-controllable haptic
interfaces, affordable computational geometric modeling, collision detection and response
techniques, a good understanding of the human perceptual needs, and a dramatic increase
Haptics Rendering and Applications
220
in processing speed and memory size. With the commercial availability of haptic interfaces,
software toolkits, and haptics-enabled applications, the field of human-haptics interaction is
experiencing an exciting exponential growth.
3. A roadmap to haptic interfaces in games
Haptic research in the realm of home entertainment and computer games has blossomed
during the past few years. The game experience comprises four aspects: physical, mental,
social, and emotional. In particular, force feedback technology enhances the physical aspects
of the game experience by creating a deeper physical feeling of playing a game, improving
the physical skills of the players, and imitating the use of physical artefacts. Plenty of games
available in the market take advantage of the haptic effects offered by mainstream haptic
interfaces. Car racing games can be enhanced when players drive over a rough section of
road by receiving vibrations in their joysticks or steering wheels. However, how did we
come to realize such novel interfaces that enhance our gaming experience? In this section we
present a roadmap to past and present work, which can give a picture of how haptic
interfaces came to be what they are today.
3.1 Evolution of the videogame industry
It is know that Egyptians were playing games since around 3000 BC when an expedition in
1920s by Sir Leonard Wooley found two board games in tombs. Thus the Royal Game of Ur
gets its name from the two game boards which are associated with first dynasty of Ur
(before 2600 BC). Each of the game boards contains a set of twelve squares and a set of six
cases which were linked by a bridge of two cases (for review consult "Histoire des jeux de
société by Jean Marie Lhéte, 1994). The rules of the game are very similar to backgammon,
the goal is to introduce seven pawns and move them along to designated path and be the
first to have them also out of the game. At this point it is possible to state that games have
existed along the existence of humanity. Thus many games have remained as they were
designed but the way to play them has changed radically. The sole relationship between
human societies with games can fill entire books.
The video game industry itself could be dated from the late 19th century, more precisely in
1889 when Fusajiro Yamauchi established a company called Marufuku to distribute and
manufacture a Japanese playing card game-Hanafuda. In the 1950s Marufuku company
became Nintendo which means 'leave luck to heaven". Meanwhile in the USA, Martin
Bromley buys slot machines and creates a game rooms concept named Service
Games(SEGA) (Kent, 2001).
Later on, David Rosen opens a business in Japan and he starts shipping photo booths. Then
Rosen imports coin-operated electron mechanical games machines to launch this industry in
Japan. At the beginning of 60s, a student from Massachusetts Institute of Technology(MIT),
Steve Russell developed an interactive game computer called Spacewar.
At the time Rosen Enterprises merges with Services Games to become Sega Enterprises.
Then, in mid sixty's Ralph Baer starts interactive television games at Sander Associates
meanwhile Sega in Japan was charging $0.25 cents per play arcade games to overcome with
the high shipping costs, which then eventually become the standard price for playing arcade
The Role of Haptics in Games
221
games. In 1970 Magnavox licenses Rapl Baer's television game from Sander Associates and a
year later released as Odyssey based on Ping-Pong Baer's game. In the mid 1970’s Atari
designed a prototypical Home Pong unit and Namco starts creating video games.
In the late 70's Atari released the video game computer system known as 2600 and Mattel
introduced a line of LED-based handled video games, also Nintendo released its first home
video game in Japan. In 1979 Atari releases Asteroids, which became an all time bestselling
game and Mattel Electronics introduces the Intellivision game console. In addition Milton
Bradley released Microvision, the first handled programmable game system. In the 80's the
practice of selling home versions of arcade machines hits the business. The same year Pac-
Mac was released which is the most popular game by Namco (Kent, 2001).
In 1983 Cinematronics released Dragon's Lair, the first arcade game to feature laser-disc
technology. One year later Nintendo released the Family Computer (Famicom) in Japan and
then become in the USA the Nintendo Entertainment System(NES). In the mid 80s Tetris
was created by Alex Pajinov and also Atari released the 7800 game console. Later on NEC
released the 16 -bit/8 -bit hybrid PC-Engine game console in Japan and Sega unveils 16-bit
Meha Drive game console. Early 90s Sega ships SEGA CD peripheral for Genesis game
console. In 1993 Atari launches the 64-bit Jaguar and Id software publish Doom for PCs. One
year later Sega releases 32X, a peripheral device that increases the power of the Genesis. In
mid 90s Sony releases PlayStation in the United States and Nintendo Virtual Boy and also
unveils the 64-bit Nintendo game console in Japan. At the late 90s Sega releases Dreamcast
game console in the USA. In 2000 Sony released PlaStation 2 and Micorosft unveils plans for
XBOX. in 2001 Nintendo release GameCube and Microsoft Xbox worldwide (Kent, 2001).
Development of video games industry has also been based on a close relationship with the
well established amusement industry (Kent, 2001). To extend such criteria, Kent has stated
that pinball games had made a huge impact in the development of videogame and
computer industry in general. In his book he assumes the position that the creator of Baffle
Ball game David Gottlieb was a great influence in the development that we know today as
video game. In the first stage of development the Baffle Ball game was used with no
electricity and it was built in a countertop (worktop) cabinet and had only one moving part
the plunger. Players could enjoy the game by spending a penny1 and get seven balls to try
into scoring holes. Baffle ball was successful at the time that Gottlien created a company to
deal with a shipping load of 400 cabinets per day.
Later on, competition was in place, thus this industry in the 70s had five game
manufactured companies to place and distribute equipment in restaurants, bar, bowling and
stores. On the other hand the development of computers was growing fast from the 60s
when the computers occupied the whole room to the introduction of transistor which
replaced the vacuum valves and later on the silicon chips which replace the transistors
making computers more robust and fast. Thus more academic society and students were
involved in the development of computer languages and interaction. The first game was
created by engineers at Sander Associate a contractor company. The game was created from
idea of which a hard-wired logic circuit projected a spot flying across the screen. The
original idea was for players to catch the spot with manually controlled dots. Over time, the
1 A penny is a coin or a type of currency used in several English-speaking countries. It is often the
smallest denomination within a currency system source: Wikipedia 2011
Haptics Rendering and Applications
222
player's dots evolved into paddles and the game became "ping-pong”. One the main
developers of this concept was Baer who sold the idea to Magnavox but they first of
outrageously priced and poorly advertised at the time that this game had gone almost
unnoticed until Magnavox released later on as Odyssey which change dramatically the way
to play games (Kent, 2001).
3.2 Enhancing the experience with the sense of touch
In a first generation we started out with the modest beginnings of Pong, a two dimensional
simulated table tennis where players were only allowed to move the game-paddle vertically
to control a moving dot, or Pac-man which through a maze eats Pac-dots, Tetris and video
pinball, video games. Then we moved to a second generation with the Super Mario Brothers
where the player controlled Mario in a one-player control based strategy and a second
player acted as Mario's brother Luigui in a second controller approach. The game was
intended to race through the Mushroom Kingdom, survive the main antagonist Browser's
forces and save the princess Toadstool, Wolfenstein 3D, Command and Conquer, Microsoft
Flight, then to Quake the first shooter video game which was released on June 1996. The
game engine of Quake popularized many advances in the 3D game domain such as
polygonal modeling instead of pre-rendered sprites, in addition pre-rendered lightmaps and
also allowing end user to partially program the game (QuakeC). Then half-life, World of
Warcraft.
In this new era of gaming, we have moved far from the Pong game trend, to more realistic
virtual scenarios. Particularly, such perception of reality can be enhanced by the
development of computerized games influenced by technology such as haptics to make the
entertainment more sophisticated.
The first work on development of haptic devices such as joysticks was carried out at
Massachussetts Institute of Technology (MIT) and University of North Carolina at Chapel
Hill (UNC), which resulted in a 3-DOF device that simulates an object’s inertia and surface
texture (Minsky et al. 1990).
Currently, vibration feedback joysticks and steering wheels from companies like Logitech
are widely used as input devices in video games. In this sense, haptic research has
introduced new forms of complexity in the development of games by emulating the user
experience based on this particular bidirectional feedback. Pioneering attempts at
introducing modern haptics to gaming include Haptic Battle Pong (Morris et al. 2004), a
pong clone with force-feedback that, thru haptics, displays contact between a ball and a
paddle using the PHANTOM Omni device (Sensable 2010). The PHANTOM is used to
position and orient the paddle and to render the contact between the ball and the paddle as
physical forces.
Nilsson and Aamisepp (Nilsson and Aamisepp, 2003) worked on the integration of haptics
into a 3D game engine. They have investigated the possibility of adding haptic hardware
support to Crystal Space, an open source 3D game engine.
By using existing, well-developed game engine components such as Unity 3D, a scene graph
library and physics engine, and augmenting them with the design and implementation of
haptic rendering algorithms, it is possible to create a highly useful haptic game development
The Role of Haptics in Games
223
environment. This can result in a rich environment, which provides players or users with a
higher sense of immersion, as well as new and interesting ways to interact with the game
environment (Andrews et al. 2006). In addition, this simulated world can be used to do
research on applications such as physical rehabilitation, driver training simulations, and more.
There is also a haptic device called HandJive designed for interpersonal entertainment (Fogg
et al.1998). The concept is described as a handheld object that fits in one hand and allows
remote play through haptic input and output. It communicates wirelessly with similar
devices, and provides haptic stimuli. In fact, haptic devices are becoming more accessible to
the average computer and console user and will play an important role in providing
innovative forms of entertainment. As further evidence, in 2008, Novint Technologies
introduced the Novint Falcon device, which is affordable, even for mainstream consumers
(Novint 2010). This device is now integrated with several popular video games.
4. Haptic controllers
4.1 Force feedback wheels
Although drivers obtain a substantial amount of information from driving from vision,
information from other sensory modalities may also provide relevant information about the
state of the car or even the surrounding environment (Liu & Chang, 1995). Force feedback
steering racing wheels provides mainly realistic resistance to the wheel rotation and
providing precise throttle response. This is achieved by emulating the real steering system
through a mechanism that also provides vibration feedback. From a tele-operatiing system
perspective, the energy which flows between the driver and the vehicle front wheel through
the mechanical linkage can be considered to be mainly effort and flux exchange which
corresponds to force and velocity (Mohellebi et al, 2001).
In addition driving-simulators fidelity is usually defined by the quality of its visual and
motion cueing system (Katzourakis et al. 2011). Katzourakis experimented that the quality of
driving-simulator's haptic cues are very important and rely on the hardware and control
properties of the system design. Despite the hardware characteristics there is also important to
consider the model of vehicle dynamics. An experimental work was carried out by Toffin et al
in order to answer questions such as how do drivers use the force appearing on the steering
wheel when driving in curves?. Well, they have found that a wide range of adaptation results
that occurs mainly at the haptic level rather than in the internal model of the vehicle dynamics.
Many features provided by commercial force feedback steering wheels is based on exclusive
design and technology proprietary of IMMERSION Technologies.
In 1992 Thrustmaster Enzo Ferrari Force Feedback Racing wheels was released which did
not provide a satisfactory sense of precise force feedback for some users. In 1998 Logitech
the leading company of gaming controllers for PCs and consoles released the Logitech
Formula™ Force GP racing wheel with more than one million sales worldwide and actually
it was the first force feedback wheel found in any game console. The force feedback in
racing wheels attracted a more sophisticated method for racing aficionados immersed in
more realistic playing environments.
In 2002 Logitech introduces a Speed Force Feedback wheel for GameCube. In 2005 the R440
Force Feedback Wheel from Saitek has busted open the doors to affordable racing fun. In
Haptics Rendering and Applications
224
2006 more realistic racing simulators and more affordable appeared in the scene. For
instance the Logitech G25 wheel offers advanced features which, until now, could be found
only in specialized or custom-made racing simulators that sometimes cost thousands of
dollars to assemble, thus introduced the industry's first two-motor force-feedback
mechanism with the G25 wheel console, providing stronger and more precise feedback as
well as 2 1/5 rotation feature(900 degrees). With a second motor, the directional forces are
more realistic and evenly distributed throughout the wheel – drivers will feel everything
from the banks in the road to impact with walls, structures and other cars.
In 2007 things were tuned up with Forza Motorsport 2 on the Xbox 360 using the Microsoft
Wireless Force Feedback Wheel. Force feedback is an extremely useful haptic interface. It
provides real-time info on several key aspects of Forza Motorsport 2’s physics model.
Obviously, force feedback simulates the steering wheel torque created by having the front
tires on different terrain types, such as asphalt, rumble strips, or grass. It also simulates load
balance between tires as well as slippage. At the end of 2007, Saitek has launched its new
R660GT Force Feedback Wheel, a USB racing wheel with force feedback and switchable
gearstick. The R660GT wheel comes with a solid G-clamp to securely fix the wheel to a desk,
and the soft handle finish ensures a strong grip. Its setup includes pedals that can be
customized to suit the driver, so that they can operate independently of each other or can be
combined to suit different games or a player's driving style. Unlike most other driving
wheels, the R660GT pedals accurately reflect the positioning of pedals in a real car, pivoted
from the top rather than the bottom, whilst the brake offers more resistance than the
accelerator. 2009 is the era of wireless, therefore Logitech Speed Force Wireless Racing
Wheel Works With EA’s Highly Anticipated Racing Game Need for Speed™ Undercover.
4.2 Joysticks
The joystick has been the principal fight control in the cockpit of many aircraft, particularly
military fast jets. It has also been used for controlling machines such as cranes, trucks,
underwater unmanned vehicles, and wheelchairs but also very popular to control video
games. It is an input device consisting of a stick that pivots on a base and reports its angle or
direction to the device it is controlling. Its origins date since early 20th century and are
related to the aviation pioneers where its mechanical origins are uncertain but the term
“joystick” was coined to Robert Loraine.
An arcade ticks is a large-format controller for use with home game consoles or computers.
Most joysticks are two-dimensional, having two axes of movement (similar to a mouse), but
one and three-dimensional joysticks do exist. An analog joystick is a joystick which has
continuous states, i.e. returns an angle measure of the movement in any direction in the
plane or the space (usually using potentiometers) and a digital joystick gives only on/off
signals for four different directions, and mechanically possible combinations (such as up-
right, down-left, etc.). Digital joysticks were very common as game controllers for the video
game consoles, arcade machines, and home computers of the 1980s.
The force feedback joystick is similar to ordinary ones however it has in addition a couple of
electrical motors and gear train or belt system and microprocessor. The X-axis and Y-axis
shafts are connected to the stick both engage a belt pulley. The other end of the belt for each
axis engages a motor's axle. In this setup, rotating the motor axle will move the belt to pivot
the shaft, and pivoting the shaft will move the belt to rotate the motor axle. The belt's function
The Role of Haptics in Games
225
is to transmit and amplify the force from the motor to the shaft. One thing that differentiates
force feedback joysticks from traditional joysticks is the lack of a centering spring. This spring
keeps the joystick's neutral position in the middle (at 0,0 in terms of the x- and y-axis). Force
feedback joysticks do not have the traditional centering spring; instead they rely on small
servos or motors to push the joystick into the center of the range of movement
In 1977, the Atari 2600 game console was launched with a joystick that had one button to
use. In 1978 a patent was released which disclosed a joystick controller that includes a
plurality of pressure actuated switches disposed about the axis of the joystick handle. The
ZX Interface 2 was a peripheral from Sinclair Research for its ZX Spectrum home computer
released in September 1983. It had two joystick ports that were mapped to actual key presses
and the hardware came with ROM cartridge which was very limited. Released in 1987, the
Atari XE Game System (XEGS) came with a detachable keyboard, joystick, and light gun.
Essentially, the XE was a repackaged Atari 65XE, hence the compatibility with almost all
Atari 8bit software and hardware.
In 1989 a joystick handle which is telescopic like a car antenna was released. This joystick
can save space when not in use and it can save space when packed for shipment. The
joystick length is customizable to the individual user's input needs as described. Another
type is a miniature joystick incorporated into the functional area of the keyboard. This
miniature joystick has been successfully incorporated into a number of laptop computers. In
1995, Dear Ace Pilot The Phoenix Flight and Weapons Control System by Advanced Gravis
is getting lots of raves among gamers. It's fast, ergonomic and completely programmable
with 24 buttons that can map to keyboard or joystick functions.
Microsoft's latest Force Feedback 2 Joystick comes complete with feature rich enhancements
and new force feedback effects. The most practical aspect of the force feedback in this game
is that if the plane stalls in flight, the stick will shake just like in a real aircraft. You can also
feel the thud when the landing gear retracts. The force feedback for these events is rather
small - while you can feel the stick shake when the gear retracts, it isn't something that is
going to disturb your control over the joystick. It's the small details like these, the added
sensations, that will add to the gaming experience.
In 2004, Logitech is introducing its latest controller for the PlayStation 2 - the Logitech Flight
Force joystick during the 2004 International Consumer Electronics Show in Las Vegas. For
an immersive experience, the three-component Logitech Flight System G940 features a force
feedback joystick, dual throttle and rudder pedals and has more than 250 programmable
button options integrated into a fully featured HOTAS (or Hands On Throttle-and-Stick)
4.3 Game pads
It is the most common kind of game device that provides input in a computer game or
entertainment system used to control a playable character or object. Its design was conceived
to be hold in both hands with thumbs and finger used to press buttons to provide input. Thus
the main goal of game controller is to govern the movement/action of payable body/objects in
a video computer game. Gamepads usually feature a set of action buttons handled with the
right thumb and a direction controller handled with the left. Later on some common additions
to the standard pad include shoulder buttons placed along the edges of the pad.
Haptics Rendering and Applications
226
Third generation of video game controllers included the Nintendo Entertainment System
(NES) featured a brick-like design, Master System which has a similar design as NES and
the Atari 7800. The fourth generation included Genesisi Mega Drive by Sega, TurboGrafx-
16/PC-Engine and the Super Nintendo Entertainment System which had a more rounded
dog-bone like design and added two more face buttons, "X" and "Y", arranging the four in a
diamond formation.
In the Fifth Generation the Apple Bandai Pippin had a short live console designed by Apple
Computer Inc. The Atari Jaguar was the first and last Atari console to employ the modern
gamepad. Another example is the Neo Geo CD which was similar in shape and size to Sega
Genesisi/Mega Drive game pads. Other examples include, The Sega Saturn, whose control
pad introduced an analogue stick and analog triggers, Virtual Boy controller designed to use
dual joypads, which was envisioned to control objects in a 3D environment. Sony developed a
four direction D-pad, four actions were referred not by color or letter/number like most pads
instead were colored shapes; triangle, circle, cross and square. The basic design and layout was
based on that of Nintendo's SNES controller, as the PlayStation was originally developed as a
CD add-on for the SNES, before becoming a console in its own right. It was the default pad for
the first year of the PlayStation, until the release of the Dual Analog. It was often cloned for PC
gamepads. The Nintendo 64 started to have both an analog stick and a D-pad. It has the
traditional A,B,L and R buttons, along with a Z trigger button on its underside.
4.4 Haptic vests and jackets
Originally designed for medical investigation, Physician Mark Ombrellaro developed a
haptic vest that enabled the wearer to feel the impact of bullets, explosions and/or even
hand taps in the trunk of the player’s body. The 3RD Space Vest is commercialized by TN
Games and is advertised as the only gaming peripheral device that allows you to feel what
your game character feels. It works with pressured air to provide the player pressure and
impact forces that can emulate a wide scope of direction and magnitude (TN Games, 2011).
Others have worked on similar systems, and although they have not focused on the
application in games specifically, it is clear how such technologies could also be applied in a
game scenario. In the University of Ottawa in Canada, Jongeun Cha and others have
worked in a device to enhance teleconferencing. They developed a jacket that enhances
communication with the physical and emotional connection by allowing participants give
encouraging pats to one another (Cha et al, 2009). Similarly, researchers at the National
University of Singapore developed the Huggy Pajama wearable system, where the remote
communication between parent and child is enabled through virtual hugs. This is possible
by using a doll with embedded pressure sensors as input device and a haptic jacket as the
actuator for the hug at the other end. The hug is reproduced by air pockets and the
experience is further enhanced by adding a heating element to the device to mimic the
warmth of a hug (Teh et al, 2008).
5. Haptics in mobile gaming
With the advent of smart phones, and other personal and portable devices such as MP3
players and tablets, which everyday include a richer set of sensory, processing, and memory
capacities, videogames have finally make the jump from good old PCs and Consoles to the
The Role of Haptics in Games
227
mobile world. Smartphones and related devices are capable of rendering videogame
graphics and sound in high definition, which aims at immersing the player in the game
experience even with the limited size of the screen.
Because mobile video games are inherently restricted to small screens, game developers
must rely on advanced user interface design to fulfill the input/output streams of media
information necessary to play the game. Furthermore, most of the portable devices that
serve as a platform for modern mobile games feature a touch screen as the main way of
interaction with the device and its software interface. Most of the time such devices, either
completely lack or, have very few physical buttons that the player can use to play the game.
Even when physical buttons are present on the device, they are often located in places
where using them as part of the game input controller makes it awkward for the player. In
terms of haptic feedback, this is a limitation of such devices, since touch screens lack that
physicality that players have grown used to with pc and other gaming consoles and their
controllers.
To help with the lack of physical buttons and the sense of touch that they provide, mobile
phones and devices are often capable of reproducing a vibration feeling. By controlling the
moment, and duration of the vibration, to be synchronized with the touch of the screen,
such mobile devices compensate the lack of hard buttons by giving a haptic feedback to the
user whenever he/she presses a soft button on the touch screen. Although this is often good
enough for common in-device tasks such as entering text on a virtual keyboard, from a
gaming perspective the approach is very limited, as is the hardware available in most
devices. Games require a higher degree of variation on the vibration to produce more rich
and realistic experiences. Different levels of intensity, duration or even a specific vibration
pattern must be applied for different events in the game play. The challenge in providing
such degree of high definition on the haptic feedback is twofold. On one hand, hardware on
the devices must be sufficiently advanced to permit the variation of intensity at which the
internal motors will cause the vibration. And on the other the software must be sufficiently
sophisticated to control the motors in such a way that a wide array of vibration effects can
be rendered.
Research has been mostly focused in developing vibrotactile rendering techniques that can
allow for a sophisticated touch feedback experience. Different algorithms have been tried to
control the vibration motors and render the desired effects. Sang-Youn and others proposed a
Traveling Vibrotactile Wave rendering technique which makes the effect of a traveliing
sensation across the device display. It is based on controlling the vibrations of two motors,
which permits to specify the location where the overlapping of two waves occurs. This control
further allows the generation of a sensation of continuous flowing vibration by constantly
changing the overlapping point, adjusting the timing of motor actuation. They tested their
rendering technique with a ball rolling game (Sang-Youn, 2009). In another related work, they
tested a miniaturized vibrotactile rendering system based on an eccentric vibration motor and
a solenoid actuator, which generates vibrotactile information having a large bandwidth and
amplitude. Their intention was to generate event-specific vibrotactile effects for a car racing
game in a mobile device. The system could render human discernible effects for the events of
collision, driving on a bump, and driving on a hard shoulder (Sang-Youn, 2006).
A good example on the latest advances in this direction is the MOTIV platform from
Immersion. Their solution provides a TouchSense embedded controller technology to
Haptics Rendering and Applications
228
enables a high degree of control over the vibration motors of the device. On the software
side, they provide MOTIV Integrator for OEMs to include in their devices a vibration profile
manager that leverages on TouchSense, and MOTIV Software Development Kit to enable
developers to build third party applications and video games that also take advantage of the
high definition vibration effects provided by their platform (Immersion, 2011).
Another approach has been taken by Senseg, a company based in Finland whose latest
haptic rendering solution is a promise to revolutionize the way we perceive computer
generated haptic feedback on touch screens and other real world surfaces. Their E-Sense
Technology is based on biophysics; where instead of having the device’s surface vibrate,
they create a sensation of vibration directly in the finger’s skin by using an ultra-low
electrical current from an electrode to produce a small attractive force. The modulation of
that force allows the production of any number of touch sensations that can range from
vibrations, clicks, textured surfaces and more. A direct implementation of this technology is
in the mobile device industry, and the provided software APIs would allow the
development of video games with a sophisticated array of touch sensations (Senseg, 2011).
In terms of evaluation of the effectiveness of the haptic feedback on mobile gaming, Chui
Yin Wong and peers tested the playability of a mobile game, using a vibrotactile feedback
soft keypad, against playing the same game with a hard physical keypad. According to their
findings, although players had higher scores using the soft keypad, they reported a
preference on using the hard keypad in term of a better experience in the game play (Chui
Yin Wong, 2010).
6. Haptics and games for the visually impaired
Although haptics has proven to be a valuable technology to help gamers to have more
immersive and realistic experiences, there is yet another type of game for which haptics is of
essential value. Haptics is a very well suited type of interaction to address the problem of
accessibility to games for the visually impaired.
There a currently different technologies that have improved the accessibility of electronic
media to the blind people, such as automated reading software, voice synthesis, tactile
displays with Braille and speech recognition. Yet, those are mostly focused in input/output
of text and there is still some types of media that are still difficult to represent in a non-
visual modality, particularly graphics (2D or 3D), which form the basis for any computer
game. Computer haptics seems to be a suitable solution for this problem of game
accessibility, but still there are plenty of problems and research to be done on the field. It is
hard enough to have an accurate haptic representation of an object in a virtual world that
can be properly recognized by someone who is not able to look at it, and even harder to do
it for the multiple objects and events that need to be perceived during game play. In fact, in
most instances the haptic interface must be accompanied by some audio feedback as a
complement, as to no overload the user with haptic signals (Yuan, 2008).
There have been two main approaches to enable visually impaired individuals into
computer games. The first one is to design games that are inherently designed and
implemented with the purpose of not relying on visual feedback. That is games whose
playability is completely subject to interaction based on audio or haptic clues.
The Role of Haptics in Games
229
A survey of strategies for making games accessible to the blind shows that audio is the most
common used channel to either complement or substitute the visual channel (Yuan et al.
2010). AudioGames.net is an online community portal that has a repository of games whose
interface is solely based on audio and the natural haptic interface of keyboard and mouse
buttons (Audiogames, 2011). In terms of pure haptic games there are examples such as
Haptic Sudoku for the blind, developed by Gutschmidt and others. In this technological
solution they emulate visual perception completely by haptic perception. Players can feel
the Sudoku board and scan the numbers by sense of touch using a haptic display, while
audio cues alert the user of the outcome of their actions (Gutschmidt et al. 2010). Another
example is Finger Dance, a sound based game for blind people. In FingerDance, user try to
match musical rhythm patters to keystrokes in the keyboard. The game has no visual
feedback at all (Miller et al. 2007).
The second approach is sensory substitution. In this case the cues that would normally come
from the visual channel are replaced by haptic stimuli. This allows for the modification of
currently existing games that were originally designed for non disabled individuals and
adapted to be played by the blind.
Blind Hero is a typical example of a game where sensory substitution has been applied to a
very well known video game (Guitar Hero from Red Octane). Guitar Hero is a rhythm
action game that is played by using a guitar shaped input device with colored buttons on it
that must be pressed following the corresponding visual cues that appear on the screen at
the peace of some rock tune. In the modified version, Yuan and Fomer replaced the visual
cues with haptic stimuli coming from a haptic glove that has small pager motors that
stimulate the tip of each finger (Yuan and Fomer, 2008).
VI-Tennis and VI-Bowling are another example of sensory substitution from their analogous
version of the Wii Console (Wii Sports, 2011). In this case the haptic interface is based on a
motion sensing controller enhanced with vibrotactile and audio cues that allows the players
detect the key events in game play. In the case of VI-Tennis the controller would provide
vibrotactile feedback to reflect the event of the ball bouncing and the timeframe at which the
ball should be hit in return, while audio cues were left the same they ware on the original
Wii Tennis game (Morelli et al., 2010a). For the VI-Bowling they implemented a technique
called Tactile Dowsing where the Wii Remote is used to sense the direction to where the
bowling ball should be thrown. With the Tactile Dowsing the player moves the remote left
and right in a horizontal pattern. The receiver will detect the motion of the remote and the
software prompts a vibrotactile feedback on the remote in a pulsing pattern. The delay
between vibrotactile pulses is regulated so that the closer the player points to the optimal
direction of the throw, the lower the delay and vice versa. This way the player can sense the
direction the same way by trying to position the remote in a direction that has a near
continuous vibrotactile feedback (Morelli et al., 2010b).
7. Haptics and serious games
In general, the term “Serious Games” refers to computer video games that are used with a
purpose beyond or other than entertainment. However some discussion has been going on
for some time about the formal definition and some other details about what may or may
not constitute a serious game.
Haptics Rendering and Applications
230
According to a definition stated by the people on the Serious Games initiative, started in
2002 by Ben Sawyer and others (Serious Game Initiative, 2011), a serious game can be
defined as follows:
“applications of interactive technology that extend far beyond the traditional videogame market,
including: training, policy exploration, analytics, visualization, simulation, education, health,
and therapy.”
Serious games have been developed for many areas, including education, government,
military, corporate, healthcare, politics, religion and arts. Haptics technology has been
present to different extent in the majority of these areas. However, we argue that two of the
most prominent serious games application areas where the advantages of haptic technology
can really be appreciated are healthcare and education.
7.1 Healthcare
In this particular area there has been great interest in research how computer haptics
technology can assist in rehabilitation process of people who has had some damage to their
motor skills, while playing a game. The combination of the properties inherent to computer
haptics (force and tactile feedback) with the appealing and motivating factor offered by
Virtual Environments have provided a framework for the development of various
rehabilitation systems that often involve the patient, and the therapists.
Haptic Hand writing sessions and Ten Pin Bowling are two examples of haptic assisted
game applications developed by Xu and others at the University of Shanghai for Science and
Technology in China. Their Ten Pin Bowling is intended for training of the motor function
of post-stroke patients, based on Virtual Reality. They also demonstrate that haptic based
hand writing is an efficient way to improve motor skills, postural stability, control and hand
to eye coordination (Xu et al, 2010).
Jack and others worked on serious games rehabilitation using a haptic glove. In their work,
the interaction with a virtual environment is enabled by using a CyberGlove and a Rutgers
Master II-ND (RMII) force feedback glove. The Virtual Environment was designed to
promote the training rehabilitation of a specific parameter of hand movement: range, speed,
fractionation or strength. Their system would adapt to the level of rehabilitation achieved by
the patient (Jack et al. 2001).
Similarly Huber and collaborators developed a home based tele-rehabilitation system for
children with Hemiplegia. They made modifications to a play station game console to
support the use of a haptic glove and their custom made virtual environment game. The
game consisted in making specific hand movements to scare away butterflies that would
appear on the virtual environment (Huber et al. 2008).
Broeren et al. Studied the effects of virtual reality and haptics in stroke rehabilitation by
using a VR station loaded with a library of games and a hand held haptic stylus device. User
interacted with the virtual objects of the games, while the workstation collected data about
the 3D hand movements of the patients. They found that the enhanced rehabilitation
experience was highly motivational to the patients (Broeren et al. 2008).
The Role of Haptics in Games
231
7.2 Education
In the education field, most of the work has been towards providing a more immersive and
hands-on approach to learning class content. In particular chemistry is one of the subjects
often targeted thru haptic assisted learning tools (Fjeld and Voegtli, 2002).
As a good example, Sato and others designed a haptic grip and an interactive system with
haptic interaction as a teaching aid. They focused on the interaction between two water
molecules by constructing an environment to feel Van der Waals force as well as
electrostatic force with haptic interaction (Sato et al. 2008). In a similar way, Comai and
collaborators propose a framework for the implementation of haptic bases systems for
chemistry education (Comai et al 2010).
A second point of interest for haptics in education is that of learning hand writing skills. Eid
and others seveloped a multimedia system for learning hand writing in different languages.
The system is based on a haptic stylus controlled by software in such a way that it can guide
the movement of the learner in an analogous way that a teacher would hold his/her hand.
The system supports various languages including Arabic and Japanese. The amount of strict
guidance of the system can be adjusted so that overtime the user makes the writing by
him/her self (Eid et al. 2007).
In a similar work, kindergarten children were subject to hand writing training using a
Visuo-Haptic device to increase the fluency of handwriting production of cursive letters.
Forty two children participated in the experiment that showed that the fluency of
handwriting production for all letters was higher after the training with the tool than those
children which were not subject to it (Palluel-Germain et al., 2007).
8. Summary
Computer video games have come a long way and have evolved from a pure graphics and
visual rendering into a fully fledged multimedia experience. Videogames try to appeal to all
senses of the human player. Video graphics are very sophisticated and the definition, speed
and overall quality of the visual stimuli is at its best. Audio has become an integral part of
any computer video game as well, from small audio clues about events on the game to full
soundtracks that give character to the titles. We argue that video and audio were the
dominant media channels in the videogame industry in what could be called the first
generation of games.
In this new era where computers are miniaturized and with even higher capacities of
processing and memory, developers and researchers have focused on extending the sensory
experience to include the sense of touch. Initial implementations include vibrotactile
feedback with a rumble effect on game input controllers. However, in an attempt to achieve
a more immersive experience other haptics modalities have been explored, particularly force
feedback. There are already haptic devices available in a consumer level market that have
complex hardware implementation that allow for more realistic reactions from the controller
to game dynamics and user actions.
Another trend on games has been mobility. The wide availability of low cost, high
performance mobile phones and tablet PCs has created a whole new market for game titles
Haptics Rendering and Applications
232
that can be played on the go. Along with this trend haptic technology has been introduced
even on those constrained environment by means of sophisticated algorithms that can create
a whole array of vibrotactile effects for each type of game condition.
Finally, games have ceased to be a purely entertainment element. The so called serious games
are making people to engage in more practical and meaningful activities (in the fields of
education, government, military, corporate, healthcare, politics, religion and arts), while still
having the fun that game play always brings. Even in these particular type of games, haptic
technology promises to enhance the activity and may be even provide a type of interaction or
achieve a specific goal that would not be possible by using any other media channel.
In the future, it can be expected that machine haptics will evolve and improve into a more
robust technology reaching a wide spectrum of human endeavour. And we are sure that
gaming in all of its forms will always be one of the fields where the haptics potential can be
exploited.
9. References
Andrews S., J. Mora, J. Lang, W.S. Lee, "HaptiCast: a Physically-based 3D Game with Haptic
Feedback", Proc. of Future Play 2006, London, ON, Canada, October 2006
Broeren, J., Bjorkdahl, A., Claesson, L., et al. Virtual rehabilitation after stroke. Studies in
Health Technology and Informatics 136, (2008), 77-82.
Cha, Jongeun; Mohamad Eid; Ahmad Barghout; ASM Mahfujur Rahman; Abdulmotaleb El
Saddik. (2009). HugMe: synchronous haptic teleconferencing. In Proceedings of the
17th ACM international conference on Multimedia (MM '09). ACM, New York, NY,
USA, 1135-1136. DOI=10.1145/1631272.1631535
http://doi.acm.org/10.1145/1631272.1631535
Chui Yin Wong; Chu, K.; Chee Weng Khong; Tek Yong Lim; , "Evaluating playability on
haptic user interface for mobile gaming," Information Technology (ITSim), 2010
International Symposium in , vol.3, no., pp.1093-1098, 15-17 June 2010 DOI:
10.1109/ITSIM.2010.5561513
Comai Sara, Davide Mazza, and Lorenzo Mureddu. 2010. A haptic-based framework for
chemistry education. In Proceedings of the 5th European conference on Technology
enhanced learning conference on Sustaining TEL: from innovation to learning and
practice (EC-TEL'10), Martin Wolpers, Paul A. Kirschner, Maren Scheffel, Stefanie
Lindstaedt, and Vania Dimitrova (Eds.). Springer-Verlag, Berlin, Heidelberg, 614-619.
Eid Mohamad A., Mohamed Mansour, Abdulmotaleb H. El Saddik, and Rosa Iglesias. 2007.
A haptic multimedia handwriting learning system. In Proceedings of the
international workshop on Educational multimedia and multimedia education
(Emme '07). ACM, New York, NY, USA, 103-108. DOI=10.1145/1290144.1290161
http://doi.acm.org/10.1145/1290144.1290161
El Saddik, Abdulmotaleb. The Potential of Haptics Technologies IEEE Instrumentation
Measurement, February, 2007
Fogg BJ, Lawrence D. Cutler, Perry Arnold, and Chris Eisbach. 1998. HandJive: a device for
interpersonal haptic entertainment. In Proceedings of the SIGCHI conference on
Human factors in computing systems (CHI '98), Clare-Marie Karat, Arnold Lund,
Jo\&\#235;lle Coutaz, and John Karat (Eds.). ACM Press/Addison-Wesley
Publishing Co., New York, NY, USA, 57-64. DOI=10.1145/274644.274653
http://dx.doi.org/10.1145/274644.274653
The Role of Haptics in Games
233
Fjeld M. and Voegtli B. 2002. Augmented Chemistry: An Interactive Educational Workbench.
In Proceedings of the 1st International Symposium on Mixed and Augmented Reality
(ISMAR '02). IEEE Computer Society, Washington, DC, USA, 259-.
Gutschmidt René, Maria Schiewe, Francis Zinke, and Helmut Jurgensen. 2010. Haptic
emulation of games: haptic Sudoku for the blind. In Proceedings of the 3rd
International Conference on Pervasive Technologies Related to Assistive
Environments (PETRA '10), Fillia Makedon, Ilias Maglogiannis, and Sarantos
Kapidakis (Eds.). ACM, New York, NY, USA, , Article 2 , 8 pages.
DOI=10.1145/1839294.1839297
http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1839294.1839297
Huber, M., Rabin, B., Docan, C., et al. PlayStation 3-based tele-rehabilitation for children
with hemiplegia. Virtual Rehabilitation, 2008, (2008), 105-112.
Jack, D., Boian, R., Merians, A.S., et al. Virtual reality-enhanced stroke rehabilitation. IEEE
Trans. Neural Sys. And Rehab. Engr., 9, 3 (2001), 308-318.
Jurgelionis, A., Bellotti, F., IJsselsteijn, W. and de Kort, Y. (2007) Evaluation and testing
methodology for evolving entertainment systems. Proceedings of ACE 2007
International Conference on Advances in Computer Entertainment Technology,
Workshop 'Methods for Evaluating Games - How to measure Usability and User
Experience in Games'. ACM.
Katzourakis, D.; Gerard, M.; Holweg, E.; Happee, R.; , "Design issues for haptic steering
force feedback on an automotive simulator," IEEE International Workshop on
Haptic Audio visual Environments and Games, 2009. HAVE 2009., vol., no., pp.1-6,
7-8 Nov. 2009 doi: 10.1109/HAVE.2009.5356141
Kent, Stephen L. The Ultimate History of Video Games: From Pong to Pokemon--The Story
Behind the Craze That Touched Our Lives and Changed the World. hree Rivers
Press; 1 edition (October 2, 2001)
Miller Daniel, Aaron Parecki, and Sarah A. Douglas. 2007. Finger dance: a sound game for
blind people. In Proceedings of the 9th international ACM SIGACCESS conference
on Computers and accessibility (Assets '07). ACM, New York, NY, USA, 253-254.
DOI=10.1145/1296843.1296898
http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1296843.1296898
Minsky, M., Ming, O., Steele, O., Brooks, F.P., and Behensky, M., Feeling and seeing: issues
in force display, Proc. of the 1990 Symposium on Interactive 3D Graphics, pp. 235-
241, 1990.
Mohellebi, H. Kheddar, A. Espie, S. (2009) Adaptive Haptic Feedback Steering Wheel for
Driving Simulators. IEEE Transactions on Vehicular Technology, Volume: 58 Issue:
4. pp 1654 - 1666
Morelli Tony, John Foley, Luis Columna, Lauren Lieberman, and Eelke Folmer. (2010a). VI-
Tennis: a vibrotactile/audio exergame for players who are visually impaired. In
Proceedings of the Fifth International Conference on the Foundations of Digital
Games (FDG '10). ACM, New York, NY, USA, 147-154.
DOI=10.1145/1822348.1822368
http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1822348.1822368
Morelli Tony, John Foley, and Eelke Folmer. (2010b). Vi-bowling: a tactile spatial exergame
for individuals with visual impairments. In Proceedings of the 12th international
ACM SIGACCESS conference on Computers and accessibility (ASSETS '10). ACM,
New York, NY, USA, 179-186. DOI=10.1145/1878803.1878836
http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1878803.1878836
Haptics Rendering and Applications
234
Nilsson, D., and Aamisepp, H., Haptic hardware support in a 3D game engine. Master
thesis,Department of Computer Science, Lund University, May 2003.
Palluel-Germain R., F. Bara, A. Hillairet de Boisferon, B. Hennion, P. Gouagout, and E.
Gentaz. 2007. A Visuo-Haptic Device - Telemaque - Increases Kindergarten
Children's Handwriting Acquisition. In Proceedings of the Second Joint
EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual
Environment and Teleoperator Systems (WHC '07). IEEE Computer Society,
Washington, DC, USA, 72-77. DOI=10.1109/WHC.2007.13
http://dx.doi.org/10.1109/WHC.2007.13
Sang-Youn Kim; Jae-Oh Kim; Kyu Kim; , "Traveling vibrotactile wave - a new vibrotactile
rendering method for mobile devices," Consumer Electronics, IEEE Transactions
on, vol.55, no.3, pp.1032-1038, August 2009 DOI: 10.1109/TCE.2009.5277952
Sang-Youn Kim; Kyu Yong Kim; Byung Seok Soh; Gyunghye Yang; Sang Ryong Kim; ,
"Vibrotactile rendering for simulating virtual environment in a mobile game," IEEE
Transactions on Consumer Electronics , vol.52, no.4, pp.1340-1347, Nov. 2006 DOI:
10.1109/TCE.2006.273154
Sato Makoto, Xiangning Liu, Jun Murayama, Katsuhito Akahane, and Masaharu Isshiki.
2008. A haptic virtual environment for molecular chemistry education. In
Transactions on edutainment I, Abdennour El Rhabili (Ed.). Lecture Notes In
Computer Science, Vol. 5080. Springer-Verlag, Berlin, Heidelberg 28-39.
Teh, James Keng Soon; Adrian David Cheok, Roshan L. Peiris, Yongsoon Choi, Vuong
Thuong, and Sha Lai. (2008). Huggy Pajama: a mobile parent and child hugging
communication system. In Proceedings of the 7th international conference on
Interaction design and children (IDC '08). ACM, New York, NY, USA, 250-257.
DOI=10.1145/1463689.1463763 http://doi.acm.org/10.1145/1463689.1463763
Xu Zhaohong, Hongliu Yu, and Shiju Yan. 2010. Motor rehabilitation training after stroke
using haptic handwriting and games. In Proceedings of the 4th International
Convention on Rehabilitation Engineering \& Assistive Technology (iCREATe '10).
Singapore Therapeutic, Assistive \& Rehabilitative Technologies (START) Centre,
Kaki Bukit TechPark II,, Singapore, Article 31 , 4 pages.
Yuan Bei and Eelke Folmer. 2008. Blind hero: enabling guitar hero for the visually impaired.
In Proceedings of the 10th international ACM SIGACCESS conference on
Computers and accessibility (Assets '08). ACM, New York, NY, USA, 169-176.
DOI=10.1145/1414471.1414503
http://doi.acm.org.proxy.bib.uottawa.ca/10.1145/1414471.1414503
Yuan Bei. 2009. Towards Generalized Accessibility of Video Games for the Visually
Impaired. Ph.D. Dissertation. Univ. of Nevada, Reno, Reno, NV, USA. Advisor(s)
Frederick C. Harris and Eelke Folmer. AAI3355610.
Yuan, Bei., Folmer, E., and Harris, F. C. Game accessibility; a survey. Universal Access in the
Information Society 10, 1 (2010), 88-111.
Audio Games. Games based on sound. http://audiogames.net/. Retrieved June, 2011
Crystal Space 3D. http://www.crystalspace3d.org/. Retrieved June, 2011
Immersion. TouchSense Technology. http://www.immersion.com/products/motiv.
Retrieved June, 2011.
Novint Technologies. http://www.novint.com/. Retrieved June, 2011
SensAble Technologies. http://www.sensable.com/. Retrieved June, 2011
Senseg. E-Sense Technology. http://senseg.com/technology/senseg-technology. Retrieved
June, 2011.
TN Games. The 3rd Space Vest. http://tngames.com/products. Retrieved June, 2011
... SG is rapidly becoming an essential tool in the medical domain used with different goals, such as improving health behaviors, training, and research (Wattanasoontorn et al., 2013). These real simulations are developed through a multimodal interaction (de Freitas & Liarokapis, 2011) using animations, texts, language, graphics, haptics, audios, and so on (Arnab et al., 2011;Chicchi et al., 2018;Laamarti et al., 2014;Orozco et al., 2012). For instance, they can better include recorded voices and sounds to simulate social interactions in a real-world environment or to include distractors. ...
Article
Full-text available
Nowadays, there is a broad range of methods for detecting and evaluating executive dysfunction ranging from clinical interview to neuropsychological evaluation. Nevertheless, a critical issue of these assessments is the lack of correspondence of the neuropsychological test's results with real-world functioning. This paper proposes serious games as a new framework to improve the neuropsychological assessment of real-world functioning. We briefly discuss the contribution and limitations of current methods of evaluation of executive dysfunction (paper-and-pencil tests, naturalistic observation methods, and Information and Communications Technologies) to inform on daily life functioning. Then, we analyze what are the limitations of these methods to predict real-world performance: (1) A lack of appropriate instruments to investigate the complexity of real-world functioning, (2) the vast majority of neuro-psychological tests assess well-structured tasks, and (3) measurement of behaviors are based on simplis-tic data collection and statistical analysis. This work shows how serious games offer an opportunity to develop more efficient tools to detect executive dysfunction in everyday life contexts. Serious games provide meaningful narrative stories and virtual or real environments that immerse the user in natural and social environments with social interactions. In those highly interactive game environments, the player needs to adapt his/her behavioral performance to novel and ill-structured tasks which are suited for collecting user interaction evidence. Serious games offer a novel opportunity to develop better tools to improve diagnosis of the executive dysfunction in everyday life contexts. However, more research is still needed to implement serious games in everyday clinical practice.
... That can be applied in different contexts like education, healthcare, or interpersonal communication. Most researchers agree that digital serious games contain different media, which can be combinations of text, graphics (Consolvo at al., 208), animations (Lin et al., 2006), audio and video (Yim et al., 2007), haptics (Arnab et al., 2011), (Orozco et al., 2012) and others. Besides that, the term "serious" in serious games comes from their role of passing a message or contribution like knowledge, skills, or certain content to the player. ...
Article
Full-text available
Cultural heritage is a priceless asset for humanity, as it shows our achievements throughout the ages. Rapidly developing technologies and tools provide new opportunities in the process of preserving the cultural heritage. That process includes searching, research, identification, documentation, registration, conservation, restoration and adaptation of cultural-historic objects and resources. It is very important to include children in this process, to increase their awareness, stimulate their willingness to participate in its preservation and develop their affiliation with the values of the pan-European cultural heritage. Access to these resources, however, is not always possible or suitable for kids that are increasing their national identity and patriotism. That brings the need of developing new and different tools and methods for presenting digital cultural assets to a wider and more diverse audience and using them for educational purposes. In this article we describe a software component for Virtual Reality simulations and using them to present cultural-historic content to kids. We also demonstrate a three-dimensional lesson, taught in the web browser, which includes a collection of 3D objects for simulating the authentic room of the Bulgarian revolutionary Vasil Levski.
... That can be applied in different contexts like education, healthcare, or interpersonal communication. Most researchers agree that digital serious games contain different media, which can be combinations of text, graphics (Consolvo at al., 208), animations (Lin et al., 2006), audio and video (Yim et al., 2007), haptics (Arnab et al., 2011), (Orozco et al., 2012) and others. Besides that, the term "serious" in serious games comes from their role of passing a message or contribution like knowledge, skills, or certain content to the player. ...
Conference Paper
Full-text available
Cultural heritage is a priceless asset for humanity, as it shows our achievements throughout the ages. Rapidly developing technologies and tools provide new opportunities in the process of preserving the cultural heritage. That process includes searching, research, identification, documentation, registration, conservation, restoration and adaptation of cultural-historic objects and resources. It is very important to include children in this process, to increase their awareness, stimulate their willingness to participate in its preservation and develop their affiliation with the values of the pan-European cultural heritage. Access to these resources, however, is not always possible or suitable for kids that are increasing their national identity and patriotism. That brings the need of developing new and different tools and methods for presenting digital cultural as- sets to a wider and more diverse audience and using them for educational purposes. In this article we describe a software component for Virtual Reality simulations and using them to present cultural-historic content to kids. We also demonstrate a three-dimensional lesson, taught in the web browser, which includes a collection of 3D objects for simulating the authentic room of the Bulgarian revolutionary Vasil Levski.
Thesis
Full-text available
Accessibility requirements are often met on paper but in practice, some problems may still occur. The purpose of this thesis was to create a simulator, that allows test- ing different environments accessibility issues in virtual reality. The simulator is in- tended to work as a testing tool that can help to test buildings and its accessibility be- fore the building is constructed. This thesis goes through subjects like human re- sources, hardware and technical requirements, that are needed to create this simula- tor. Testing is done in a virtual reality environment with a wheelchair. Virtual wheelchair reacts to the environment like a real wheelchair. Environments that are used for this simulator must be placed precisely. Every threshold, door and doorknob must be in right place to make test results of this simulator valid. The thesis also discusses the problems and demands that are faced when modelling extensive environments like shopping centers or school buildings. Virtual reality can be very immersive, and this thesis undergoes what factors make the virtual reality experience immersive. The simulator is required to be immersive to make the test experience memorable. A memorable experience is much more per- sonal than a simple test report. This accessibility simulator was a big enthusiastic project that gathered a lot of infor- mation about how to create such a simulator. With the background work done, it is now easy to continue the development towards the essential production stage. Demos that were made for this project can work as a proof of the concept but should not be used as a base for the production version of the accessibility simulator. Next phase of the development is to make a working prototype and test it in a collab- oration with accessibility professionals. Prototype of an accessibility simulator needs to be tested with a real-world counterpart of an environment like a shopping center. According to test results with the prototype, it may be developed further towards the first public release.
Chapter
Mobile games have become a very common way of playing because of its portability and simplicity. In addition to rendering the environment atmosphere in game, sound can also play an important functional role in conveying information. However, the situation that people play mobile games without sound is very normal, both for hearing-impaired people and for normal people in public environment. Vibration is an important form of feedback information, which can enhance the player experience in soundless situation. But in most situations, they are only used to build atmosphere, not to convey specific information. There are few researchers have directly measured the effect of vibration on assisting mobile game and improving players’ engagement. This study compared the difference between playing game with and without vibration through a role-playing game (RPG) mod, to examining the effects of vibration on players’ engagement and which kind of vibration works better. Data analysis revealed that the participants who played the game with vibration were more engaged than participants who played without vibration. The results of this study will guide practitioners to explore more ways of how vibration can be used in mobile games to improve players’ engagement (both for normal people and hearing-impaired people).
Article
Game feel design is the intentional design of the affective impact of moment-to-moment interaction with games. In this paper we survey academic research and publications by practitioners to give a complete overview of the state of research concerning this aspect of game design. We analysed over 200 sources and categorised their content according to design purposes. This resulted in three different domains of intended player experiences: physicality, amplification, and support. In these domains, the act of polishing, which determines game feel, takes the shape of tuning, juicing, and streamlining respectively. Tuning the physicality of game objects creates cohesion, predictability, and the resulting movement informs level design. Juicing is the act of polishing amplification and it results in empowerment and provides clarity of feedback. Streamlining allows a game to act on the intention of the player, supporting the execution of actions in the game. These three design intents are the main means through which designers control minute details of interactivity and inform the player's reaction. The presented framework and its nuanced vocabulary can lead to an understanding of game feel that is shared between practitioners and researchers as highlighted in the concluding future research section.
Preprint
Game feel design is the intentional design of the affective impact of moment-to-moment interaction with games. In this paper we survey academic research and publications by practitioners to give a complete overview of the state of research concerning this aspect of game design, including context from related areas. We analysed over 200 sources and categorised their content according to the design purpose presented. This resulted in three different domains of intended player experiences: physicality, amplification, and support. In these domains, the act of polishing that determines game feel, takes the shape of tuning, juicing, and streamlining respectively. Tuning the physicality of game objects creates cohesion, predictability, and the resulting movement informs many other design aspects. Juicing is the act of polishing amplification and it results in empowerment and provides clarity of feedback by communicating the importance of game events. Streamlining allows a game to act on the intention of the player, supporting the execution of actions in the game. These three design intents are the main means through which designers control minute details of interactivity and inform the player's reaction. This framework and its nuanced vocabulary can lead to an understanding of game feel that is shared between practitioners and researchers as highlighted in the concluding future research section.
Conference Paper
Full-text available
Lack of physical activity is a serious health concern for individuals who are visually impaired as they have fewer opportunities and incentives to engage in physical activities that provide the amounts and kinds of stimulation sufficient to maintain adequate fitness and to support a healthy standard of living. Exergames are video games that use physical activity as input and which have the potential to change sedentary lifestyles and associated health problems such as obesity. We identify that exergames have a number properties that could overcome the barriers to physical activity that individuals with visual impairments face. However, exergames rely upon being able to perceive visual cues that indicate to the player what input to provide. This paper presents VI Tennis, a modified version of a popular motion sensing exergame that explores the use of vibrotactile and audio cues. The effectiveness of providing multimodal (tactile/audio) versus unimodal (audio) cues was evaluated with a user study with 13 children who are blind. Children achieved moderate to vigorous levels of physical activity- the amount required to yield health benefits. No significant difference in active energy expenditure was found between both versions, though children scored significantly better with the tactile/audio version and also enjoyed playing this version more, which emphasizes the potential of tactile/audio feedback for engaging players for longer periods of time.
Article
Full-text available
This paper presents a testing and evaluation methodology for evolving pervasive gaming and multimedia systems. We introduce the Games@Large system, a complex gaming and multimedia architecture comprised of a multitude of elements: heterogeneous end user devices, wireless and wired network technologies, and various multimedia and gaming services. It enables an easy access to the offered services through the use of a versatile system architecture. Testing and evaluating the design takes central stage in the project through an iterative process which has a great impact on overall design course and the final product in terms of assuring a proper quality of service and usability of the system. A comprehensive testing and evaluation methodology for newly evolving entertainment systems is developed after the adaptation of the metrics, used for video game and multimedia evaluation.
Article
Full-text available
Modern haptic technology allows users to receive realistic kinesthetic and tactile cues in a computer generated environment. When applied to video games, it gives players a higher sense of immersion as well as new and interesting ways to interact with the game environment. In this paper, we present a game which acts as an experimental framework for assessing haptic effects in 3D games. In HaptiCast, players assume the role of a wizard with an arsenal of haptically-enabled wands which they may use to interact with the game world. We discuss the integration of haptic feedback in this first-person shooter style video game that uses a "vanilla" 3D game engine. The haptic rendering algorithms and effects used in our approach are presented, along with some non-haptic features which enhance this modality.
Conference Paper
Full-text available
There have been escalating research interests on areas relating to haptic modality in recent years, especially towards mobile devices. However, there seems to be limited studies in determining the relation between haptic user interfaces and its influence on the playability of mobile games. This paper aims to measure playability of mobile games by comparing two different types of haptic interfaces, namely hard and soft keypad, for mobile gaming. The results show that the participants (N=12) scored highest in using a soft keypad (mean = 591.92, SD = 322.57) as compared to playing games using a hard keypad (mean = 471, SD = 359.29). However, this paper highlights that the majority of users prefer using the hard keypad method in virtue of greater player experience with regards to game play.
Conference Paper
Full-text available
This paper describes the deployment of a torque controlled force feedback steering device for human in the loop automotive simulations. The high bandwidth of the velocity controlled three phase brushless servomotor allows for a realistic representation of the virtual dynamics. The electromagnetic interference produced by the industrial servomotor and the controller made the elaboration stage challenging. To overcome the former issues careful amplification and filtering was ineluctable. The final dynamical model of the steering system used and the corresponding motor controller constitute an efficient haptic device for providing intuitive kinesthetic feedback. The system can feed back torques up to 20 NM with accuracy of 0.02 NM.
Conference Paper
Full-text available
Lack of sight forms a significant barrier to participate in physical activity. Consequently, individuals with visual impairments are at greater risk for developing serious health problems, such as obesity. Exergames are video games that provide physical exercise. For individuals with visual impairments, exergames have the potential to reduce health disparities as they may be safer to play and can be played without the help of others. This paper presents VI Bowling, a tactile/audio exergame that can be played using an inexpensive motion-sensing controller. VI Bowling explores tactile dowsing: a novel technique for performing spatial sensorimotor challenges, which can be used for motor learning. VI Bowling was evaluated with six blind adults. All players enjoyed VI Bowling and the challenge tactile dowsing provided. Players could throw their ball with an average error of 9.76 degrees using tactile dowsing. Participants achieved an average active energy expenditure of 4.61 kJ/Min while playing VI Bowling, which is comparable to walking.
Conference Paper
Full-text available
The usual approach to developing video games for people with visual impairment is sensory substitution. Elements of the visual display are replaced with auditory and/or haptic displays. Our approach differs. The purpose of the Finger Dance project is to research and develop accessible solutions to games that are inherently audio: musical rhythm-action games such as Dance Dance Revolution. However, these games still rely on visual cues that instruct the user on how to play along with musical rhythms. Finger Dance is an original audio-based rhythm-action game we developed specifically for visually impaired people. Working with both blind and sighted gamers using a human-centered development approach, players are able to play the game on their own and are enthusiastic about it. This paper discusses the game's design, development process and user studies.
Article
Rehabilitation training is crucial for stroke recovery and has to be performed on a daily basis. In order to motivate the stroke patients to practice motor trainings and improve rehabilitation process, haptic handwriting and Ten Pin Bowling game with Novint Falcon device are proposed as stroke rehabilitation tools. Ten Pin Bowling game is a virtual reality training which is intended to create a simulated environment for the retraining of motor function. Haptic handwriting can improve motor skill, postural stability and control, hand eye coordination of stroke patients. Some haptic applications for stroke rehabilitation trainings have been used to clinical experiments in Chinese hospitals. Furthermore, motor rehabilitation using haptic applications is encouraging and motivating, and the effectiveness of the proposed motor training method is confirmed via experiments.
Article
Controlling a virtual vehicle is a sensory-motor activity with a specific rendering methodology that depends on the hardware technology and the software in use. We propose a method that computes haptic feedback for the steering wheel. It is best suited for low-cost, fixed-base driving simulators but can be ported to any driving simulator platform. The goal of our method is twofold. 1) It provides an efficient yet simple algorithm to model the steering mechanism using a quadri-polar representation. 2) This model is used to compute the haptic feedback on top of which a tunable haptic augmentation is adjusted to overcome the lack of presence and the unavoidable simulation loop latencies. This algorithm helps the driver to laterally control the virtual vehicle. We also discuss the experimental results that demonstrate the usefulness of our haptic feedback method.