Content uploaded by Josefin Starkhammar
Author content
All content in this area was uploaded by Josefin Starkhammar on Mar 14, 2016
Content may be subject to copyright.
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
ACOUSTIC TOUCH SCREEN FOR DOLPHINS
FIRST APPLICATION OF ELVIS - AN ECHO-LOCATION VISUALIZATION AND
INTERFACE SYSTEM
J. Starkhammar Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden
M. Amundin Kolmårdens Djurpark, Kolmården and Linköping University, Linköping, Sweden
H. Olsén Linköping University, Linköping, Sweden
M. Ahlmqvist Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden
K. Lindström Dept. of Electrical Measurements, Lund University, Lund, Sweden
H. W. Persson Dept. of Electrical Measurements, LTH, Lund University, Lund, Sweden
1 INTRODUCTION
Dolphin sonar has been extensively studied over several decades, and much of its basic
characteristics are well known (Au, 1993) [1]. Most of these studies have been based on an
experimental setup where the dolphin has been trained to be voluntarily fixed, so its directional
sonar beam could be recorded with fixed hydrophones. Although this allows for very exact
measurements, it most likely has prevented the full dynamic potential of the dolphin’s sonar to be
revealed. Also the dolphin’s response to scientific questions, e.g. in target detection threshold or
discrimination trials, mostly has been a “go/no go” response or pressing a yes/no paddle. This
traditional experimental methodology to measure the response makes rather coarse indications of
choice. It is difficult to refine and will be impractical with a multi-choice paradigm.
In cognitive studies with primates, e.g. the chimpanzee, a computerized symbol interface, based on
a finger operated touch screen, has been successfully used (Rumbaugh et al. 1975) [3]. Even with
birds, like chicken and doves, a similar approach has been used where the birds have used their
beak to indicate their choices (Cheng & Spetch, 1995) [4]. So far, however, to our knowledge a
conventional computer touch screen has not been attempted with dolphins, mainly because the
electro-magnetic grid over the touch screen would be short-circuited by the salt water. Delfour [2]
(2007) used a system that functionally was a touch screen. It was based on infrared light beams
projected through an underwater viewing panel, and guided by mirrors to create a grid in front of the
panel, and in front of the TV screen that was placed on the dry side of the panel. The dolphin
indicated its choice by breaking the light beams by its rostrum. This system was used in a study
testing the dolphin’s ability for self-recognition.
Although dolphins are known to use their rostrum to touch and manipulate things, we were
interested in exploiting and studying their main sensory system, their sonar. Therefore a new tool to
study dolphin sonar, psychophysics and cognitive skills was based on the dolphin’s sonar beam.
The new system was called the EchoLocation Visualization and Interface System (ELVIS) and was
developed at Lund University in cooperation with Kolmården Wild Animal Park [5]. It has been
further developed and is presently being tested in a dolphin food preference study at the Kolmården
Dolphinarium.
This system can function as an acoustic “touch screen” for the dolphins, i.e. the dolphin can indicate
a choice by aiming its sonar beam axis at designated areas on the screen. Hence, for the first time
the dolphins are given the opportunity to execute and run a computer program using their sonar
beam like we use a mouse cursor. This may be a much more intuitive response mode for dolphins
than the traditional go/no go or paddle press responses and equal to using their rostrum in the
Delfour touch screen [2]. The system is highly adaptable to testing a variety of different scientific
questions, since the core of the interface features is software based.
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
One primary reason to perform this particular study was to evaluate if the system will be possible to
use in future food preference investigations at Kolmården Dolphinarium. The aim of this study was
to develop a training procedure for introducing the “acoustic touch screen” concept to the dolphins,
to investigate if the concept of the interactive features of the system is comprehensible for the
dolphins and to test the software in order to develop it further and optimize it for this purpose.
2 METHOD
2.1 System configuration
ELVIS is based on a matrix of 16 hydrophones as seen in Figure 1. The distance between the
hydrophones is 300mm. The hydrophones are attached to a semi-transparent screen lowered into
the water of the pool, in front of an underwater acrylic panel. The hydrophones pick up the dolphins’
sonar signals aimed at the screen. The signals are transferred via cables to an amplifier and signal
conditioning unit, and from there via the parallel port to a computer, where the signal analysis is
performed by custom designed LabVIEW software. This software constitutes the core of the
interactive features of the system. Among many features, it makes it possible to trace the sonar
beam axis in real time by generating a round colour spot on the computer screen, corresponding to
the maximum intensity in the sound beam. The sound intensity can be coded in colour and/or light
intensity. In order to improve the rather coarse resolution given by the 16 hydrophones, the exact
location of the maximum sound intensity point is derived through interpolation between the
hydrophones in the matrix.
The resulting computer screen image is continuously projected back onto the hydrophone matrix
screen, using a standard PC projector, hence giving the dolphin an immediate visual feedback to its
sonar output. In the food preference study presented here, the theoretical centre of the beam was
traced by displaying a dark red spot on the screen. This colour was chosen to make the spot
inconspicuous or even invisible to the dolphin (Madsen and Herman, 1980,) [6], but still make it
possible for the experimenter to trace the scanning of the sonar beam over the screen.
Figure 1. The basic configuration of ELVIS (Echolocation Visualization and Interface
System). The screen in this particular mode plots the sound pressure level distribution in
the dolphin sonar beam.
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
The software used in this food preference study designates “buttons” or active areas on the screen,
indicated by visual symbols. See Figure 2. These symbols represented different fish species.
According to Yaman [7] these symbols should be easily discriminated visually by the dolphins.
Which symbols to be displayed on the screen were selected by the systems operator prior to each
trial.
The symbols were randomly placed over the entire screen, but were always placed over a
hydrophone. This was done to make sure that each symbol could be activated at any time, even if
the dolphin was very close to the screen, and the whole sonar beam may fall completely in between
the hydrophones. The size of these active areas of the screen as well as the trig level can easily be
altered so that, as the dolphin’s skill in handling the task improves, more accurate aiming and higher
sound pressure levels in the dolphin’s sound beam can be required.
2.2 Procedure
When the dolphin aims its sonar beam axis at a symbol, and generates clicks above a set threshold
level, it flashes to indicate a “hit” and a bridging stimulus (a 400 ms, 10 kHz sinus tone) is played
through two speakers placed close to the underwater acrylic panel. In this study, each of three such
symbols represented a different fish species (mackerel, capelin and squid; Figure 2). When the
dolphin “clicked” on one of them, it was rewarded by the fish represented by it. To help evaluating if
the choices were deliberate or random, a fourth symbol was introduced. This was essentially a
“wrong” button, and “clicking” on it resulted in the trial being aborted without any reward being
given.
Three female bottlenose dolphins (Tursiops Truncatus) were used in this study:
Vicky, born in 1973, arrived at Kolmården in 1979.
Ariel, born on October 6, 1996, at Kolmården. Mother: Vicky.
Luna, born on January 10, 2001, at Kolmården. Mother: Vicky.
Figure 2. Each symbol represents a fish species, except for the unfilled circle which
represent “false”. Only symbols and no text are displayed on the screen.
Standard operant conditioning procedures were used for training the dolphins to perform the task.
Approximately the same amount of fish of each species (approx. 50g) was given for each correct
choice.
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
3 RESULTS AND DISCUSSION
3.1 Training Procedure
The training procedure and the level of understanding the task could be divided into six main steps.
Each training session was video taped (the screen and the behaviour of the dolphin from above)
and selected parameters were logged by the software in spreadsheet files. These files contain data
such as the timestamp of a choice, the active dolphin, the projected symbols (their positions were
monitored by filming the screen), the chosen symbol, the required time of echolocating at the
symbols to induce a trig of the reward signal, the trig intensity level, the size of the active area
around the symbols, and the trig level.
1. At the very first training session only one symbol was projected onto the screen. In this
initial phase of the training only capelin was used. See Figure 2. To encourage the dolphins
to echolocate towards a symbol, a metal object was held in front of it. The trig level was low
and a relatively large area around the symbol was included in the active area, to make it
easier for the dolphin to obtain a trig/correct choice.
2. The metal object was phased out and the dolphins were required to aim their sonar beam
axis at the symbol, each time appearing in the same place on the screen. The active area
around the spot was decreased and the trig level was increased.
3. The position of the symbol was randomly altered for each trial. The dolphin was required to
click deliberately on the symbol, wherever it was positioned.
4. A second symbol, representing mackerel, was introduced and appeared at a random
position together with symbol number one, forcing the dolphins to make a choice in order to
receive the reward. To avoid random trigs during the approach, the dolphin was required to
line up in front of the screen before the symbols were displayed.
5. Symbol number three, associated with squid, was introduced, requiring the dolphin to select
between all three symbols at the same time.
6. Symbol number four, the “error” symbol, was introduced when the dolphin was performing
the task with confidence and clicking at symbols without hesitation. Only this symbol and
one of the food symbols were displayed at a time. Clicking on the “error” symbol resulted in
the trial being aborted, i.e. the screen was black and no reward signal was played. The
dolphin had to return to the trainer, received no fish and was given the cue to make a new
choice. This symbol was used as a control, aimed at indicating if the dolphin was capable of
making deliberate choices between the symbols.
The training sessions lasted 5-15 minutes for each dolphin, including 5-40 fish rewards, and were
carried out two to three times a week. They were all very motivated to perform the task. Therefore
the number of repetitions during each training session was limited only by the amount of fish
available for these training sessions. The total daily ratio had to be shared with other daily training
tasks, e.g. for the public display programmes and husbandry training.
3.2 Individual differences
Luna was the first to understand that the light symbol indicated the active area on the screen. She
reached step 6 after 16 training sessions within a total of 223 trials. The development of the other
two subjects stagnated at step 5, and their behaviour indicated that they had not even reached step
3. They apparently associated the rewards with aiming their sonar beam axis towards a transducer,
but failed to connect this to the symbols. Hence, in some trials they systematically tested the
individual transducers until they by chance echolocated at the one covered by a symbol. This may
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
indicate that they were locked into the acoustic domain, and that the echo characteristics of the
transducers were prioritized in the process of associating the stimuli to the task, and not the visual
symbols. It may also indicate that the dolphins have a “blind spot” along the sonar beam axis,
making it impossible for them to see the symbol while hitting it with the sonar beam axis. Still it
should be possible for them to select a symbol, since often, when the symbols were lit, they were
displayed outside the long axis of their snout. Their systematic tesing of hydrophones appeared to
occur more frequently when the symbols were displayed at the top (near the water surface) of the
screen. This also indicated that the performance of these two dolphins was influenced by the their
visual problems.
One reason for Luna’s better performance might be that she lined up 0.5-1m away from the screen,
whereas the other two had their rostrum only 0.1-0.2m from the screen. This probably helped her in
getting a better overview of the entire screen, hence making it easier for her to see the symbols at
any position on the screen.
Luna also had a different “clicking” technique than the other two. She did not use her sonar at all
until she had lined up in front of the screen and located the symbol of her choice. Then she
generated a precise click train directly at it. Vicky and Ariel both echolocated during approach and
then almost constantly after lining up in front of the screen, even when no symbols were shown.
When the symbols were displayed, they continued to echolocate while turning their sonar beam
towards the hydrophone of their choice.
The preliminar analysis indicates that the dolphins did not make any deliberate choice between the
three fish species. This may be due to all fish being equally appreciated – they were all a part of
their normal fish diet, and were used indiscriminately used during normal training - or just reflect that
the dolphins failed to discriminate between the symbols. Luna’s behaviour during the trials with the
“error button” indicates that the latter may be true. She “clicked” equally often on the “error button”
and displayed frustration and confusion when the screen was blanked and no bridging tone was
played. One conclusion that can be drawn from this is that the dolphins should have been taught
the concept of discrimination before being offered the option to select.
3.3 Evaluation of software
The software worked as intended and turned out to be very robust. Most of the interactive features
were automated. However, since it turned out to be cruicial not to display the symbols until the
dolphin were lined up in front of the screen, this had to be implemented by starting the software
manually at precisely the correct moment. After a correct response, the programme had to be
stopped manually. Thereafter the symbols had to be deleted from the screen by running a short
sequence of the programme with the setting “no symbols displayed”. These non-automated features
will be eliminated in future versions of the software.
The bridging stimulus was played via the speakers placed close to the underwater acrylic panel and
via the PA system. It was feared that this might have introduced another difficulty for the dolphins in
understanding to whom the signal was intended. Normally they would expect the whistle to come
from the trainer who gave the the task cue. However, this turned out not to be a problem. All three
dolphins reacted immediately to the stimulus, and never hesitated to swim back to the trainer to
collect the fish reward.
Since the core of the interface system is software based, is it easy to alter i. e. the properties of the
visual feedback to help optimize the performance and find a solution that makes the system as
intuitive as possible to the dolphins.
Proceedings of the Institute of Acoustics
Vol. 29. Pt.3 2007
4 CONCLUSIONS
The interface function of the system worked as intended. The dolphins quickly understood the
requirement of echolocating towards the screen. They were highly motivated to perform the task
and did not have a problem with understanding that the bridging stimulus, played by the software,
was associated with their performance at the screen.
One major advantage with the system set-up is the possibility to adjust the visual feedback via the
software. The initial training of the dolphins showed that the concept of an acoustic touch screen
was comprehensible to all of them. However, individual differences in the learning to use the visual
symbols indicate that improvements in the visual feedback are motivated. This was demonstrated
by the behaviour of two of the dolphins, that apparently searched for the discriminative stimulus in
the acoustic properties of the hydrophones, instead of associating it with the visual symbol. This
problem needs to be understood and solved to reach the full potential of the system as an intuitive
response tool.
The dolphins did not show a preferrence for any of the three fish species. This may be a true non-
preference, but also indicate a failure of the dolphins to discriminate between the symbols. Whether
this was due to a visual problem or a result of the training regime remains to be resolved.
From all the results discussed, the major conclusion must be that this “acoustic touch screen” has
the potential to be a valuable research tool for future psychophysical and/or cognitive studies of
dolphins, e.g. for the more elaborate food preference investigations that will be carried out at the
Kolmården Dolphinarium.
REFERENCES
1. Au, W. L.,The Sonar of Dolphins. Springer-Verlag, New York, 277pp. (1993)
2. Delfour F. Marine mammals in front of the mirror – body experiences to self-recognition: A
cognitive ethological methodology combined with phenological questioning. Aquatic
Mammals 32(4):517-527.(2007)
3. Rumbaugh D.M, Gill, T.V, von Glasersfeld, E, Warner, H, and Pisani, P., Conversations
With a Chimpanzee in a Computer-Controlled Environment. Biol Psychiatry. 10(6):627-41.
(1975)
4. Cheng, K., & Spetch, M. L., Stimulus control in the use of landmarks by pigeons in a touch-
screen task. Journal of the Experimental Analysis of Behaviour, 63, 187-201. (1995)
5. Nilsson, M., Lindström, K., Amundin, M., Persson, H. W., Echolocation and Visualization
Interface System, Clinical Physiology and Functional Imaging, 24, 3, 169-178, (2004)
6. Madsen, C.J. and Herman, L.M., Social and Ecological Correlates of Cetacean Vision and
Visual Appearance. In: Cetacean Behavior- Mechanisms and Functions. L.M. Herman (ed.)
John Wiley & Sons, NY. (1980)
7. Yaman, S. von Fersen, L., Dehnhardt G., Güntürkü O., Visual Lateralization in the
Bottlenose Dolphin (Tursiops truncatus): Evidence for a Population Asymmetry.
Behavioural Brain Research, 143, 109-114. (2003)