Content uploaded by Mitchell Tyler
Author content
All content in this area was uploaded by Mitchell Tyler on Apr 21, 2014
Content may be subject to copyright.
November 13, 2003 13:42 WSPC/179-JIN 00026
Journal of Integrative Neuroscience, Vol. 2, No. 2 (2003) 159–164
c
Imperial College Press
Short Communication
CLOSING AN OPEN-LOOP CONTROL SYSTEM:
VESTIBULAR SUBSTITUTION THROUGH THE TONGUE
MITCHELL TYLER,
∗,†
YURI DANILOV
∗,†
and PAUL BACH-Y-RITA
∗,†,‡
∗
Wicab, Inc, 3510 W. Beltline Hwy, Middleton, WI 53562, USA
†
Department of Biomedical Engineering,
‡
Department of Orthopedics and Rehabilitation Medicine,
University of Wisconsin, Madison, WI 53706, USA
∗
metyler1@facstaff.wisc.edu
Received 16 April 2003
Accepted 27 July 2003
The human postural coordination mechanism is an example of a complex closed-loop
control system based on multisensory integration [9, 10, 13, 14]. In models of this process,
sensory data from vestibular, visual, tactile and proprioceptive systems are integrated
as linearly additive inputs that drive multiple sensory-motor loops to provide effective
coordination of body movement, posture and alignment [5–8, 10, 11]. In the absence of
normal vestibular (such as from a toxic drug reaction) and other inputs, unstable posture
occurs. This instability may be the result of noise in a functionally open-loop control system
[9]. Nonetheless, after sensory loss the brain can utilize tactile information from a sensory
substitution system for functional compensation [1–4, 12]. Here we have demonstrated
that head-body postural coordination can be restored by means of vestibular substitution
using a head-mounted accelerometer and a brain-machine interface that employs a unique
pattern of electrotactile stimulation on the tongue. Moreover, postural stability persists
for a period of time after removing the vestibular substitution, after which the open-loop
instability reappears.
Keywords: Vestibular; sensory substitution; electrotactile stimulation; brain plasticity.
1. Introduction
Persons who have bilateral vestibular damage, such as from an adverse reaction
to antibiotic medications, experience functional difficulties that include postural
“wobbling” (both sitting and standing), unstable gait, and oscillopsia that make it
impossible, for example, to walk in the dark without risk of falling. This condition
presents the unique opportunity to: (i) study a model of an open-loop human control
∗
Corresponding author.
159
November 13, 2003 13:42 WSPC/179-JIN 00026
160 Tyler, Danilov & Bach-y-Rita
system, and (ii) to re-establish head-postural control by means of vestibular sub-
stitution using a head-mounted accelerometer and an electrotactile brain-machine
interface (BMI) through the sense of touch on the tongue.
The control of normal upright posture is mediated by a complex system that
relies on the integration of multiple sensory inputs (e.g., vestibular, visual, tactile,
proprioceptive, and auditory) [5–8, 10, 11]. It remains unclear as to specifically how
this sensory fusion acts in self-orientation. Nonetheless, the models developed to
predict the effect of the various inputs have demonstrated that these systems provide
convergent and redundant information about the position of body segments relative
to each other and to overall body orientation in space. In particular, detection of
linear and angular acceleration of the head may be used to resolve self-motion from
that of the surrounding visual environment.
In the absence of a functional vestibular system, head-posture may be detected
by an artificial sensor and presented to the brain through a substitute sensory chan-
nel: electrotactile stimulation on the tongue. We have previously demonstrated the
merits of the tongue as a brain-machine interface (BMI) [1, 4]. For the brain to cor-
rectly interpret information from a sensory substitution device, it is not necessary
that the information be presented in the same form as the natural sensory system.
For example, we do not see with the eyes; the optical image does not go beyond the
retina, where it is turned into spatio-temporal patterns of action potentials (AP’s)
along the optic nerve fibers [6]. The brain then recreates the images from analysis of
the impulse patterns. Thus, for a sensory substitution event to occur, one need only
to accurately entrain action potentials in an alternate information channel, which
do not differ significantly for the individual senses. With training, the brain learns
to appropriately interpret that information and utilize it to function as it would
with data from the intact natural sense.
The use of vestibular sensory substitution produces a strong stabilization effect
on head and body coordination in subjects with bilateral vestibular dysfunction
(BVD). Under experimental conditions (in we which removed visual and tactile in-
puts: see Methods), we identified three characteristic and unique motion features
(mean-position drift, sway, and periodic large-amplitude perturbations) that con-
sistently appear in the head-postural behavior of BVD subjects (see Fig. 1). With
vestibular substitution (VS) however, the magnitude of these features are greatly
reduced or eliminated. Further analysis of the experimental data revealed that these
perturbations are periodic (within individual) and did not occur just at the extremes
of motion, which would suggest they are triggered by proprioceptive mechanisms.
Specifically, we found that these postural “spikes” do not always correct for the
large coincident postural excursions, but in many instances appeared to actually
cause instability. We postulate that in the absence of the integrated inputs to a
normally closed-loop multisensory control process an intrinsically unstable system
becomes vulnerable to noise (from both internal and external sources), as character-
ized by Jeka [9]. The result of providing vestibular substitution supports this specific
November 13, 2003 13:42 WSPC/179-JIN 00026
Closing an Open-Loop Control System 161
20
40
60 80
Time (sec.)
Sway (deg.)
Normal
BVD
BVD + VS
Normal BVD BVD + VS
L/R
A/P
Average RMS ± SD
a
b
c
5
10
Sway (deg.) Time (s)
50
100
Without - VS
With - VS
-20 0 20
-20
0
20
-20
0
20
-20
0
20
-20
0
20
A
P
Sway(deg.)
-20
0
20
-20
0
20
50
100
-20
0
20
-20
0
20
-20 0 20
-20
0
20
LR
A
P
R
L
Sway (deg.)
Fig. 1. Vestibular substitution stabilization effect. (a) Graph of head displacement in both an-
terior/posterior (A/P), and medial-lateral (M/L) directions for an adult subject with eyes closed
and sitting upright without back support. Top: Typical profile for an unaffected individual. Mean
amplitude: ±1.4 deg. (M/L); ±1.8 deg. (A/P), is centred about zero. Center: Subject with bilateral
vestibular dysfunction (BVD). Mean amplitudes: ±3.0 deg. (M/L); ±7.6 deg. (A/P). Note the slow
drift of the mean position, and occurrence of periodic (∼ 23 s.) perturbations. Bottom: The same
subject while using tactile vestibular substitution (VS). Mean amplitudes of angular displacement
are reduced to: ±1.4 deg. (M/L); ±3.1 deg. (A/P). (b) “Spaghetti” plots of the same displacement
profiles. Left: 3-Dimensional graph showing head position as a function of time (vertical axis). Inset:
2-D projection onto horizontal plane. Right: Performance with tactile vestibular substitution (VS).
(c) Box plots showing average (RMS) and Standard Deviation (SD) of angular displacement after
linear regression across time to extract magnitude and direction of mean-position drift. Left pair:
Overall results for eight UA subjects. Center pair: Cumulative results of eight successfully com-
pleted trials for one BVD subject. Right pair: Cumulative results of eight trials for same clinical
subject using (BVD with VS). Mean M/L performance approaches that of normal head-postural
behavior. Motion in A/P direction is slightly larger and more variable, but clearly superior to the
unaided condition.
November 13, 2003 13:42 WSPC/179-JIN 00026
162 Tyler, Danilov & Bach-y-Rita
0 0 100100
01002000100 200
0 100 200 3000 100 200 300
Sway (deg.)
45
0
- 45
45
0
- 45
45
0
- 45
Time (sec.)
b
c
a
Fig. 2. Vestibular substitution after-effect. Results of the second experiment (EC-VS → EC). Left
half of each row is with VS, right half is the post-VS period. A consistent proportionality of relative
head-postural stability without VS to the period with VS exists across the three test durations
(a: 100, b: 200 and c: 300 sec. for each phase). Oscillations consistently begin with small A/P
motion at the head at approximately 30% into the non-VS period, while the torso remains initially
stable. Head-movement gradually increases in amplitude, and at approximately 70% of the post-VS
(EC) period motion begins to involve the torso, again growing in amplitude over time until the
subject is unable to maintain stability.
characterization of the head-postural control system, and demonstrates that vestibu-
lar information plays a crucial role in the overall control postural process.
During these experiments the BVD subjects reported feeling “normal”, “stable”,
or having reduced perceptual “noise” while using VS and for short periods after re-
moving the stimulation. This “after-effect” was specifically explored in a second
experiment involving a single BVD subject by recording head position during pre-
scribed periods of VS, followed without interruption by equal periods without tactile
or other feedback. As can be seen in Fig. 2, there is an initial period of relative quiet
after removal of VS. Then at about 30% into the non-VS period, small amplitude
anterior-posterior (A/P) oscillations begin to appear at the head, progressively in-
creasing in amplitude with time. At approximately 60% into this period torso A/P
motion becomes evident, and increases with time until finally the entire upper body
reaches instability and the experiment is terminated. The duration of the post-VS
instability appears to be linearly related to the duration of VS period (within the
ranges tested). These results again provide evidence that the presence of meaning-
ful substitutive input to the multisensory postural control process is sufficient to
November 13, 2003 13:42 WSPC/179-JIN 00026
Closing an Open-Loop Control System 163
produce stability approaching that of unaffected individuals [cf. Fig. 1(c)]. Con-
versely, in the absence of valid data from vestibular, visual and tactile sources, the
system appears inherently noisy and unstable.
The results presented here support the concept of developing practical tactile
sensory substitution and augmentation systems based on brain plasticity [2]. The
technology may also be applicable to vestibular stress situations such as for astro-
nauts and pilots who are subject to spatial disorientation [10]. The tongue BMI may
also be applied to other sensory substitution systems such as for blindness, deafness
or diabetic insensate feet, and to augmentation systems such as for urban search
and rescue (with an infrared camera) or for underwater orientation and navigation
[1–4, 12]. Finally, the BMI also offers a tool for studies of brain reorganization, some
of which have already been reported [2, 4].
2. Methods
A miniature 2-axis accelerometer (Analog Devices ADXL202) was mounted on a low-
mass plastic hard hat. Anterior-posterior and medial-lateral angular displacement
data (derived by double integration of the acceleration data) were fed to a previously
developed tongue display unit (TDU) that generates a patterned stimulus on a 144-
point electrotactile array (12 × 12 matrix of 1.8 mm diameter gold-plated electrodes
on 2.3 mm centers) held against the superior, anterior surface of the tongue [1].
Subjects readily perceived both position and motion of a small “target” stimulus
on the tongue display, and interpreted this information to make corrective postural
adjustments, causing the target stimulus to become centered.
Four subjects with bilateral vestibular dysfunction (BVD: 2 M, 2 F: mean age
49.8 yr., SD = 9.7 yr.) and eight unimpaired subjects (5 M, mean age = 40.6 yr.,
SD = 15.5 yr.; 4 F, mean age = 41 yr., SD = 9.8 yr.) were studied using repeated
measures in two basic conditions. Subjects were seated in a modified Romberg posi-
tion (elbows lightly cupped in opposite hands): Eyes Closed (EC), and Eyes Closed
with Vestibular Substitution (EC-VS) for trials of 100 seconds duration. In a second
experiment, a single BVD subject was tested in the EC-VS mode for 100, 200, or
300 seconds, followed without interruption by the EC condition for an equal period
(EC-VS → EC).
Acknowledgments
This study was supported by an NIH SBIR grant (1 R43 DC04738), and the Uni-
versity of Wisconsin-Madison Industrial and Economic Development — Robert F.
Draper Technology Innovation Program.
References
[1] Bach-y-Rita P., Brain Mechanisms in Sensory Substitution (Academic Press, New York,
1972).
November 13, 2003 13:42 WSPC/179-JIN 00026
164 Tyler, Danilov & Bach-y-Rita
[2] Bach-y-Rita P., Nonsynaptic Diffusion Neurotransmission and Late Brain Reorganiza-
tion (Demos-Vermande, New York, 1995).
[3] Bach-y-Rita P., Collins C. C., Saunders F., White B. and Scadden L., Vision substitu-
tion by tactile image projection, Nature 221 (1969) pp. 963–964.
[4] Bach-y-Rita P., Kaczmarek K., Tyler M. and Garcia-Lara J., Form perception with a
49-point electrotactile stimulus array on the tongue, J. Rehab. Res. Develop. 35 (1998)
pp. 427–431.
[5] Horak F. B. and Hlavaca F., Somatosensory loss increases vestibulospinal sensitivity,
J. Neurophysiol. 86 (2001) pp. 575–585.
[6] Horak F. B. and Macpherson J. M., Postural orientation and equilibrium. In Handbook
of Physiology, ed. by Shepard J. and Rowell L. (Oxford University Press, NY, 1996)
pp. 255–292.
[7] Jeka J. and Lackner J. R., Fingertip contact influences human postural control, Exp.
Brain Res. 100 (1994) pp. 495–502.
[8] Jeka J., Oie K. S., Sch¨oner G., Dijkstra T. H. M. and Henson E. M., Position and
velocity coupling of postural sway to somatosensory drive, J. Neurophysiol. 79 (1998)
pp. 1661–1674.
[9] Jeka J., Oie K. S. and Kiemel K., Multisensory information for human postural control:
Integrating touch and vision, Exp. Brain Res. 134 (2000) pp. 107–125.
[10] Lackner J. R., Multimodal and motor influences on orientation: Implications for adapt-
ing to weightless and virtual environments, J. Vestibular Res. 2 (1992) pp. 307–322.
[11] Lackner J. R., DiZio P., Horak F., Krebs D. and Rabin E., Precision contact of the
fingertip reduces postural sway of individuals with bilateral vestibular loss, Exp. Brain
Res. 126 (1999) pp. 459–466.
[12] Sampaio E., Maris S. and Bach-y-Rita P., Brain plasticity: “Visual” acuity of blind
persons via the tongue, Brain Res. 908 (2001) pp. 204.
[13] Van der Hooij H., Jacobs R., Koopman B. and Grootenboer H., A multisensory inte-
gration model of human stance control, Biol. Cybern. 80 (1999) pp. 299–308.
[14] Van der Hooij H., Jacobs R., Koopman B. and Van der Helm F. H., An adaptive model
of sensory integration in a dynamic environment applied to human stance control, Biol.
Cybern. 84 (2001) pp. 103–115.
