Content uploaded by Steven Kemper
Author content
All content in this area was uploaded by Steven Kemper on Jul 23, 2020
Content may be subject to copyright.
301
Tremolo-Harp: A Vibration-Motor Actuated Robotic String
Instrument
Steven Kemper
Mason Gross School of the Arts
Rutgers University
!85 George St.
New Brunswick, NJ 08901
skemper@mgsa.rutgers.edu
ABSTRACT
The Tremolo-Harp is a twelve-stringed robotic instrument, where each
string is actuated with a DC vibration motor to produce a mechatronic
“tremolo” effect. It was inspired by instruments and musical styles that
employ tremolo as a primary performance technique, including the
hammered dulcimer, pipa, banjo, flamenco guitar, and surf rock guitar.
Additionally, the Tremolo-Harp is designed to produce long, sustained
textures and continuous dynamic variation. These capabilities
represent a different approach from the majority of existing robotic
string instruments, which tend to focus on actuation speed and
rhythmic precision. The composition Tremolo-Harp Study 1 (2019)
presents an initial exploration of the Tremolo-Harp’s unique timbre
and capability for continuous dynamic variation.
Author Keywords
Musical Robotics, Mechatronics, Chordophone, Actuators, Tremolo
CCS Concepts
• Applied computing → Sound and music computing; Performing
arts;
1. INTRODUCTION
The musical term “tremolo,” ori ginating from the late Renaissance,
describes a rapidly articulated, repeating series of notes [4]. Whi le thi s
term comes from the Western art music tradition, tremolo articulation
is used as a performance technique for a wide variety of instruments
from around the world. While any instrument capable of rapidly
reiterating the same note(s) can be considered to produce tremolo, for
some instruments and musical styles, tremolo is the primary
performance technique. This is especially true for struck/plucked
string instruments, where tremolo not only produces an interesting
timbre, but also allows for continuous sustain of specific notes that
would otherwise decay quickly. Instruments and musical styles that
feature tremolo include the hammered dulcimer, pipa, banjo, flamenco
guitar, and surf rock guitar. Inspired by these instruments and styles,
the Tremolo-Harp is designed to create mechatronic “tremolo,” where
vibration motors directly actuate a series of strings, producing unique
timbral results. This instrument is also capable of continuous dynamic
variation by adjusting the intensity of vibration.
2. PRIOR WORK
The Tremolo-Harp was designed to produce continuous, tremolo-like
sounds where the dynamic shape can be controlled over time. This
instrument represents a development in the area of robotic instruments
that draws upon two different sources: robotic
1
string instruments and
electromagnetically-actuated string instruments.
2.1 Robotic String Instruments
Since the early 2000s, several robotic string instruments have
been developed, including [2, 9, 15, and 17]. While most of these
instruments are capable of producing tremolo articulations either
through repeated picking or “hammer-on” playing, the speed of
these articulations can be limited, as in the case of picking
mechanisms mounted on rotary stepper motors [9, 20].
Additionally, most robotic string instruments that are actuated
with a picking mechanism do not allow for dynamic control,
though the Protochord instrument created at the Victoria
University of Wellington represents a novel approach in this area
[12]. Solenoid-based hammer-on articulations are capable of
producing rapid tremolo and dynamic change. For example,
Cyther’s actuators are capable of producing rapid tremolos with
continuous dynamic changes, though faster striking requires
shorter on-times, reducing the dynamic range as the repetition
rate increases [2].
Some robotic string instruments are capable of producing
continuous sound. For example, there are several robotic systems
designed to bow violins, such as Koji Shibuya’s violin-playing
robot [16], as well as Godfried-Willem Raes’s automated bass
hurdy gurdy <Hurdy> [14]. While these instruments can produce
tremolo-type effects, their sizes and range of movement limit the
rate of repetition of the bowing mechanism.
2.2 Electromagnetically-Augmented String
Instruments
In the past several years, designers have also explored
continuous actuation of string instruments through audio-signal
driven electromagnetic transduction [10]. Notable examples of
this approach include the Electromagnetically-Prepared Piano
[3] and the Magnetic Resonator Piano [8], which both employ
audio-signal driven electromagnets to resonate piano strings,
allowing for continuous dynamic change over the course of a
note. This technique has also been explored in several of Raes’
robotic instruments, including the previously-mentioned
1
This paper employs the term “robotic instruments” for both
mechatronic instruments, which possess autonomous
performance capabilities as well as truly robotic instruments,
which include feedback from the environment. Colloquially,
especially in the field of music, both types of instruments are
often classified as “robotic.”
302
<Hurdy>, as well as the electromagnetically-actuated Aeolian
harp <Aeio> [13]. A similar technique was also explored in the
author’s earlier robotic instrument MARIE, co-created with
Expressive Machines Musical Instr ument s (EMMI) [15].
Though these technologies represent interesting hybrids of
electronic/acoustic sound production where the string becomes a
physical filter, their sonic output tends to be quite pure in tone,
lacking the unique timbral nuance produced by the mechanical
articulation of a string (though this is controllable based on input
signal).
3. DESIGN
3.1 Structure and Configuration
The Tremolo-Harp consists of twelve strings tuned chromatically from
E2-D#3 connected to a 24” x 16” t-slotted aluminum frame by 3D-
printed parts (see Figures 1-2). The dimensions of the instrument allow
it to fit in a fl ight case that conforms to standard checked baggage
sizing. Vibration motors are suspended next to each string. These
motors are wired in parallel with LEDs that illuminate according to the
signal being sent to the vibration motors, which serves to visualize the
string actuation. Dampening sol enoids are affixe d to the opposite end
of the instrument and are used to control sustain. A 5V power supply
is used for the vibration motors/LEDs and a 24V supply for the
solenoids. A Teensy 3.6 microcontroller is programmed as a USB
MIDI device, enabling MIDI output from a computer to control the
vibration motors and dampening solenoids [19]. St ring vibrat ions are
transduced using five standard electric guitar pickups and output via a
1/4” TS jack. The Tremolo-Harp’s output can be plugged directly into
a guitar amplifier or audio interface, and the output signal level is
controllable through a potentiometer on the side of the instrument.
Figure 1: Tremolo-Harp top view
Figure 2: Tremolo-Harp side view
3.2 Actuation
As previously mentioned, the Tremolo-Harp’s primary mode of
actuation consists of vibration motors striking the strings. After testing
a variety of different motors, the Jinlong Machinery & Electronics Co.,
Ltd’s Z4KH2B0470652 11000 RPM 3VDC motor emerged as the
best option in terms of size and vibrating force (see Figure 3). Pulse
width modulation (PWM) is used to control the voltage being sent to
the motors, which allows for continuous changes in dynami cs. An
early, smaller prototype of the Tremolo-Harp used an Arduino Uno
microcontroller; however, this produced an audible hum due to the
fixed PWM frequency of 490Hz on most pins (pins 5 and 6 output
980Hz) when using the analogWrite() function [1]. This frequency
was significantly amplified by the electric guitar pickups. Attempts to
mitigate this problem led to the use of the Teensy 3.6 microcontroller,
which has an easily-adjustable PWM frequency. The ideal frequency
for 8 bits of PWM resolution at a 96MHz clock speed is listed as
187500Hz in Teensy documentation [18]. Setting the PWM frequency
to this ultrasonic value eliminated the hum produced by the Arduino.
Additionally, the Teensy 3.6 provided enough PWM-capable and
standard digital pins to control twelve strings of vibration motors and
dampening solenoids.
Figure 3: Jinlong Machinery & Electronics Co., Ltd’s
Z4KH2B0470652 11000 RPM 3VDC vibration motor
4. SOFTWARE DESIGN AND CONTROL
4.1 MIDI Control Modes
In order to maximize the capabilities of the instrument, the Tremolo-
Harp is programmed with four control modes (see Table 1).
“Standard” control consists of activating both the vibration motors and
dampers, similarly to the action of a piano. This is achieved by sending
MIDI note numbers 52-63, where the velocity values are mapped to 8-
bit PWM values, controlling the intensity of vibration. MIDI note
numbers 40-51 allow for control of the vibration motors independently
of the dampers, which produces a muted effect, and note numbers 64-
75 allow for direct control of the dampers, which can be used to extend
the sustain of a note, or to produce percussive, hammer-on
articulations. One of the most important capabilities of the Tremolo-
Harp is the ability to continuously adjust the dynamics of the vibration
motors striking the strings. To facilitate this, control change messages
52-63 directly adjust the PWM value, and can be used to change
dynamics following a note on message.
Table 1: Tremolo-Harp control modes
Control Mode
MIDI Note Number*
Vibration Motors Only
40-51
Vibration Motors and
Dampers
52-63
Dampers Only
64-75
Vibration Motors
(continuous)
52-63 (*cc message)
4.2 Max for Live Software Interface
The goal of producing an instrument with continuously-varying
dynamic control made programming the Tremolo-Harp more
complicated than previous approaches to robotic stringed
instruments. For example, on EMMI’s Automatic Monochord
Instrument (AMI), note on messages are sufficient to depress a
tangent (to control pitch) and pick the string [15]. However,
exploring the capabilities of the Tremolo-Harp for sustain and
303
continuously-varying dynamics means that a variety of different
control techniques may be used for a single note gesture. For
example, a standard note-on message may be sent to initiate a
note, then control-change messages may be sent to adjust the
dynamics, with the damper solenoids remaining on to sustain the
note, akin to the sustain pedal on a piano. While it is possible to
automate both the note on and control change messages, for
example using automation curves in a digital audio workstation
or directly through a musical programming language, a
“Noteshaper” Max for Live device was developed to provide
specific control for this type of note gesture. This device allows
the user to draw in amplitude envelopes, set the duration, toggle
note off messages (allowing the dampers to remain lifted when
the note “ends”), and to add a time offset for note off messages,
which allows each string to sustain for a specified period of time
(see Figure 4). This device is controlled by MIDI note on
messages from Live. The adjustable parameters (Env. Preset #,
Env. Duration, Note Offs, and Note Off Delay Time) can also be
controlled through automation in Live.
Figure 4: Tremolo-Harp Noteshaper Max for Live device
(control for first three strings shown).
4.3 Sonic Output
Figures 5 and 6 show a waveform and spectrogram of a 10-
second triangular envelope using the Noteshaper Max for Live
device on E2 that produces a linear fade in and out of the PWM
value. The signal was sent through a Radial J48 DI box into an
RME Fireface UCX interface and recorded in Logic at 44.1 kHz
sampling rate/24 bits. This produced a dynamic range between -
68.2dB (minimum RMS) and -18.6dB (maximum RMS). By
looking at the waveform and spectrogram one can see that while
dynamic changes are possible, the response of the instrument is
nonlinear. On the one hand, this nonlinearity represents an
inherent feature of the Tremolo-Harp, which produces a uniquely
rich timbre based on its mechanical design. This approach is less
discretely controlled than other pick-based robotic string
instruments and produces more variable results. On the other
hand, part of this nonlinear response comes from the relationship
between PWM values and the response of the vibration motor
itself, which is nonlinear. In the future I plan to develop a
firmware-based lookup table that will produce a more linear
relationship between PWM values and vibration intensity. This
will allow the user more control over the Tremolo-Harp’s
dynamics without affecting the instrument’s unique sound.
Figure 5: Waveform display of 10-second triangular
envelope on E2. Created in Sonic Visualiser.
Figure 6: Spectrogram display of 10-second triangular
envelope on E2. Created in Sonic Visualiser.
5. MUSICAL RESULTS
The majority of contemporary robotic string instruments (with some
notable exceptions listed in Section 2) are built using designs that focus
on the production of discrete attacks (e.g. through picking
mechanisms) rather than continuous sound. As a result, musicians
working with these instruments tend to focus on musical gestures such
as hyper-virtuosic speed and complex rhythms [5]. This has certainly
been true in my own compositions for robotic instruments. As a result,
I wanted to explore the capabilities for mechatronic expression that
could be produced by an instrument that would primarily focus on
longer, more sustained sounds [6].
Tremolo-Harp Study 1 (2019) represents the first piece composed
for the Tremolo-Harp ( see li nk to video at the end of the paper). This
study explores the instrument’s unique timbral properties as well as its
capabilities to create long, sustained textures and continuous dynamic
changes. The overall musical concept for this piece is one of slowly-
evolving chords of differing durations that move across the range of
the instrument. While this composition does not incorporate all of the
performance techniques that the Tremolo-Harp is capable of, for
example hammer-ons played by the dampening solenoids, it does
demonstrate the instrument’s unique timbre and ability to produce
continuously-changing dynamics and sustained textures.
While the Tremolo-Harp can function as a solo instrument, its ability
to play sustained, chordal textures makes it well-suited to accompany
both human performers and existing solenoid-based robotic string and
percussion instruments that I have developed. A second study for this
instrument that is still under development explores human-robot
interaction by pairing the Tremolo-Harp with a live elect ric g uitarist.
Anticipation of this paring is one reason the Trem olo-Har p is tuned to
a chromatic scale beginning on E2.
To enable real-time interaction between a guitarist and the Tremolo-
Harp, I have developed a Max for Live device that applies pitch
tracking and envelope following to the guitar’s signal. This device
interfaces with the Noteshaper device described in section 4.2 and
allows the Tremolo-Harp to respond to both the guitar’s pitch and
dynamic shape. The init ial version of this device focuses on translating
these parameters to the Tremolo-Harp. For example, the envelope
following module can detect a picking gesture, which has a short
attack and longer decay. The Tremolo-Harp can th en perform this
gesture following the amplitude of the guitar’s signal at the original
pitch, at a pitch offset, or with a chord. By adjusting attack and decay
settings in the envelope following module, the Tremolo-Harp’s
gestures can be made longer or shorter than the sound of the live input.
Early explorations have focused on three types of guitar playing:
traditional picking (as explained above), using an EBow
(electromagnetic bowing device), and hammer-ons. The EBow
produces accompanying long, sustained gestures on the Tremolo-
Harp, while hammer-ons are mapped to short, staccato notes played
by the dampening solenoids.
304
6. CONCLUSION
The Tremolo-Harp represents a new approach to robotic string
instruments that employs a unique method of string articulation to
produce a distinctive timbre inspired by tremolo performance
technique, as well as the capability for long sustained textures and
continuous dynamic variation. This instrument provides
complementary capabilities to existing robotic string instruments,
which tend to focus on speed and precise timing. On a technical level,
refinements to this instrument will include development of the PWM
value lookup table described in section 4.3 and the use of PWM to
control the dampening solenoids. This would reduce the mechanical
sound of these actuators and could be used for other timbral effects.
Further musical exploration of this instrument will include developing
new studies that focus on human-robot interaction, incorporating
novel performance gestures, such as rapid hammer-ons played by the
dampening solenoids, and including the Tremolo-Harp in a growing
ensemble of robotic instruments.
7. ACKNOWLEDGMENTS
This project was completed with support from the Rutgers
University Research Council, the Rutgers Aresty Research
Center, and undergraduate research assistants Ryan Mulroney,
Zetao Yu, and Brad Miller. Fabrication support was provided by
Stacey Carton and the Rutgers University Libraries as well as the
Rutgers Makerspace.
8. ETHICAL STANDARDS
Funding for the Tremolo-Harp was provided by the Rutgers
University Research Council and personal research funds. This
project is free from conflicts of interest.
9. REFERENCES
[1] Arduino analogWrite()
https://www.arduino.cc/reference/en/language/functions/analog
-io/analogwrite/
[2] S. Barton et al. Cyther: a human-playable, self-tuning robotic
zither. In Proceedi ngs of the 2017 New Interfaces for Musical
Expression conference, Copenhagen, Denmark, 2017, 319-324.
[3] P. Bloland. The Electromagnetically-Prepared Piano and its
Compositional Implications, In Proceedin gs of the 2007
International Computer Music Conference. Copenhagen,
Denmark, 2007, 125-128.
[4] D. Fallows. Tremolo. In Grove Music Online, 2001, Retriev ed
January 16, 2020 from
http://www.oxfordmusiconline.com/view/10.1093/gmo/978156
1592630.001.0001/omo-9781561592630-e-0000028314
[5] S. Kemper. Composing for Musical Robots: Aesthetics of
electromechanical music. Emille: The Journal of the Korean
Electro-Acoustic Music Society 12 (2014), 25-31.
[6] S. Kemper, and S. Barton. Mechatronic Expression:
Reconsidering Expressivity in Music for Robotic Instruments.
In Proceedi ngs of the 18th International Conference on New
Interfaces for Musical Expression Virginia Te ch Univ ersity,
Blacksburg, VA, 2018, 84-87.
[7] L. Maes, G-W. Raes and T. Rogers. The Man and Machine
Robot Orchestra at Logos. Computer Music Journal 35, No. 4,
2011, 28–48.
[8] A. McPherson. The Magnetic Resonator Piano: Electronic
Augmentation of an Acoustic Grand Piano. Journal of New
Music Research 39, 3, 2010, 189–202.
[9] J. Murphy, et al. Expressive Robotic Guitars: Developments in
Musical Robotics for Chordophones. Computer Music Journal
39, 1, 2015, 59–73.
[10] D. Overholt, E. Berdahl and R. Hamilton. Ad vancements i n
Actuated Musical Instruments. Organised Sound 16, 2, 2011,
154–165.
[11] H. Park, e t al. A Study about Robotic Hand and Finger for
Violin Playing Robot. International Journal of Applied
Engineering Research 10, 11, 2015, 27553-27557.
[12] J. P. Y. Placencia, J. Murphy, and D. Carnegie. Exploring
Dynamic Variations for Expressive Mechatronic
Chordophones. In Proceedings of the 2019 Conference on New
Interfaces for Musical Expression (NIME’19), Federal
University of Rio Grande do Sul, Porto Alegre, Brazil, 2019,
413-418.
[13] G-W. Raes. <Aeio>
https://www.logosfoundation.org/instrum_gwr/aeio.html
[14] G-W. Raes. <Hurdy>
https://www.logosfoundation.org/instrum_gwr/hurdygurdy.htm
l
[15] T. Rogers, S. Kemper, and S. Barton. MARIE: Monochord-
Aerophone Robotic Instrument Ensemble. In Proceedings of
the 15th International Conference on New Interfaces for
Musical Expression (NIME) Louisiana State University, Baton
Rouge, LA, 2015, 408-411.
[16] K. Shibuya. Violin Playing Robot and Kansei. In Musical
Robots and Interactive Multimodal Systems. Springer, Berlin,
Heidelberg, 2011, 179–193.
[17] E. Singer, K. Larke, and D. Bianciardi. LEMUR GuitarBot:
MIDI Robotic String Instrument. In Proceedings of the 2003
Conference on New Interfaces for Musical Expression (NIME-
03), Montreal, Canada, 2003, 188-191.
[18] Teensy Pulsed Output: PWM & Tone
https://www.pjrc.com/teensy/td_pulse.html
[19] Teensy USB MIDI https://www.pjrc.com/teensy/td_midi.html
[20] R. Vindriis, A. Kapur, and D. Carnegie. A Comparison of
Pick-Based Strategies for Robotic Bass Playing. In
Proceedings of the Electronics New Zealand Conference,
2011, 67–72.
10. Video Example
Tremolo-Harp study 1: https://youtu.be/DUXXjP3IIYs
