Conference PaperPDF Available

ShArc: A Geometric Technique for Multi-Bend/Shape Sensing CCS Concepts •Hardware → Emerging interfaces; Sensors and actua- tors

Authors:

Abstract and Figures

Figure 1: a. A capacitive ShArc sensor uses a series of flexible strips that are joined together at one end. The outer electrode strips are held a constant distance apart via an elastic sleeve which compresses them against a series of spacers. A circuit measures the relative shift between the electrode layers as the strips are formed into the curves. b. When the strips are bent, the ends no longer align due to the varying radii of curvature. c and d. Interacting with the prototype and real-time reconstruction of dynamic bends. ABSTRACT We present ShArc, a precision, geometric measurement technique for building multi-bend/shape sensors. ShArc sensors are made from flexible strips that can be dynamically formed into complex curves in a plane. They measure local curvature by noting the relative shift between the inner and outer layers of the sensor at many points and model shape as a series of connected arcs. Unlike jointed systems where angular errors sum with each joint measured, ShArc sensors do not accumulate angular error as more measurement points are added. This allows for inexpensive, robust sensors that can accurately model curves with multiple bends. To demonstrate the efficacy of this technique, we developed a capacitive ShArc sensor and evaluated its performance. We conclude with examples of how ShArc sensors can be employed in applications like gesture input devices, user interface controllers, human motion tracking and angular measurement of free-form objects.
Content may be subject to copyright.
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
ShArc:
A Geometric Technique for Multi-Bend/Shape Sensing
Fereshteh Shahmiri Paul H. Dietz
Georgia Institute of Technology - Tactual Labs Tactual Labs
Atlanta, USA Redmond, USA
sshahmiri3@gatech.edu paul.dietz@tactuallabs.com
Figure 1: a. A capacitive ShArc sensor uses a series of flexible strips that are joined together at one end. The outer electrode strips
are held a constant distance apart via an elastic sleeve which compresses them against a series of spacers. A circuit measures the
relative shift between the electrode layers as the strips are formed into the curves. b. When the strips are bent, the ends no longer
align due to the varying radii of curvature. c and d. Interacting with the prototype and real-time reconstruction of dynamic bends.
ABSTRACT
We present ShArc, a precision, geometric measurement tech-
nique for building multi-bend/shape sensors. ShArc sensors
are made from flexible strips that can be dynamically formed
into complex curves in a plane. They measure local curvature
by noting the relative shift between the inner and outer layers
of the sensor at many points and model shape as a series of
connected arcs. Unlike jointed systems where angular errors
sum with each joint measured, ShArc sensors do not accu-
mulate angular error as more measurement points are added.
This allows for inexpensive, robust sensors that can accurately
model curves with multiple bends. To demonstrate the efficacy
of this technique, we developed a capacitive ShArc sensor and
evaluated its performance. We conclude with examples of how
ShArc sensors can be employed in applications like gesture in-
put devices, user interface controllers, human motion tracking
and angular measurement of free-form objects.
Author Keywords
ShArc; Sensor; Bend; Multi-Bend ; Shape; Capacitive.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from permissions@acm.org.
CHI ’20, April 25–30, 2020, Honolulu, HI, USA.
© 2020 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-6708-0/20/04 ...$15.00.
http://dx.doi.org/10.1145/3313831.3376269
CCS Concepts
Hardware Emerging interfaces; Sensors and actua-
tors;
INTRODUCTION
Recent decades have seen tremendous progress in the devel-
opment of high accuracy sensors and their low cost, mass
production. Much of this has been driven by smartphones
which include an impressive array of sensors. Despite these
advancements, there are still many things about the physical
world that have proven surprisingly difficult to sense with an
inexpensive, precision device. We consider the challenging
problem of sensing the shape of a dynamically deforming
object.
The desire to understand shape arises in many applications. In
robotics, rotary joints are frequently cascaded to allow dexter-
ous, multi-axis motion that must be monitored to be actively
controlled. Launching a large rocket has been compared to
"pushing on a string", and it requires a detailed understand-
ing of dynamic flexure. Bridges, storage tanks, planes, and
many other structures are subject to repeated load cycling, and
understanding deformation in these systems can help prevent
catastrophe. More germane to the Human-Computer Inter-
action (HCI) community, our bodies are quite flexible. In
medicine and sports performance, it is often important to un-
derstand the range and type of motion. Motion capture is
critically important to both the gaming and movie industries.
In virtual and augmented reality, a real-time understanding of
Paper 142
Page 1
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
detailed hand pose allows compelling interactions. For per-
formance, musicians and other artists can manipulate shape
intuitively to provide expressive control of key systems.
In this work, we present ShArc - a new technique for low cost,
precision, dynamic sensing of bends and the detailed shape of
curves. ShArc - a portmanteau of "Shift Arc" - employs a stack
of flexible strips that can be formed into complex curves in a
plane. The sensor measures curvature by noting the relative
shift between inner and outer layers of the sensor at many
points. For example, as illustrated in Figure 2 , if the sensor is
wrapped around a cylinder, the inner most strip will be formed
into a radius similar to the cylinder, while the outer strip will
effectively conform to a slightly larger radius dependent on
the spacing between the layers. The outer strip will need more
length to cover the same angular extent. If equal length strips
are conjoined on one end and bent around a cylinder, the other
ends will separate, similar to what happens when the pages
of a book are bent. This relative shifting provides a measure
of curvature. We measure this relative shift at many locations
along the strips, giving an accurate measure of curvature at
many points.
ShArc sensors allow for a unique combination of true multi-
bend/shape sensing, robustness to measurement errors, stabil-
ity, and low cost. In this paper, we present the theory behind
ShArc and some of its unique characteristics. We then exam-
ine a prototype implementation which uses capacitive sensing
and characterize its performance. Finally, we consider some
example applications in the HCI domain.
RELATED WORK
In this section, we consider prior work in flexible, self-
contained, bend/shape sensors. We first consider how bend is
physically detected and then discuss specific implementations.
Strain Sensing
The most common way of detecting flexure is by measuring
the changing properties of a material under strain [36]. Strain
is a problematic proxy for flexure. Stretching, environmental
conditions, and other factors can induce strain that can not be
easily distinguished from that due to bending [5, 6, 26]. Con-
tinual strain cycles can also cause material fatigue [22, 24, 36].
Most resistive strain sensors have high-latency and are unable
to measure the absolute angles of bend. The hysteresis prop-
erties of conductive materials produce varying conductivity
during cyclic loading [4, 22, 23, 24, 42, 44]. Most resistive and
FBG (Fiber Bragg Grating) sensors are non-linear in response
to large strains [24, 34].
Geometric Sensing
An alternative to strain sensing is what we call geometric sens-
ing. These sensors much more directly measure curvature by
sensing geometric changes that are a result of bending. Exam-
ples include [24, 29], which similar to ShArc, measure relative
displacements of different sensor layers. ShArc improves on
these earlier systems by supporting the direct detection of
multiple bends, which we will show to have superior error
performance.
While [29] also measures shift via a change in capacitance,
it uses an extended electrode pattern. If a sharp curve is
introduced in the middle of the pattern, only the later half of
the electrode pattern will be shifted. If the same sharp curve
is introduced before the pattern, the shift will happen over
the entire length, yielding a very different result. The issue
with this extended electrode design is that even with a single
bend the degree of bending and the placement of the bend
are conflated. Our differential electrode design is much more
compact and has superior noise and shielding performance. It
provides a linear response with shift and is largely immune to
misalignment in the direction orthogonal to shift.
ShArc sensors use multiple, thin spacers providing a more
consistent electrode spacing while maintaining high flexibility.
Because shift is distributed among many layers, the relative
movement between layers is reduced, increasing durability.
Multibend Sensors and Error Propagation
Most of the prior work uses sensors that give a single measure
of bend. In order to sense complex curves, one can employ a
series of single bend sensors [11, 21, 38, 40, 43], building a
model of connected joints. This works best when the underly-
ing thing to be sensed is well modelled as a series of linkages.
However, placement of the sensors requires a priori under-
standing of the joint locations. For example, when modelling
human joints such as a finger, there is significant variation in
location from person to person, precluding a general solution
[16, 35, 36].
Complex curves may require a large number of single bend
sensors to provide an adequate understanding of shape. Unfor-
tunately, each additional bend sensor contributes measurement
error which accumulates to progressively degrade the overall
accuracy of the system [41]. This severely limits the maxi-
mum number of single bend sensors that can reasonably be
employed. As we will show, ShArc sensors overcome this
limitation.
Recently, machine learning approaches have been applied to
understanding the output of systems with many single bend
sensors [6, 20, 33]. While these systems have the potential
to combine the readings from large numbers of single bend
sensors such that error does not accumulate in such a direct
fashion, they require extensive training. It is also unclear
if any reduction in accumulated error comes from imposing
constraints that make the system less general.
Specific Implementations
The most common strain-based bend sensors are resistive
[6, 11, 19, 21, 36, 38, 40], optical [3, 7, 16, 17, 18, 42,
45] including Fiber Bragg Grating (FBG) sensors [23, 44],
piezoelectric [12, 25] or capacitive [1, 13]. We consider each
of these, and discuss their operation.
Resistive bend sensors are similar to resistive strain gauges,
but are optimized for much larger bends. A layer of resis-
tive material is placed on a flexible substrate and undergoes
strain as the sensor is bent [4]. A bend away from the side
with the resistive material causes tensile strain, increasing the
resistance.
Paper 142
Page 2
CHI 2020 Paper
Resistive sensors suffer from significant drift due to fatigue,
aging of materials and environmental conditions, and require
constant re-calibration to achieve even modest accuracy [4, 24].
Because they provide only a single measure of bend, they can
not distinguish shape for complex curves. Although resistive
bend sensors have many limitations, they are quite inexpensive
and easy to interface to, allowing use in many applications [6,
11, 15, 19, 21, 27, 36, 38, 40, 43]. The best known of these is
the Mattel’s PowerGlove [39], an early consumer hand pose
interface device used for gaming on Nintendo systems. [21,
27, 38, 43] have embedded commercial flex sensors into both
soft and rigid materials to create different control interactions
like switches or sliders. [6, 11, 40] have used ink jet printing
on customized shapes to create game controllers and toys in
two and three dimensions.
Fiber Optic Shape Sensors (FOSS) can be constructed from
a flexible tube with reflective interior walls and a light trans-
mitter and receiver. [7, 17, 45] are examples of optical bend
sensors used as input devices in the HCI domain. FOSS re-
cover the bend shape by measuring changes in intensity, phase,
polarization or wavelength of the light while the flexible tube
is bent [3, 23, 24, 44, 42]. Fiber Bragg Grating sensors employ
an optical fiber that has been processed to create a grating
that interacts with light of a specific wavelength. As the fiber
is bent, the grating is mechanically expanded or compressed,
which shifts the wavelength of interest. A tunable laser is used
to scan for the new wavelength of the deformed grating. Dif-
ferent wavelength grating patterns can be placed at different
locations along the fiber, allowing the degree of bending to be
independently measured at each location [23, 24].
FOSS can be extremely thin and light weight with little re-
striction on the length of the sensor [24]. They are relatively
precise and immune to electromagnetic inference. While these
sensors provide impressive performance, it comes at a high
price. A tunable laser interrogator may cost as much as USD
$10,000 - a price that severely limits practical applications.
While the fiber can be quite thin, the interrogators tend to be
large and power hungry. They require sophisticated signal
processing [10, 14, 24, 30], complex fabrication processes
and calibration. They have a restricted range of measure-
ment for curvatures and fall into non-linearity very quickly
[3, 24].These properties limit their use cases to very specific
applications like high-end medical devices [23, 24, 28, 31].
Piezoelectric bend sensors are based on deformation and strain
in piezo materials. Such deformations change the surface
charge density of the material and cause charge transfer be-
tween the electrodes. The amplitude and frequency of the
signal is directly proportional to the applied mechanical stress
[25]. Piezoelectric sensors, similar to triboelectric sensors,
suffer from drift and only provide signal while in motion. This
limits their application to dynamic bending only and not static
or low-frequency deformations [12, 25, 33, 37].
Capacitive bend sensors work either by material strain [1, 2,
13, 32] or displacement between sensor layers [29]. Either way
the geometric changes vary the effective overlapping surface
areas for capacitive coupling and/or the spacing between con-
ductors as a function of the bending angle. Capacitive sensors
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
can be more linear than other techniques [2, 29]. They are
inexpensive to produce and more stable than resistive sensors.
Given these desirable properties, we chose to implement a
capacitive version of the ShArc measurement technique.
SHARC THEORY OF OPERATION
The basic operation of a ShArc sensor can be understood by
imagining a pair of measuring tapes of length
L
, separated by
a spacer of thickness
t
as shown in Figure 2. On one end,
the strips are joined together, much like the binding of a book.
In the flat orientation, it is easy to see that the markings on
the two tape measures should align. However, if the pair is
formed around a cylinder of radius
r
, the inner tape measure
will be formed into a circular arc of radius
r
, while the outer
tape measure will be formed into a circular arc of radius
r + t
.
Because they are conjoined on one end, the zero markings of
the two tape measures will still align, but the other markings
will get progressively misaligned. This is because it takes more
tape to subtend the same angle on a larger radius. Assuming
that the sensor is formed around a circular arc, we can calculate
the radius knowing only the spacing and the relative shift
between the tape measures. Relative shift can similarly be
measured at many points along the sensor, each allowing us
to measure the curvature of successive segments. In this way,
we can measure complex curves that are well modelled as a
series of circular arcs.
Figure 2: ShArc sensors can be understood by considering
two measuring tapes, bound together at one end, and held a
fixed distance apart. When flat, the markings align. When
bent, the markings become progressively more misaligned.
Constructing Arcs
A ShArc sensor, shown in Figure 3, consists of two strips - a
reference strip, and a sliding strip - conjoined at one end and
held apart by a spacer of thickness
t
. The goal is to acquire
the shape of the reference strip. At known intervals along the
reference strip,
Lr[n]
, it can measure the corresponding shifted
position,
Ls[n]
, along the sliding strip. By corresponding,
we mean that if you construct a normal to the curve of the
reference strip at the measurement point, you measure where
it intersects the sliding strip.
Figure 3: A ShArc sensor consists of a reference and a sliding
strip separated by a constant distance
t
and joined on one end.
Measurement points divide the unit into segments.
Paper 142
Page 3
CHI 2020 Paper
Single Arc
We first consider the simple case of a single circular arc seg-
ment. As shown in Figure 4, this segment (segment
n
) is
shaped into a circular arc of radius
r[n]
in a counter-clockwise
direction. Thus, the reference strip has radius
r[n]
while the
sliding strip is inside with a smaller radius of
r[n] t
. We
define the starting angle
θ [n]
, which is the tangent at the be-
ginning of the arc. The ending angle is the tangent to the arc
at its end, θ [n + 1].
Figure 4: Definitions for segment n
We define the length of the reference strip in segment n as:
ΔLr[n] = Lr[n+ 1] Lr[n] (1)
and the length of the corresponding sliding strip as:
ΔLs[n] = Ls[n+ 1] Ls[n] (2)
Similarly, we define the total subtended angle of this segment
as:
Δθ [n] = θ [n + 1] θ [n] (3)
For the case of positive curvature (
r[n] > 0
and
Δθ[n] > 0
), the
sliding strip is shaped into a tighter curve than the reference
strip. Thus,
ΔLs[n] < ΔLr[n]
, even though they subtend the
same angle, Δθ [n].
Working in radians, the length of the reference segment is:
ΔLr[n] = r[n]Δθ [n] (4)
and the length of the corresponding sliding segment is:
ΔLs[n] = (r[n] t)Δθ[n] (5)
Given the two lengths and the spacer thickness, we can solve
for the radius of curvature of this segment:
tΔLr[n]
r[n] = (6)
ΔLr[n] ΔLs[n]
This same equation applies when the curve proceeds clock-
wise, giving a more negative ending angle and negative radius
of curvature.
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
We can also solve for the subtended angle of the arc:
ΔLr[n] ΔLs[n]
Δθ [n] = (7)
t
Multiple Arcs
Applying equations 6 and 7 provides a series of circular arcs of
known length, angular extent, and radius of curvature. These
must be pieced together to model the complete reference strip
curve. Unlike jointed systems, ShArc sensors have continuous
flexure along their length. They are inherently continuous in
their first derivative. To maintain a continuous first derivative
from segment to segment, we require that the tangents of
adjoining segments match. To put this another way, the ending
angle of each segment matches the starting angle of the next
segment.
Consider a single arc as shown in Figure 5. The arc begins
at a known starting point,
(x[n],y[n])
, and at an initial known
angle of
θ [n]
and proceeds to an unknown ending point,
(x[n+
1],y[n+ 1]), at an unknown ending angle of θ[n+ 1].
Figure 5: Definitions for calculating the position of a segment.
The change in angle from starting point to the ending point is
just the turning of the segment angle, Δθ [n].
To find the x, y translation, we find the increment in x and
y over the arc and add this to the previous point. For conve-
nience, we imagine that the center of the radius of curvature
of the arc is at the origin and calculate the change in endpoint
positions. This difference is then applied to the known starting
point.
For this calculation, we need to know the angles from the
center that form the arc. A normal angle to
θ [n]
is
θ[n]
π
2
. For an arc of positive radius of curvature, this gives the
angle pointing out from the center of the radius of curvature.
If the radius of curvature is negative, it points the opposite
direction. This results in a sign flip that is corrected by using
the signed radius of curvature. The endpoints can then be
found iteratively via equations 8 and 9:
x[n + 1] =
x[n] + r[n]cos(θ [n + 1] π ) r[n]cos(φ[n]
2
π )
2
(8)
y[n + 1] =
y[n] + r[n]sin(θ[n+ 1] π ) r[n]sin(φ[n]
2
π )
2
(9)
Paper 142
Page 4
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
These equations can be slightly simplified using trig identities.
x[n + 1] = x[n] + r[n] (sin(θ[n + 1]) sin(θ[n])) (10)
y[n+ 1] = y[n] + r[n] (cos(θ[n] ) cos(θ [n + 1])) (11)
These equations describe the series of circular arcs that model
the bend. A circular arc is typically described by its center
(
Cx[n], Cy[n]
), its radius of curvature
r[n]
, a starting angle
θ [n]
,
and an angular extent
Δθ[n]
. The center of an arc segment can
be found by starting at (x[n],y[n]), and following the radius to
the arc center
(Cx[n], Cy[n])
. The starting angle is found from
the normal at the point
(x[n], y[n])
, which is
(θ [n] π/2)
.
The center is then:
π
Cx[n] = x[n] + r[n]cos(θ [n] )
2 (12)
= x[n] + r[n]sin(θ [n])
π
Cy[n] = y[n] + r[n]sin(θ [n] )
2 (13)
= y[n] r[n]cos(θ[n])
Note that the use of the signed radius of curvature ensures that
we are following the normal to the center.
The starting angle is:
π
(θ [n] )sign(r[n]) (14)
2
The sign is needed to flip the angle if the arc proceeds clock-
wise. The extent of the arc is
θr[n]
, which is also a signed
value.
Error Propagation Properties
On multi-axis robot arms, position is usually determined via
a series of encoders - one on each joint. High precision en-
coders are typically required because any errors in each joint
measurement accumulate. For a planar arm, the angular error
at the end of the arm is simply the sum of all the measurement
errors in each joint. The location error of the endpoint is also
wildly impacted by all of the joint errors - particularly the ones
at the beginning of the arm.
In stark contrast, ShArc sensors have far more benign error
propagation properties. These arise from the fact that measure-
ment errors for each arc are NOT independent.
Consider the case of a ShArc sensor with two measurement
points. To find the curvature of the first segment, we determine
the relative shift at the first measurement point. Let us presume
that this measurement is corrupted by noise, and our reading
of shift at this point is incorrectly low. Next we measure
the relative shift in the second segment. This measurement is
made by taking the total shift at the second measurement point,
and subtracting off the shift from the first measurement point.
The error at the first point will now cause a corresponding
error in the second segment that is opposite in sign from the
error in the first segment. Thus, the two segments will end
up with curvature errors that tend to cancel each other out. In
fact, we will show that the error in final angle is completely
unimpacted by the error at the first measurement point.
We define the starting point of the curve as:
x[0] = 0
y[0] = 0 (15)
θ [0] = 0
By definition,
Lr[0] = 0
and
Ls[0] = 0
. We can now calculate
the ending angle of segment 0:
θ [1] = Δθ[0] + θ [0]
= Δθ[0]
ΔLr[0] ΔLs[0] (16)
= t
Lr[1] Ls[1]
= t
Next, we find the ending angle of segment 1.
θ [2] = Δθ[1] + θ [1]
ΔLr[1] ΔLs[1] Lr[1] Ls[1]
= + (17)
t t
Lr[2] Ls[2]
= t
As can be seen, the ending angle calculation has
NO
depen-
¯
dence on any earlier measurements. This means that any errors
in earlier measurements do not contribute error in the ending
angle of each segment.
This property of ShArc sensors provides an important advan-
tage over traditional solutions which string together a series
of angular encoders. To accurately model a complex curve,
many segments will be required. But the more angular en-
coders one adds to a system, the more angular error there will
be. This places a practical limit on the number of encoders
that can be strung together. In sharp contrast, ShArc sensors
do not incur additional angular error when adding more mea-
surement points. This makes ShArc sensors an ideal fit for
measuring highly complex curves that require many segments
for accurate modelling.
It should also be noted that these error propagation characteris-
tics also make ShArc sensors highly robust in the face of local
measurement errors. We have found that even relatively noisy
measurement data tends to produce overall curves that match
reality surprisingly well.
THE SHARC SENSOR PROTOTYPE
In order to validate the ShArc sensing technique, we con-
structed the prototype device shown in Figure 1. ShArc sen-
sors measure the relative shift between two flexible layers that
are held a fixed distance apart. While there are many ways to
craft such a system, we sought a design that would be easy
to implement, provide reasonable precision, and allow for
continuous flexure over the length of the device.
Paper 142
Page 5
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 6: Detailed layout of the sensor strips - units are in millimeters.
Our design was inspired by digital calipers that employ capac-
itive sensing to determine position. Baxter [8, 9] describes
the widely adopted technique wherein a pattern of transmit
electrodes moves along a corresponding pattern of receive
electrodes. Position is determined by examining the change in
coupling capacitance between the transmit and receive elec-
trodes.
Our prototype device leverages standard flexible printed cir-
cuit board technology to create transmit and receive strips with
precise electrode patterns. These are shown in Figure 6. The
transmit strip has 8 equally spaced electrodes which align with
8 differential electrode pairs on the receive strip. When the
strips are flat, each transmit electrode will be centered over a
receive pair such that the differential capacitance is zero. As
the two strips shift relative to each other, the transmit pads
will move out of alignment with the receive pads, unbalanc-
ing the differential capacitance. The electrodes have been
designed to have significant overlap to minimize the impact of
skew and fringing fields, giving a linear change in differential
capacitance with respect to shift.
To keep the transmit and receive pads at a fixed spacing, we
interpose a series of polyimide strips. The amount of shift is
proportional to the thickness of the spacing, so we choose a
reasonable value of 0.5mm. While this spacing could have
been achieved with a single 0.5mm spacer, it would result in
a fairly stiff sensor. More importantly, as the bending radius
starts to approach the thickness of the spacer, one would expect
significant stress deformations, creating uneven spacing. To
solve this issue, we use 5 layers of 0.1mm spacers, which
yields a device that is quite pliable while maintaining accurate
spacing. Another advantage of using many thin spacers is
that the shifting is spread among the layers. The relative shift
between any two layers is extremely small, minimizing surface
wear.
The strips are held pressed together via a Spandex sleeve,
while still allowing them to shift against each other along the
length. A clamp passes through alignment holes on the strips
to constrain motion on that end. Gold finger contacts allow
the strips to be insert into connectors on opposite sides of the
controller board.
Figure 7 shows how the electrodes shift as the sensor is flexed.
The transmit pads are shown in green, and the differential
receive pads are shown in blue and red respectively. When
flat, the transmit pads are centered under the receive pads.
If the sensor is formed into a circular arc, the pads become
increasingly misaligned.
We use a single channel, 24-bit differential capacitance to dig-
ital converter (Analog Devices AD7745/AD7746) to perform
the capacitance measurements. Using a series of ultra-low
capacitance multiplexers (Texas Instruments TMUX1511), we
can successively measure shift at 8 points along our strips. The
capacitances we are measuring are sub-pico farad, and there
are substantial parasitic capacitances due to the proximity of
various traces. When the sensor is laid flat, we measure the
static impact of these parasitic capacitances and subtract this
value off of later readings to find the differential capacitance
due to the electrodes.
The current circuit has not been optimized for speed or power.
It can do a full sweep of the sensor about 10 times per second,
while drawing less than 100mW. Both of these specifications
could be substantially improved with modest effort.
EVALUATION AND RESULT
In this section, we consider both the theoretical and actual
performance of the sensor.
We used a simple parallel plate model to calculate the theoret-
ical change in differential capacitance as a function of shift.
Assuming a dielectric constant
k
of 3.5 for polyimide, our
geometry should yield a sensitivity of 0.062pF /mm of shift.
As shown in Figure 7, the transmit pads are designed to lay
centered on the corresponding receive pad pairs when the
sensor is laid flat. As the sensor is bent, the receive and
transmit pads misalign. As shown in Figure 7c, the shift can
only go
+/ 3mm
before the transmit pad extends beyond the
corresponding receive pads. This limits the maximum bend
Paper 142
Page 6
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
Figure 7: The relationship between the relative shift and differential capacitance. a. Transmit electrodes are centered between two
receive electrodes when sensor is laying flat, giving zero differential capacitance. b. When sensor is bent, the transmit electrodes
overlap one receive electrode more than the other, creating a non-zero differential capacitance. The change in capacitance indicates
the degree of shift. C. A top and side view of the overlap between transmit and receive electrodes when the differential capacitance
is minimum and maximum.
that the sensor can measure. As we have shown, the shift
at any point is only a function of the ending angle. For any
segment n:
Lr[n] Ls[n]
θ [n] = (18)
t
With a shift of
+/ 3mm
and a thickness of
0.5mm
, our max-
imum ending angle is
6 radians
, or about
244
. The ending
angle at any measurement point should not exceed this, and
to ensure good linearity, needs to be somewhat less. If more
range is required, the spacer layer can be decreased, creat-
ing less shift for a given curvature. The trade off is that this
creates a corresponding loss of resolution. Alternatively, the
electrodes can also be designed to accommodate more shift.
It should be noted that this constraint does not limit the number
of bends. For example, if the sensor was formed into a sinu-
soid, the shift would cyclically rise and fall, returning to zero
at the end of each cycle. (If the amplitude was high enough,
our maximum ending angle could be exceeded at some points,
but this is not dependent on the number of bends.) However,
if the sensor is formed into a single circular arc as in Figure
7a, the shift continues to linearly accumulate. If the sensor is
wrapped too tightly, it will exceed the allowable range.
If the maximum angle is grossly exceeded, the transmit pad
may completely move out of range of the receive pads, giving
a differential capacitance of zero, which will report as flat.
Interestingly, if the max angle is exceeded, but a compensating
bend leaves the end of the strip less than the max angle, those
later angles in the end piece will report correctly.
There is a numerical issue with using bend radius to describe
essentially flat curves, where the bend radius goes to infinity.
The sensor works quite well in reporting a flat curve, and
there is no upper limit on radius of curvature. Because any
sufficiently large radius is essentially flat, it makes little sense
to consider the absolute radius error in these cases. This is
why we chose to limit our measurement to a radius of 200mm.
If a curve can not be adequately described by the number of
arcs available (e.g. when the number of bends exceeds the
number of arcs), the failure mode of the sensor is relatively
benign. So long as the sensor is still physically intact (i.e. that
the strips are still held at a constant distance apart), the shift at
any point will still reflect the total angle at that point, and this
will be read correctly at each measurement point. This means
that while some detail is lost, the understanding of overall
low-spatial frequency shape will typically be pretty good, but
could have confusing artifacts. It is fairly analogous to what
happens when a waveform is sampled at less than the Nyquist
rate.
To test the performance of our device, we created a number of
circular arc test forms of known radii (Figure 9-a) onto which
the sensor could be placed. We chose to test a range of radii
from
40 mm
to
600 mm
, which given the length of our sensor,
covers a reasonable range.
We repeatedly placed the sensor onto the test forms (Figure 9-
b), collected data, and calculated the radius for each segment.
The results are shown in Table 1 and Figure 10. There are
a number of things to note about this data. First, the sensor
did an excellent job of estimating the radii for values below
200mm, staying within 2% of the true value. While the error
appears to increase for very large radii, it should be understood
Paper 142
Page 7
CHI 2020 Paper
that these curves are almost flat, with very slight variations
in curvature (normally defined as
1/r
). So these seemingly
larger errors actually have little impact on the accuracy of the
curve reconstruction.
Figure 8 shows example curve reconstructions when the sensor
is placed on forms with radii of curvature ranging from 90mm
to 200mm along with their ideal curves. Despite the presence
of measurement errors, the curves closely track the ideal val-
ues. This is largely due to the error propagation properties
of ShArc sensors. Looking closely at the graphs you can see
that when a measured curve starts to drift off of the ideal, later
segments help pull it back on. Again, this is in sharp contrast
to systems that independently encode a series of joints.
Figure 8: Example sensor data taken on forms ranging in radii
from 9 to 20 cm. The dashed lines represent the ideal curves.
Sources of Error
In order to improve the sensor, it is important to understand
what the major sources of error are in the system.
Systematic sources of error:
As noted previously, even when the transmit and receive pads
are aligned in the flat orientation, there are parasitic capac-
itances due to the asymmetric layout that cause a non-zero
differential capacitance. Since these are static, they can be
measured in the flat position, and then subtracted off of all
future measurements.
Our understanding of permativity and spacer thickness are
relatively imprecise. Sheet material is typically specified with
a +/-5 % thickness tolerance. There may also be small errors
in the sizing of the electrodes and the gain of the converter.
All of these factors result in a change in the sensitivity, and
can be compensated for with a single constant. In practice, we
find this constant by adjusting it until there is a good fit to a
known curve.
While we ignored fringing fields in our analysis, they are a sig-
nificant source of non-linearity, particularly when approaching
the shift limit. It would be straightforward to characterize this
non-linearity and compensate for it.
Random sources of error:
We observed two major types of random errors in system.
Reference
Radii(mm)
40
50
60
70
80
90
100
125
150
175
200
300
400
500
600
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
CV
-0.39
0.35
-0.12
-0.03
0.11
1.97
0.49
-0.23
0.24
-1.03
-0.59
0.15
0.12
0.05
0.02
Constructed
Radii (mm)
40.23
49.85
60.82
70.76
79.01
89.90
99.08
126.16
147.43
175.35
201.58
285.03
380.83
404.60
453.69
Error (%)
-0.58 0.22
0.29 0.10
-1.37 0.15
-1.07 0.03
1.25 0.13
0.15 0.30
1.11 0.55
-0.92 0.27
1.80 0.42
-0.17 0.18
-0.60 0.35
4.98 0.73
4.79 0.56
19.07 1.04
24.38 0.46
STD
Table 1: Measurement Results
Electrical noise limits the practical resolution of the capaci-
tance to digital converter. More significantly, we saw issues
with mechanical repeatability.
To investigate electrical noise, we conducted a simple exper-
iment in which we laid the sensor flat and read the output
for 500 counts. The maximum observed deviation for all 8
channels was 0.793 fF. We then lifted the sensor, and returned
it to the flat position ve times. Under these conditions we
observed a maximum change of 7.221fF - about an order of
magnitude worse than the electrical noise floor. We are in-
vestigating the cause of this lack of mechanical repeatability,
and suspect that it is largely due to insufficient restoring force
being provided by the elastic sleeve to keep the layers tightly
pressed together.
Calibration
The relationship between curvature and differential capaci-
tance is determined by geometry and the dielectric constant
of the spacer. Given the accuracy of the electrode patterns,
most of the variation from device to device is caused by the
small changes in spacer material thickness. We have found
that this variation is small enough as to be negligible for most
applications, and that a single constant can be used for all
devices of the same design. The parasitic capacitances are
more impacted by the precise placement of components on the
PC board, and since these are currently assembled by hand, it
makes sense to subtract off a unique baseline for each device.
So no per device calibration is required - just the baseline
subtraction.
For increased accuracy, one could do a more detailed measure-
ment of the change in capacitance with curvature, and model
the small non-linearity due to fringing fields. We did not do
this because the device was surprisingly accurate without this
step. But we could imagine that demanding applications might
benefit from a model of this non-ideality.
APPLICATIONS AND INTERACTION TECHNIQUES
Gesture Input Devices
A common application for resistive bend sensors has been
finger tracking on glove input devices. Having only a mea-
surement of gross flexure, these provided a crude estimate of
finger position. Even that requires a model of finger structure
Paper 142
Page 8
CHI 2020 Paper
Figure 9: a. Calibration frames with reference radii from 4.0
cm to 60.0 cm, b. Placing the sensor on to the test forms to
measure the known radii.
Figure 10: Top: Constructed vs reference radii
(mm/mm)
in
green line and the ideal characteristic curve in blue dashed
line. The red dots show the variation of measured data, arises
from random noise. Bottom: Measurement errors
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
and specific joint locations which vary dramatically among
different users. ShArc sensors can overcome these limitations.
To demonstrate this, we attached the prototype sensor to a
user’s wrist and index finger as shown in Figure 11. We
then used the resulting data to drive a Unity model, updating
joint positions in real-time. The sensor reports an 8 segment
shape model over its length, which is adequate resolution
for mapping the deformation data onto the wrist and finger
joint positions. This provides accurate and continuous motion
tracking, enabling in-air gesture control of user interfaces for
navigation, selection, hover, pressing, scrolling, etc.
Posture Monitoring
ShArc sensors are ideal for many health applications where
a detailed understanding of body motion is desired. Because
they are precise, low-cost, low power, light weight and slim
in form factor, they can easily be incorporated into patient
wearable systems.
As a simple demonstration, we constructed a long ShArc sen-
sor, shown in Figure 12, about half a meter long, 24mm wide
and about 1mm thick. The electrode patterns are the same as
used in the other prototype, but spread out to cover the longer
distance. We attached the sensor to a compression garment
using Velcro strips so that it tracks the motion of the spine.
Again, we implemented a simple posture monitoring applica-
tion in Unity (Figure 13) to visualize the range of motion of
the spine.
Figure 12: ShArc sensor built in two different size
We discovered several issues in using this second prototype
for skeletal tracking. First, with only 8 segments in our model
there is not enough resolution to track all of the vertebrae
independently. Second, our Unity application did not have
a detailed spine model limiting the accuracy of our model.
Third, better integration of the sensor into a wearable form is
required. A smaller, wireless version would be vastly preferred
than the needlessly bulky, tethered prototype.
Figure 11: ShArc sensor as a gesture input device, allows
tracking joints in the wrist and index finger. Figure 13: Using a ShArc sensor to track spine curvature
Paper 142
Page 9
CHI 2020 Paper
Angular Ruler
While it is easy to measure rectilinear objects, many aestheti-
cally designed things feature free-form curves which are diffi-
cult to characterize. ShArc can act as a simple angular ruler
which reports the precise shape of such geometries. As Figure
14 shows, we have designed an interactive user interface so
that a user can click on any point along the curve and collect
detailed information about the radii of the curvatures and the
bend degree at that specific point.
Figure 14: A ShArc sensor used as an angular ruler. An
interactive user interface allows real time collection of curve
data.
DISCUSSION
When we first conceived the idea of measuring relative shift
between two strips to measure complex curves, we recognized
that measurement error on one segment would cause a com-
pensating error on the other direction on the next. As we
have shown, both theoretically and in practice, this allows for
excellent performance, even in the face of significant mea-
surement errors. This is a startling result and it is why ShArc
techniques should be preferred over joint encoding for curves
with significant complexity.
Our capacitive prototype demonstrates that ShArc sensors with
good performance can be easily produced. However, this is
clearly a first generation implementation, with much room for
improvement in the future.
Mechanical Issues:
Mechanical repeatability was seen to be a major cause of error.
When replaced on the same form, we would often see change
in segment measurements from the prior measurement (even
if the overall curve fit was good). We note that the elastic
sleeve mostly provides force on the edges of the strips rather
than the faces. Very small changes in layer separation can
cause significant errors, effectively scaling the differential
capacitance. An improved mechanical design should address
this issue. This can also be addressed electrically by switching
from a differential capacitance measurement to a ratiometric
one.
No special effort was made to use materials with high flatness.
Thin sheets are often specified with +/-5% thickness variation.
We made no effort to calibrate for thickness variations along
the spacer strips.
Modelling Issues:
As presented, ShArc sensors describe curves using a series
of circular arcs. However, not all curves are well modelled
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
by a small number of arcs. Higher order models may provide
significant advantages for some geometries.
Our implementation uses time division multiplexing, making
each shift measurement serially in time. Thus, a single curve
is constructed from data taken at different times. This causes
a dynamic distortion. In future versions, we could time align
the data in software, or preferably, use a circuit that does true
simultaneous measurement at all points.
Shielding:
Proper shielding of the electrodes will improve tolerance to
handling and the proximity of nearby conductors. While elec-
trical noise has not limited current performance, it may in the
future. Shielding will help.
Extension to multiple dimensions:
The current ShArc sensor measures the bend in one plane.
However, the underlying technique of measuring relative shifts
can be extended to three dimensions.
CONCLUSION
In this paper, we introduced ShArc, a precision, geometric
measurement technique to sense the shape of a dynamically
deforming object. ShArc devices are able to sense complex
curves which are modelled as a series of connected, circular
arcs. We outlined the operating principle of measuring relative
shift between inner and outer layers of the sensor at many
points and showed the theoretical tolerance to measurement
errors. A practical capacitive implementation was described,
and its performance characterized. Compared to traditional
bend sensors, ShArc sensors are inexpensive, precise and do
not suffer from drift of strain characteristics. This makes them
ideal for a number of applications.
ACKNOWLEDGEMENT
We would like to thank all the reviewers for their detailed
feedback. We also thank Steven Sanders and our colleagues at
Tactual Labs for their support of this project. Special thanks
go to Cathy Dietz, who created the sensor sleeves and Alex
Dietz for lending his voice to the accompanying video.
REFERENCES
[1] J Aebersold, K Walsh, M Crain, M Voor, M Martin, W
Hnat, J Lin, D Jackson, and J Naber. 2006. Design,
modeling, fabrication and testing of a MEMS capacitive
bending strain sensor. In Journal of Physics: Conference
Series, Vol. 34. IOP Publishing, 124.
[2] Julia W Aebersold, Kevin Walsh, Mark Crain, and
Michael Voor. 2010. MEMS capacitive bending and
axial strain sensor. (Dec. 21 2010). US Patent 7,854,174.
[3] Moe Amanzadeh, Saiied M Aminossadati, Mehmet S
Kizil, and Aleksandar D Raki´
c. 2018. Recent
developments in fibre optic shape sensing. Measurement
128 (2018), 119–137.
[4] Morteza Amjadi, Ki-Uk Kyung, Inkyu Park, and Metin
Sitti. 2016. Stretchable, skin-mountable, and wearable
strain sensors and their potential applications: a review.
Advanced Functional Materials 26, 11 (2016),
1678–1698.
Paper 142
Page 10
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[5] Asli Atalay, Vanessa Sanchez, Ozgur Atalay, Daniel M
Vogt, Florian Haufe, Robert J Wood, and Conor J Walsh.
2017. Batch fabrication of customizable silicone-textile
composite capacitive strain sensors for human motion
tracking. Advanced Materials Technologies 2, 9 (2017),
1700136.
[6] Moritz Bächer, Benjamin Hepp, Fabrizio Pece, Paul G
Kry, Bernd Bickel, Bernhard Thomaszewski, and Otmar
Hilliges. 2016. Defsense: Computational design of
customized deformable input devices. In Proceedings of
the 2016 CHI Conference on Human Factors in
Computing Systems. ACM, 3806–3816.
[7] Ravin Balakrishnan, George Fitzmaurice, Gordon
Kurtenbach, and Karan Singh. 1999. Exploring
interactive curve and surface manipulation using a bend
and twist sensitive input strip. SI3D 99 (1999), 111–118.
[8] Larry K Baxter. 1997. Capacitive sensors. Design and
Applications (1997).
[9] Larry K Baxter and Robert J Buehler. 1986. Capacitive
displacement measuring instrument. (May 6 1986). US
Patent 4,586,260.
[10] Shaohua Chen, Zhengrong Tong, Qida Zhao, Zhiguo
Liu, and Xiaoyi Dong. 2004. A smart bending sensor
with a novel temperature-and strain-insensitive
long-period grating. Sensors and actuators A: Physical
116, 1 (2004), 103–106.
[11] Chin-yu Chien, Rong-Hao Liang, Long-Fei Lin, Liwei
Chan, and Bing-Yu Chen. 2015. Flexibend: Enabling
interactivity of multi-part, deformable fabrications using
single shape-sensing strip. In Proceedings of the 28th
Annual ACM Symposium on User Interface Software &
Technology. Citeseer, 659–663.
[12]
Sung Yun Chung, Hwa-Jin Lee, Duck-Jae You, Sanghun
Cho, Buil Nam, Tae Il Lee, and Youn Sang Kim. 2016.
Piezoelectric Bending Motion Sensor for Simultaneous
Detection of Bending Curvature and Speed. ECS
Transactions 75, 16 (2016), 277–279.
[13] Darryl Cotton. 2018. Apparatus having touch and
bending sensing functions. (Nov. 6 2018). US Patent
App. 10/120,483.
[14] Yanyan Dong, Yanfang Lu, Changyu Shen, Jinlei Chu,
Xin Zou, Chuan Zhong, and Xinyong Dong. 2013.
Novel bending sensor based on a meniscus shaped beam
with LPG. Optik 124, 24 (2013), 6737–6739.
[15]
Khaled Elgeneidy, Gerhard Neumann, Michael Jackson,
and Niels Lohse. 2018. Directly printable flexible strain
sensors for bending and contact feedback of soft
actuators. Frontiers in Robotics and AI 5 (2018), 2.
[16]
Abu Ilius Faisal, Sumit Majumder, Tapas Mondal, David
Cowan, Sasan Naseh, and M Jamal Deen. 2019.
Monitoring Methods of Human Body Joints:
State-of-the-Art and Research Challenges. Sensors 19,
11 (2019), 2629.
[17] Nicholas Fellion, Thomas Pietrzak, and Audrey
Girouard. 2017. FlexStylus: Leveraging bend input for
pen interaction. In Proceedings of the 30th Annual ACM
Symposium on User Interface Software and Technology.
ACM, 375–385.
[18] Kevin C Galloway, Yue Chen, Emily Templeton, Brian
Rife, Isuru S Godage, and Eric J Barth. 2019. Fiber
Optic Shape Sensing for Soft Robotics. Soft robotics
(2019).
[19] Giada Gerboni, Alessandro Diodato, Gastone Ciuti,
Matteo Cianchetti, and Arianna Menciassi. 2017.
Feedback control of soft robot actuators via commercial
flex bend sensors. IEEE/ASME Transactions on
Mechatronics 22, 4 (2017), 1881–1888.
[20] Oliver Glauser, Shihao Wu, Daniele Panozzo, Otmar
Hilliges, and Olga Sorkine-Hornung. 2019. Interactive
hand pose estimation using a stretch-sensing soft glove.
ACM Transactions on Graphics (TOG) 38, 4 (2019), 41.
[21] Antonio Gomes, Andrea Nesbitt, and Roel Vertegaal.
2013. MorePhone: a study of actuated shape
deformations for flexible thin-film smartphone
notifications. In Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems. ACM,
583–592.
[22] RL Hollingshead, L Henry-Etesse, E Tankere, D
Kamper, and T Tan. 2017. Characterization of hysteresis
in resistive bend sensors. In 2017 International
Symposium on Wearable Robotics and Rehabilitation
(WeRob). IEEE, 1–2.
[23] Minsu Jang, Jun Sik Kim, Soong Ho Um, Sungwook
Yang, and Jinseok Kim. 2019. Ultra-high curvature
sensors for multi-bend structures using fiber Bragg
gratings. Optics express 27, 3 (2019), 2074–2084.
[24] Useok Jeong and Kyu-Jin Cho. 2016. A novel low-cost,
large curvature bend sensor based on a Bowden-cable.
Sensors 16, 7 (2016), 961.
[25] Takashi Kato, Akio Yamamoto, and Toshiro Higuchi.
2003. Shape recognition using piezoelectric thin films.
In IEEE International Conference on Industrial
Technology, 2003, Vol. 1. IEEE, 112–116.
[26] Sean Jason Keller, Tristan Thomas Trutna, David R
Perek, and Bruce A Cleary III. 2018.
Resistive-capacitive deformation sensor. (Sept. 4 2018).
US Patent App. 10/067,007.
[27] Johan Kildal, Susanna Paasovaara, and Viljakaisa
Aaltonen. 2012. Kinetic device: designing interactions
with a deformable mobile interface. In CHI’12 Extended
Abstracts on Human Factors in Computing Systems.
ACM, 1871–1876.
[28] Vandana Mishra, Nahar Singh, DV Rai, Umesh Tiwari,
GC Poddar, SC Jain, SK Mondal, and Pawan Kapur.
2010. Fiber Bragg grating sensor for monitoring bone
decalcification. Orthopaedics & Traumatology: Surgery
& Research 96, 6 (2010), 646–651.
Paper 142
Page 11
CHI 2020 Paper
CHI 2020, April 25–30, 2020, Honolulu, HI, USA
[29] James S Neely and Phillip J Restle. 1997. Capacitive
bend sensor. (March 11 1997). US Patent 5,610,528.
[30] Yanhui Qi, Lin Ma, Jiang Sun, Zexin Kang, Yunlong
Bai, and Shuisheng Jian. 2015. Highly sensitive bending
sensor based on multimode–multimode–coreoffset fiber
structure. Optics & Laser Technology 75 (2015), 52–56.
[31] Ginu Rajan and B Gangadhara Prusty. 2016. Structural
health monitoring of composite structures using fiber
optic methods. CRC press.
[32] Shawn P Reese. 2015. Angular displacement sensor of
compliant material. (Dec. 29 2015). US Patent
9,222,764.
[33] Christian Rendl, David Kim, Sean Fanello, Patrick
Parzer, Christoph Rhemann, Jonathan Taylor, Martin
Zirkl, Gregor Scheipl, Thomas Rothländer, Michael
Haller, and others. 2014. FlexSense: a transparent
self-sensing deformable surface. In Proceedings of the
27th annual ACM symposium on User interface software
and technology. ACM, 129–138.
[34] Giovanni Saggio and Giancarlo Orengo. 2018. Flex
sensor characterization against shape and curvature
changes. Sensors and Actuators A: Physical 273 (2018),
221–231.
[35]
Giovanni Saggio, Antonio Pallotti, Laura Sbernini, Vito
Errico, and Franco Di Paolo. 2016. Feasibility of
Commercial Resistive Flex Sensors for Hand Tracking
Applications. Sensors & Transducers 201, 6 (2016), 17.
[36] Giovanni Saggio, Francesco Riillo, Laura Sbernini, and
Lucia Rita Quitadamo. 2015. Resistive flex sensors: a
survey. Smart Materials and Structures 25, 1 (2015),
013001.
[37] Fereshteh Shahmiri, Chaoyu Chen, Anandghan
Waghmare, Dingtian Zhang, Shivan Mittal, Steven L
Zhang, Yi-Cheng Wang, Zhong Lin Wang, Thad E
Starner, and Gregory D Abowd. 2019. Serpentine: A
Self-Powered Reversibly Deformable Cord Sensor for
Human Input. In Proceedings of the 2019 CHI
Conference on Human Factors in Computing Systems.
ACM, 545.
[38]
Ronit Slyper, Ivan Poupyrev, and Jessica Hodgins. 2011.
Sensing through structure: designing soft silicone
sensors. In Proceedings of the fifth international
conference on Tangible, embedded, and embodied
interaction. ACM, 213–220.
[39] David J Sturman and David Zeltzer. 1994. A survey of
glove-based input. IEEE Computer graphics and
Applications 14, 1 (1994), 30–39.
[40] Nirzaree Vadgama and Jürgen Steimle. 2017. Flexy:
Shape-customizable, single-layer, inkjet printable
patterns for 1d and 2d flex sensing. In Proceedings of
the Eleventh International Conference on Tangible,
Embedded, and Embodied Interaction. ACM, 153–162.
[41] Sean P Walker, June Park, Neal A Tanner, and others.
2016. Reducing incremental measurement sensor error.
(March 22 2016). US Patent 9,289,578.
[42] Lefan Wang, Turgut Meydan, and Paul Williams. 2017.
A two-axis goniometric sensor for tracking finger
motion. Sensors 17, 4 (2017), 770.
[43] Kristen Warren, Jessica Lo, Vaibhav Vadgama, and
Audrey Girouard. 2013. Bending the rules: bend gesture
classification for flexible displays. In Proceedings of the
SIGCHI Conference on Human Factors in Computing
Systems. ACM, 607–610.
[44] Ran Xu, Aaron Yurkewich, and Rajni V Patel. 2016.
Curvature, torsion, and force sensing in continuum
robots using helically wrapped FBG sensors. IEEE
Robotics and Automation Letters 1, 2 (2016),
1052–1059.
[45] Thomas G Zimmerman. 1985. Optical flex sensor.
(Sept. 17 1985). US Patent 4,542,291.
Paper 142
Page 12
... The shape perception of continuum manipulators is a promising approach to enhancing positioning precision. At present, research on detection methods mainly includes using optical, 7-10 electrical, [11][12][13][14][15][16][17][18] and magnetic [19][20][21][22] signal sensors to perceive the body shape. In terms of fiber-optic sensing, Van Meerbeek et al. 7 embedded fiber-optic sensors into silicone foam bodies to endow the robot with body perception capabilities through the intensity of optical signals. ...
... The overall structure of the manipulator, which comprises three individual units, is illustrated in Figure 9a, with a total length of 280 mm. The algorithm for perceiving shape of the manipulator involves first computing the joint space variables (Algorithm 1) for each unit, followed by utilizing Equations (15) and (16) to obtain the end-position coordinates. We have devised a sophisticated trajectory (Supplementary Fig. S4; Supplementary Video S1) that incorporates compression, bending, and rotation to scrutinize the accuracy of the predictions made by the piecewise constant curvature model. ...
... Another reason for affecting the position accuracy is that when using the constant curvature model to calculate the coordinates of the endpoints of each segment, only the position in the transformation matrix [Eq. (15)] is focused on. ...
Article
Continuum manipulators can improve spatial adaptability and operational flexibility in constrained environments by endowing them with contraction and extension capabilities. There are currently desired requirements to quantify the shape of an extensible continuum manipulator for strengthening its obstacle avoidance capability and end-effector position accuracy. To address these issues, this study proposes a methodology of using silicone rubber strain sensors (SRSS) to estimate the shape of an extensible continuum manipulator. The way is to measure the strain at specific locations on the deformable body of the manipulator, and then reconstruct the shape by integrating the information from all sensors. The slender sensors are fabricated by a rolling process that transforms planar silicone rubber sensors into cylindrical structures. The proprioceptive model relationship between the strain of the sensor and the deformation of the manipulator is established with considering the phenomenon of torsion of the manipulator caused by compression. The physically extensible continuum manipulator equipped with three driving tendons and nine SRSS was designed. Comprehensive evaluations of various motion trajectories indicate that this method can accurately reconstruct the shape of the manipulator, especially under end-effector loads. The experimental results demonstrate that the mean (maximum) absolute position error of the endpoint is 1.61% (3.45%) of the manipulator length.
... An alternative solution is to use Bezier curves, where a few control points mathematically define a contour. Solutions like ShArc [70] can capture data from these control points with less effort, creating a suitable solution to replicate organic shapes parametrically. The use of augmented reality could also help to capture control points to create Bezier curves. ...
Preprint
Full-text available
Direct manipulation has been established as the main interaction paradigm for Computer-Aided Design (CAD) for decades. It provides fast, incremental, and reversible actions that allow for an iterative process on a visual representation of the result. Despite its numerous advantages, some users prefer a programming-based approach where they describe the 3D model they design with a specific programming language, such as OpenSCAD. It allows users to create complex structured geometries and facilitates abstraction. Unfortunately, most current knowledge about CAD practices only focuses on direct manipulation programs. In this study, we interviewed 20 programming-based CAD users to understand their motivations and challenges. Our findings reveal that this programming-oriented population presents difficulties in the design process in tasks such as 3D spatial understanding, validation and code debugging, creation of organic shapes, and code-view navigation.
... The added thickness of a layered material does make soft, bendable living hinges difficult to realize, but all structural materials could do it. Significant bending can cause relative motion and delamination between the layers [28]. To compensate, designers can use thinner, more bendable material like Delrin. ...
... An alternative solution is to use Bezier curves, where a few control points mathematically define a contour. Solutions like ShArc [70] can capture data from these control points with less effort, creating a suitable solution to replicate organic shapes parametrically. The use of augmented reality could also help to capture control points to create Bezier curves. ...
Conference Paper
Full-text available
Direct manipulation has been established as the main interaction paradigm for Computer-Aided Design (CAD) for decades. It provides fast, incremental, and reversible actions that allow for an iterative process on a visual representation of the result. Despite its numerous advantages, some users prefer a programming-based approach where they describe the 3D model they design with a specific programming language, such as OpenSCAD. It allows users to create complex structured geometries and facilitates abstraction. Unfortunately, most current knowledge about CAD practices only focuses on direct manipulation programs. In this study, we interviewed 20 programming-based CAD users to understand their motivations and challenges. Our findings reveal that this programming-oriented population presents difficulties in the design process in tasks such as 3D spatial understanding, validation and code debugging, creation of organic shapes, and code-view navigation.
Article
Soft robots, with their remarkable advantages in various applications, face the critical challenge of embodied perception, encompassing proprioceptive sensing and perceiving unknown environments (exteroception). In this article, we achieve simultaneous continuous shape reconstruction and external force estimation for soft bending actuators using only a proprioceptive curvature sensor. We introduce a novel distributed inductive curvature sensor designed for capturing continuous shape through electromagnetic induction. In addition, we enhance an analytical static model based on the Euler–Bernoulli curved beam theory to predict the shape under pneumatic actuation and external forces. Furthermore, a model-based optimization algorithm is proposed to estimate external forces based on the measured shape. Extensive experimental validation supports the efficacy of the proposed sensor and algorithms.
Article
Multi-sensor based motion tracking is of great interest to the robotics community as it may lessen the need for expensive optical motion capture equipment. However, the traditional convolution algorithms have difficulty adapting to the data due to the changes of joints’ relative position during motion. The time-series networks often used in the past ignore the spatial characteristics of sensors. We tackle this challenge by combining long short-term memory (LSTM) with graph convolution network (GCN), adding the prior knowledge of sensor distribution, and integrating it into the motion law through the adjacency matrix. This article proposes a novel shortcut enhanced LSTM-GCN network (SE-LSTM-GCN). It connects LSTM and GCN in sequence and extracts temporal and spatial features of data. At the same time, the shortcut is used in the network to enhance the output of two middle layers and to restore the filtered information. Our experimental results on two different motion tracking datasets show that the proposed network is able to learn the mapping relationship with better universality, less tracking error, and without increasing much training time, and can better perform human motion tracking tasks. Note to Practitioners —Accurate and real-time multiple soft sensors motion tracking suits are more accepted for their low cost. However, the soft-sensor based motion tracking is not comparable to the traditional optical equipment in prediction error. To this end, we present a novel network shortcut enhanced LSTM-GCN (SE-LSTM-GCN), consisting of shortcuts, long short-term memory (LSTM), and graph convolution network (GCN). The LSTM solves the non-linear and hysteresis of soft strain sensors, and GCN is integrated into the network since the knowledge of sensor location can be put into the adjacency matrix generated by the k-nearest neighbor (KNN). While shortcuts are used to enhance the output of middle layers to form combined features. Experimental results on two public datasets show that the proposed network is superior to competing algorithms in terms of prediction error. The network can be deployed in embedded devices, such as VR gloves to provide a better gaming experience. The current algorithm is based on the relationship between sensor data and distance. In future research, we will focus on adding other human kinematics laws to the network.
Article
Integrating multiple sensors without increasing the structural complexity and their original form factor is a major concern in robotic applications, particularly in tendon-driven systems. This study proposes a multiparameter contact force sensor with bend sensing by utilizing an improved Bowden cable design. The proposed sensor consists of an end tip with an elastomer membrane, a Bowden cable consisting of a sheath and dual inner wires, and a remotely separated sensing component. The electrical wirings are decoupled from the mechanical Bowden cable system. The displacement difference between the force-sensing and bend-sensing wires is transmitted through the sheath, indicating the contact force on the end tip. The modeling and experiments verify that the contact force applied on the end tip is linearly related to the sensor output signal and is reliable after repeated measurements. The proposed sensor, utilizing the flexible Bowden cable as its sensing modality, can be integrated into a diverse range of tendon-driven robotic applications. These applications include a robotic hand with tactile and angle sensing capabilities, as well as a soft wearable robot for the hand (Exo-glove Poly II). The compact form factor of the proposed sensor enables geometric sensing without increasing structural complexity for multiparameter sensing in robotic applications.
Article
Full-text available
While soft material actuators can undergo large deformations to execute very complex motions, what is critically lacking in soft material robotic systems is the ability to collect high-resolution shape information for sophisticated functions such as environmental mapping, collision detection, and full state feedback control. This work explores the potential of a nearly commercial fiber optic shape sensor (FOSS) and presents the first demonstrations of a monolithic, multicore FOSS integrated into the structure of a fiber-reinforced soft actuator. In this pilot study, we report an open loop sensorized soft actuator capable of submillimeter position feedback that can detect the soft actuator's shape, environmental shapes, collision locations, and material stiffness properties.
Article
Full-text available
The world's population is aging: the expansion of the older adult population with multiple physical and health issues is now a huge socioeconomic concern worldwide. Among these issues, the loss of mobility among older adults due to musculoskeletal disorders is especially serious as it has severe social, mental and physical consequences. Human body joint monitoring and early diagnosis of these disorders will be a strong and effective solution to this problem. A smart joint monitoring system can identify and record important musculoskeletal-related parameters. Such devices can be utilized for continuous monitoring of joint movements during the normal daily activities of older adults and the healing process of joints (hips, knees or ankles) during the post-surgery period. A viable monitoring system can be developed by combining miniaturized, durable, low-cost and compact sensors with the advanced communication technologies and data processing techniques. In this study, we have presented and compared different joint monitoring methods and sensing technologies recently reported. A discussion on sensors' data processing, interpretation, and analysis techniques is also presented. Finally, current research focus, as well as future prospects and development challenges in joint monitoring systems are discussed.
Conference Paper
Full-text available
We introduce Serpentine, a self-powered sensor that is a reversibly deformable cord capable of sensing a variety of human input. The material properties and structural design of Serpentine allow it to be flexible, twistable, stretchable and squeezable, enabling a broad variety of expressive input modalities. The sensor operates using the principle of Triboelectric Nanogenerators (TENG), which allows it to sense mechanical deformation without an external power source. The affordances of the cord include six interactions---Pluck, Twirl, Stretch, Pinch, Wiggle and Twist. Serpentine demonstrates the ability to simultaneously recognize these inputs through a single physical interface. A 12-participant user study illustrates 95.7% accuracy for a user-dependent recognition model using a realtime system and 92.17% for user-independent offline detection. We conclude by demonstrating how Serpentine can be employed in everyday ubiquitous computing applications.
Article
Full-text available
Measuring high curvatures is essential in various applications such as structural engineering, medical treatment, and robotics. Herein, we present a novel ultra-high curvature sensor with an ultra-thin and highly flexible structure incorporating fiber Bragg gratings (FBGs). The sensor can measure curvature of bidirectional bending up to 200 m⁻¹. In addition, the multi-bend curvature sensor in conjunction with the proposed calibration scheme enables the accurate reconstruction of a curve with varying curvature. The sensitivity and the accuracy of the curvature sensor are investigated for different sensor designs. Finally, we demonstrate the accurate shape sensing of various 2-D patterns using the multi-bend curvatures sensor.
Article
Full-text available
This paper presents a fully printable sensorised bending actuator that can be calibrated to provide reliable bending feedback and simple contact detection. A soft bending actuator following a pleated morphology, as well as a flexible resistive strain sensor, were directly 3D printed using easily accessible FDM printer hardware with a dual-extrusion tool head. The flexible sensor was directly welded to the bending actuator’s body and systematically tested to characterise and evaluate its response under variable input pressure. A signal conditioning circuit was developed to enhance the quality of the sensory feedback and flexible conductive threads were used for wiring. The sensorised actuator’s response was then calibrated using a vision system to convert the sensory readings to real bending angle values. The empirical relationship was derived using linear regression and validated at untrained input conditions to evaluate its accuracy. Furthermore, the sensorised actuator was tested in a constrained setup that prevents bending, to evaluate the potential of using the same sensor for simple contact detection by comparing the constrained response to that of the free-bending state at the same input pressure. The results of this work demonstrated how a dual-extrusion FDM printing process can be tuned to directly print highly customisable flexible strain sensors that are able to provide reliable bending feedback and basic contact detection. The addition of such sensing capability to bending actuators enhances their functionality and reliability for applications such as; controlled soft grasping, flexible wearables, and haptic devices.
Article
We propose a stretch-sensing soft glove to interactively capture hand poses with high accuracy and without requiring an external optical setup. We demonstrate how our device can be fabricated and calibrated at low cost, using simple tools available in most fabrication labs. To reconstruct the pose from the capacitive sensors embedded in the glove, we propose a deep network architecture that exploits the spatial layout of the sensor itself. The network is trained only once, using an inexpensive off-the-shelf hand pose reconstruction system to gather the training data. The per-user calibration is then performed on-the-fly using only the glove. The glove's capabilities are demonstrated in a series of ablative experiments, exploring different models and calibration methods. Comparing against commercial data gloves, we achieve a 35% improvement in reconstruction accuracy.
Article
This paper presents a comprehensive critical review of technologies used in the development of fibre optic shape sensors (FOSSs). Their operation is based on multi-dimensional bend measurements using a series of fibre optic sensors. Optical fibre sensors have experienced tremendous growth from simple bend sensors in 1980s to full three-dimensional FOSSs using multicore fibres in recent years. Following a short review of conventional contact-based shape sensor technologies, the evolution trend and sensing principles of FOSSs are presented. This paper identifies the major optical fibre technologies used for shape sensing and provides an account of the challenges and emerging applications of FOSSs in various industries such as medical robotics, industrial robotics, aerospace and mining industry.
Article
Resistive flex sensors were increasingly used in different areas for their interesting property to change their resistance when bent. In particular, they can be applied to human segment in biomedical devices to register static and dynamic postures. In spite of their interesting properties, such as robustness, low price and long life, they often demonstrate non-linear response and lower sensitivity at small bending angles. This paper provides investigation to improve flex sensors linearity and sensitivity to measure body joint angles with better accuracy. To this aim, an empirical model of the sheet (or surface) resistance of the active layer, to simulate its behavior against the layer shape and size as well as the bending angle, was provided, to investigate whether changes of the standard rectangular shape can improve sensitivity and linearity. In addition, to date commercial flex sensors have been characterized only against the bending angle with a radius of curvature smaller than the device length, so limiting the application to small joints such as finger or knee. In order to extend the flex sensor applications, for instance, to measure the trunk posture in back disease and rehabilitation monitoring, the sensor response against a radius of curvature greater than the sensor length was analyzed. Finally, a new modeling technique, based on the inverse model of the sensor characteristic, to enable fast measurements of the bending angle or the radius of curvature from sensor response also in real time, and fast calibration procedures, fitting the same model to measurements with different joint size and even device, were developed.
Conference Paper
FlexStylus, a flexible stylus, detects deformation of the barrel as a vector with both a rotational and an absolute value, providing two degrees of freedom with the goal of improving the expressivity of digital art using a stylus device. We outline the construction of the prototype and the principles behind the sensing method, which uses a cluster of four fibre-optic based deformation sensors. We propose interaction techniques using the FlexStylus to improve menu navigation and tool selection. Finally, we describe a study comparing users' ability to match a changing target value using a commercial pressure stylus and the FlexStylus' absolute deformation. When using the FlexStylus, users had a significantly higher accuracy overall. This suggests that deformation may be a useful input method for future work considering stylus augmentation.