Available via license: CC BY 4.0
Content may be subject to copyright.
TangToys: Smart Toys that can Communicate and Improve
Children’s Wellbeing
Kieran Woodward
kieran.woodward@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
Eiman Kanjo
eiman.kanjo@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
David J Brown
Nottingham Trent University
Nottingham, UK
Becky Inkster
University of Cambridge
Cambridge, UK
ABSTRACT
Children can nd it challenging to communicate their emotions
especially when experiencing mental health challenges. Technolog-
ical solutions may help children communicate digitally and receive
support from one another as advances in networking and sensors
enable the real-time transmission of physical interactions. In this
work, we pursue the design of multiple tangible user interfaces
designed for children containing multiple sensors and feedback
actuators. Bluetooth is used to provide communication between
Tangible Toys (TangToys) enabling peer to peer support groups to
be developed and allowing feedback to be issued whenever other
children are nearby. TangToys can provide a non-intrusive means
for children to communicate their wellbeing through play.
CCS CONCEPTS
•Human-centered computing →Ubiquitous and mobile com-
puting systems and tools.
KEYWORDS
Tangible User Interfaces, Children, Communication, Mental Well-
being, Emotion, Sensors
ACM Reference Format:
Kieran Woodward, Eiman Kanjo, David J Brown, and Becky Inkster. 2020.
TangToys: Smart Toys that can Communicate and Improve Children’s Well-
being. In Proceedings of UbiComp ’20. ACM, New York, NY, USA, 3 pages.
https://doi.org/10.1145/1122445.1122456
1 INTRODUCTION
The mental wellbeing of children is increasingly important as more
young people than ever before are experiencing high levels of stress
[
2
]. Tangible User Interfaces (TUIs) present new opportunities to
digitise physical interfaces to help children communicate their well-
being. Recent advances in microcontrollers and sensors enable small
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
UbiComp ’20, September 12–16, 2020, Cancun, Mexico
©2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-XXXX-X/18/06.. .$15.00
https://doi.org/10.1145/1122445.1122456
interfaces to be developed that can process and communicate sensor
data in real-time. Children’s toys represent an ideal embodiment
for TUIs as they provide sucient space for the electronics and en-
courage tactile interactions. Although a limited number of TUIs for
mental wellbeing have previously been developed, many of these
were not engaging for children and often contained physiological
sensors which prevent physical interactions commonly used by
children to interact with objects such as toys.
While there are many challenges in developing mental wellbeing
interfaces, the decreasing cost and increasing capability of network-
ing, sensors and microcontrollers is enabling new forms of inter-
faces to be developed [
16
]. An interface that could actively monitor
and enable the communication of a user’s physical interactions and
mood would be benecial for all. Through the use of Bluetooth Low
Energy (BLE), TUIs can communicate with one another enabling
real-time communication networks to be developed.
TUIs have previously been used to provide a method to com-
municate emotions and mental health states [
17
]. Emoball [
5
] is a
physical ball that enabled users to report their emotions by squeez-
ing the device. Similarly, Subtle Stone [
4
] allowed users to express
their emotions as a colour on the stone. Using colours to represent
emotions enabled the private communication of emotions to only
those who understood such colour representations. Mood TUI [
13
]
also enabled users to self report their emotions but additionally
collected data from the usersâĂŹ smartphones such as location and
heart rate. Overall, participants in this study found TUIs exciting to
use, and that a small sized device was key for sustained interactions.
The vast majority of previously developed wellbeing interfaces
have utilised self-reporting, which children in particular may nd
challenging. Recent developments in non-invasive sensors intro-
duce the possibility to objectively and intuitively measure physical
interactions and physiological changes in real-time. Motion data
collected through accelerometers, gyroscopes and magnetometers
could be used to measure physical interactions in addition to phys-
iological sensors to monitor indicators of wellbeing. Previously,
motion data has been used to infer emotions with 81.2% accuracy
across 3 classes [
19
]. However, similar studies reported lower lev-
els of accuracy between 50% - 72% [
12
] [
10
] [
7
] when inferring
emotions from motion data.
The ability to measure and communicate wellbeing through non-
invasive sensors presents many opportunities. This research intro-
duces Tangible Toys (TangToys) with the aim of communicating
arXiv:2007.05286v1 [cs.HC] 10 Jul 2020
UbiComp ’20, September 12–16, 2020, Cancun, Mexico Woodward et al.
mental wellbeing inferred from the embedded sensors. Initial proto-
types embed sensors used to measure interactions and well-being
and Bluetooth to enable real-time communication in-situ settings.
The ability for devices to communicate with one another enables
friends to communicate when socially distant and the ability to
discover other nearby users. Furthermore, this work highlights key
directions for the continued renement of TangToys.
2 TANGIBLE TOYS (TANGTOYS)
Few sensor based interfaces have been designed for children even
though children traditionally nd it challenging to communicate
their mental wellbeing [
11
]. We introduce the concept of TangToys
as children’s toys that embed electronics to measure tangible inter-
actions. The interfaces can vary in shape, size and material ranging
from soft balls and teddies designed for younger children, to 3D
printed dgeting cubes designed for older children. As children
physically interact with TangToys in the same way as traditional
toys all of the interfaces are suitable for children and encourage
engagement by resembling similar toys.
2.1 Communication Framework
Embedding sensors within toys that can communicate with one
another through Bluetooth oers many new opportunities for real-
time interactions. BLE 4.2 has a range of around 50m allowing
TangToys to communicate with one another in locations such as
playgrounds. In the following sections we present two opportunities
for real-time digital social interaction between TangToys.
Table 1 shows the ve TangToys developed during a co-design
and co-creation workshop including 2 soft teddies, a soft ball and a
3D printed cube and torus. Each TangToy includes a microcontroller
and micro SD card to record all interactions along with bluetooth
4.2 for communication. A range of sensors can be used to monitor
children’s interactions with the toys including capacitive sensors
to measure touch and 9-Degree Of Freedom Inertial Measurement
Unit (9-DOF IMU) to measure motion. Physiological sensors can
also be embedded within the toys such as Heart Rate (HR) sensors
as they directly correlated with the sympathetic nervous system
helping to monitor mental wellbeing [
14
] [
1
]. All of the TangToys
measure motion and touch interactions while only the 3D printed
interfaces designed for older children include HR sensors to mea-
sure physiological changes.
In addition to the sensors, TangToys can provide real-time inter-
ventional feedback [
18
]. Haptic feedback to provide the sensation
of touch has been included within some of the developed proto-
types. Haptic feedback provides a physical sense resembling touch
which can improve mental wellbeing [
9
] [
6
]. Additionally, visual
feedback in the form of multi-coloured LEDs has been included
within the soft ball and teddy prototypes. These forms of feedback
can function as real-time interventions if poor mental wellbeing
can be detected, helping to alert the user.
TangToys have been presented in focus groups to teachers of
young students with mild to moderate learning disabilities to pro-
vide feedback on the design and functionality of the interfaces [
15
].
Teachers considered the methods used to interact with TangToys
suitable for children and believed the way in which children interact
with the toys will indicate their wellbeing. Additionally, teachers
Table 1: Initial TangToys prototypes.
Device Image Description
Ball
A soft ball embedding 9-DOF
IMU to measure motion, capac-
itive sensors to measure touch
and Multi-coloured LEDs to per-
form visual feedback.
Cube
A 3D printed cube embedding 9-
DOF IMU, capacitive touch, HR,
EDA sensors and haptic feed-
back
Teddy
A soft teddy embedding 9-DOF
IMU, capacitive touch sensor
and visual feedback
Torus
A 3D printed tours embedding
HR, EDA, 9-DOF IMU, capac-
itive touch sensor and haptic
feedback
Teddy
A soft teddy embedding 9-DOF
IMU, capacitive touch sensor,
haptic and visual feedback
liked the design of the toys as they appear similar to other toys
helping to reduce stigma. Overall, the teachers reported the design,
sensors and communication capabilities were all suitable for chil-
dren and believed would promote the communication of wellbeing
between friends.
2.1.1 Peer to Peer Communication (P2P). By utilising P2P commu-
nication it is possible for two connected devices to communicate
with one another. This method of communication helps friends
who may be nearby but socially distanced to provide physical com-
munication that is not possible with other devices. When one user
touches or moves their toy, the paired interface can react through
the embedded visual or haptic feedback, providing a sense of phys-
ical interaction. The visual feedback and haptic patterns can dier
dependent on the way in which the partner device is interacted
with. For example, if a child is aggressively shaking their TangToy
or touching it harshly this can result in prolonged sharp haptic
feedback patterns being played on the paired device and red visual
TangToys: Smart Toys that can Communicate and Improve Children’s Wellbeing UbiComp ’20, September 12–16, 2020, Cancun, Mexico
feedback. This enables friends to physically communicate how they
are feeling and provide comfort to one another (see gure 1).
Figure 1: Two children playing using TangToys.
The range of feedback that can be oered allows for emotions
to be wirelessly communicated with haptic feedback providing a
sense of presence as it simulates touch. Therefore, capacitive sensor
data measuring touch can be actuated on the partner device using
haptic feedback to simulate physical communication. Furthermore,
haptic feedback can be used to comfort as previous work has shows
the potential of haptic feedback to improve wellbeing [8], [3].
2.1.2 Wireless Scanning. Each TangToy can also use its Bluetooth
capabilities to broadcast its presence to other TangToys. When a
TangToy detects another device nearby this can initiate feedback
being issued to alert the child of other nearby children. This allows
a child to nd other children who may require support when not
near their friends to facilitate peer to peer communication. These
children can then interact with the devices to form a support group
to communicate their wellbeing to each other. The feedback actu-
ated when detecting other devices can be impacted by the number
of nearby interfaces. For example, if a single child is detected nearby
more subtle haptic feedback can be issued compared with more
pronounced feedback when multiple children are nearby. Similar,
the colour displayed on the TangToy can change dependent on the
number of users located nearby to alert the user visually. Using
this method of interaction would not enable the same capabilities
as the P2P communication, but would enable each device to inter-
act automatically with other nearby devices, and aord a sense of
’togetherness’.
3 CONCLUSION AND FUTURE WORK
We have presented TangToys, a new concept to combine tangible
user interfaces with traditional toys. Various sensors can be embed-
ded within TangToys to communicate physical interactions such
as movement and touch in real-time with other TangToys. Using
a peer to peer communication system enables friends to commu-
nicate their wellbeing through device interactions that can then
be actuated using haptic and visual feedback on a friend’s device.
Alternatively, TangToys can simultaneously broadcast and scan for
nearby devices allowing for TangToys to discover other TangToys
and create local support networks.
In the future, TangToys should be trialled with children, poten-
tially in schools where they will be able to communicate with each
other through touch and haptic feedback. The impact of the com-
munication networks can be measured along with the duration in
which children use the interfaces. Parental monitoring could also
be included through the use of a mobile app enabling parents to
view previous interactions with the toys.
4 ACKNOWLEDGEMENT
This project has been funded by the Nurture Network (eNurture). eNurture is funded
by UK Research and Innovation (UKRI) and their support is gratefully acknowledged
(Grant reference: ES/S004467/1). Any views expressed here are those of the project
investigators and do not necessarily represent the views of eNurture or UKRI.
REFERENCES
[1]
Nouf Alajmi, Eiman Kanjo, Nour El Mawass, and Alan Chamberlain. 2013. Shop-
mobia: An Emotion-Based Shop Rating System. In 2013 Humaine Association
Conference on Aective Computing and Intelligent Interaction. IEEE, 745–750.
[2]
American Psychological Association. 2013. Stress in America Are Teens Adopting
Adults’ Stress Habits? https://www.apa.org/news/press/releases/stress/2013/
stress-report.pdf
[3]
Ruben T Azevedo, Nell Bennett, Andreas Bilicki, Jack Hooper, Fotini
Markopoulou, and Manos Tsakiris. 2017. The calming eect of a new wear-
able device during the anticipation of public speech. Sci Rep 7, 1 (2017), 2285.
[4]
Madeline Balaam, Geraldine Fitzpatrick, Judith Good, and Rosemary Luckin.
2009. Exploring Aective Technologies for the Classroom with the Subtle Stone.
Proceedings of the 28th international conference on Human factors in computing
systems - CHI ’10 (2009), 1623.
[5]
José Bravo, Ramón Hervás, and Vladimir. 2015. Ambient intelligence for health
rst international conference, AmIHEALTH 2015 Puerto Varas, Chile, December
1âĂŞ4, 2015 proceedings. Lecture Notes in Computer Science 9456 (2015), 189–200.
[6]
Brendan Corbett, Chang S. Nam, and Takehiko Yamaguchi. 2016. The Eects of
Haptic Feedback and Visual Distraction on Pointing Task Performance. Interna-
tional Journal of Human-Computer Interaction 32, 2 (feb 2016), 89–102.
[7]
Rasam Bin Hossain, Mefta Sadat, and Hasan Mahmud. 2014. Recognition of
human aection in smartphone perspective based on accelerometer and user’s
sitting position. In 2014 17th International Conference on Computer and Information
Technology, ICCIT 2014.
[8]
Chelsea Kelling, Daniella Pitaro, and Jussi Rantala. 2016. Good vibes. In Proceed-
ings of the 20th International Academic Mindtrek Conference on - AcademicMindtrek
’16. ACM Press, New York, New York, USA, 130–136.
[9]
Josephin Klamet, Matthies, and Denys J. C. 2016. WeaRelaxAble. In Proceedings of
the 3rd International Workshopon Sensor-base d ActivityRe cognition and Interaction
- iWOAR ’16. ACM Press, New York, New York, USA, 1–6.
[10]
Andreas Færøvig Olsen and Jim Torresen. 2017. Smartphone accelerometer
data used for detecting human emotions. In 2016 3rd International Conference on
Systems and Informatics, ICSAI 2016.
[11]
Public Health England. 2016. Mental health of children in England.
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/
attachment{_}data/le/575632/Mental{_}health{_}of{_}children{_}in{_}England.
pdf
[12]
Juan C. Quiroz, Min Hooi Yong, and Elena Geangu. 2017. Emotion-recognition
using smart watch accelerometer data: Preliminary ndings. In 2017 ACM Inter-
national Joint Conference on Pervasive and Ubiquitous Computing and Proceedings
of the 2017 ACM International Symposium on Wearable Computers (UbiComp ’17).
https://doi.org/10.1145/3123024.3125614
[13]
Federico Sarzotti. 2018. Self-Monitoring of Emotions and Mood Using a Tangible
Approach. Computers 7, 1 (jan 2018), 7.
[14]
Nandita Sharma and Tom Gedeon. 2012. Objective measures, sensors and compu-
tational techniques for stress recognition and classication: A survey. Computer
Methods and Programs in Biomedicine 108, 3 (dec 2012), 1287–1301.
[15]
Kieran Woodward, Eiman Kanjo, David Brown, and T. M. McGinnity. 2019. AI-
Powered Tangible Interfaces to Transform Children ’ s Mental Well-being. In The
5th IEEE International Conference on Internet of People. Leicester.
[16]
Kieran Woodward, Eiman Kanjo, David Brown, T. M. McGinnity, Becky Inkster,
and Donald J Macintyre. 2019. Beyond Mobile Apps: A Survey of Technologies
for Mental Well-being. IEEE Transactions on Aective Computing (2019).
[17]
Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Samuel Burton. 2018.
Emoecho: A tangible interface to convey and communicate emotions. In Ubi-
Comp/ISWC 2018 - 2018 ACM International Joint Conference on Pervasive and
Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium
on Wearable Computers.
[18]
Kieran Woodward, Eiman Kanjo, Muhammad Umir, and Corina Sas. 2018. Har-
nessing Digital Phenotyping to Deliver Real-Time Interventional Bio-Feedback.
In WellComp’19: 2nd International Workshop on Computing for Well-Being - UBI-
COMP 2019. https://doi.org/10.1145/1234567890
[19]
Zhan Zhang, Yufei Song, Liqing Cui, Xiaoqian Liu, and Tingshao Zhu. 2016. Emo-
tion recognition based on customized smart bracelet with built-in accelerometer.
PeerJ 4 (2016), e2258.
