Content uploaded by Eiman Kanjo
Author content
All content in this area was uploaded by Eiman Kanjo on Sep 17, 2020
Content may be subject to copyright.
TangToys: Smart Toys to Communicate and Improve Children’s
Wellbeing
Kieran Woodward
kieran.woodward@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
Eiman Kanjo
eiman.kanjo@ntu.ac.uk
Nottingham Trent University
Nottingham, UK
David J Brown
Nottingham Trent University
Nottingham, UK
Becky Inkster
University of Cambridge
Cambridge, UK
ABSTRACT
Children can nd it challenging to communicate their emotions
especially when experiencing mental health challenges. Technolog-
ical solutions may help children communicate digitally and receive
support from one another as advances in networking and sensors
enable the real-time transmission of physical interactions. In this
work, we pursue the design of multiple tangible user interfaces
designed for children containing multiple sensors and feedback
actuators. Bluetooth is used to provide communication between
Tangible Toys (TangToys) enabling peer to peer support groups to
be developed and allowing feedback to be issued whenever other
children are nearby. TangToys can provide a non-intrusive means
for children to communicate their wellbeing through play.
CCS CONCEPTS
•Human-centered computing →Ubiquitous and mobile com-
puting systems and tools.
KEYWORDS
Tangible User Interfaces; Children; Communication; Mental Well-
being; Emotion; Sensors
ACM Reference Format:
Kieran Woodward, Eiman Kanjo, David J Brown, and Becky Inkster. 2020.
TangToys: Smart Toys to Communicate and Improve Children’s Wellbeing.
In Adjunct Proceedings of the 2020 ACM International Joint Conference on
Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM Inter-
national Symposium on Wearable Computers (UbiComp/ISWC ’20 Adjunct),
September 12–16, 2020, Virtual Event, Mexico. ACM, New York, NY, USA,
3 pages. https://doi.org/10.1145/3410530.3414375
1 INTRODUCTION
The mental wellbeing of children is increasingly important as more
young people than ever before are experiencing high levels of stress
[
2
]. Tangible User Interfaces (TUIs) present new opportunities to
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic permission
and/or a fee. Request permissions from permissions@acm.org.
UbiComp/ISWC ’20 Adjunct, September 12–16, 2020, Virtual Event, Mexico
©2020 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-8076-8/20/09. . . $15.00
https://doi.org/10.1145/3410530.3414375
digitise physical interfaces to help children communicate their well-
being. Recent advances in microcontrollers and sensors enable small
interfaces to be developed that can process and communicate sensor
data in real-time. Children’s toys represent an ideal embodiment
for TUIs as they provide sucient space for the electronics and en-
courage tactile interactions. Although a limited number of TUIs for
mental wellbeing have previously been developed, many of these
were not engaging for children and often contained physiological
sensors which prevent physical interactions commonly used by
children to interact with objects such as toys.
While there are many challenges in developing mental well-
being interfaces, the decreasing cost and increasing capability of
networking, sensors and microcontrollers is enabling new forms of
interfaces to be developed [
16
]. An interface that can actively mon-
itor and enable the communication of a user’s physical interactions
and mood would be benecial for all. Through the use of Blue-
tooth Low Energy (BLE), TUIs can communicate with one another
enabling real-time communication networks to be developed.
TUIs have previously been used to provide a method to com-
municate emotions and mental health states [
17
]. Emoball [
5
] is a
physical ball that enabled users to report their emotions by squeez-
ing the device. Similarly, Subtle Stone [
4
] allowed users to express
their emotions as a colour on the stone. Using colours to represent
emotions enabled the private communication of emotions to only
those who understood such colour representations. Mood TUI [
13
]
also enabled users to self report their emotions but additionally
collected data from the users’ smartphones such as location and
heart rate. Overall, participants in this study found TUIs exciting to
use, and that a small sized device was key for sustained interactions.
The vast majority of previously developed wellbeing interfaces
have utilised self-reporting, which children in particular may nd
challenging. Recent developments in non-invasive sensors intro-
duce the possibility to objectively and intuitively measure physical
interactions and physiological changes in real-time. Motion data
has previously been used to infer emotions with 81.2% accuracy
across 3 classes [
19
]. However, similar studies reported lower levels
of accuracy between 50% - 75% [
11
] [
9
] when inferring emotions
from motion data.
The ability for children to communicate their emotions is vital
as children with diculties communicating are more at risk in
terms of social acceptance and bullying [
8
]. Furthermore, better
relationships and communication with friends oers protection
497
UbiComp/ISWC ’20 Adjunct, September 12–16, 2020, Virtual Event, Mexico Woodward et al.
against poor mental health in the future [
7
]. This research intro-
duces Tangible Toys (TangToys) with the aim of communicating
mental wellbeing inferred from the embedded sensors. Initial proto-
types embed sensors used to measure interactions and well-being
and BLE to enable real-time communication. The ability for devices
to communicate with one another enables friends to communicate
when socially distant and the ability to discover other nearby users.
Furthermore, this work highlights key directions for the continued
renement of TangToys.
2 TANGIBLE TOYS (TANGTOYS)
Few sensor based interfaces have been designed for children even
though children traditionally nd it challenging to communicate
their mental wellbeing [
10
]. We introduce the concept of Tang-
Toys as children’s toys that embed electronics to measure tangible
interactions. The interfaces can vary in shape, size and material
including soft balls and teddies designed for younger children aged
5-7 as this is when children develop self-conscious emotions and
develop an emotional front [
12
]. Additionally, 3D printed dgeting
cubes have been designed for older children aged 7-10 as children
this age are still developing their emotional awareness [
12
]. As
children physically interact with TangToys in the same way as
traditional toys all of the interfaces are suitable for children and
encourage engagement by resembling similar toys.
2.1 Communication Framework
Embedding sensors within toys that can communicate with one
another through BLE oers many new opportunities for real-time
interactions. BLE 4.2 has a range of around 50m allowing TangToys
to communicate with one another in locations such as playgrounds.
In the following sections we present two opportunities for real-time
digital social interaction between TangToys.
Table 1 shows the ve TangToys developed during a co-design
and co-creation workshop including 2 soft teddies, a soft ball, a
3D printed cube and a 3D printed torus. Each TangToy includes a
microcontroller and micro SD card to record all interactions along
with BLE 4.2 for communication. A range of sensors can be used
to monitor children’s interactions with the toys including capaci-
tive sensors to measure touch and 9-Degree Of Freedom Inertial
Measurement Unit (9-DOF IMU) to measure motion. Physiological
sensors can also be embedded within the toys such as Heart Rate
(HR) sensors as they directly correlated with the sympathetic ner-
vous system helping to monitor mental wellbeing [
14
] [
1
]. All of the
TangToys measure motion and touch interactions while only the 3D
printed interfaces designed for older children include physiological
sensors, as younger children may not understand the need to place
their nger on the physiological sensor whilst playing with the toy.
In addition to the sensors, TangToys can provide real-time inter-
ventional feedback [
18
] activated by wireless communication with
other TangToys. Haptic feedback has been included within some
of the developed prototypes issuing a physical sense, resembling
touch and providing comfort which can improve mental wellbeing
[
6
], [
3
]. Additionally, visual feedback in the form of multi-coloured
LEDs has been included within the soft ball and teddy prototypes.
Table 1: Initial TangToys prototypes.
Device
Image Description
Ball
A soft ball embedding 9-DOF IMU to
measure motion, capacitive sensors
to measure touch and Multi-coloured
LEDs to perform visual feedback.
Cube
A 3D printed cube embedding 9-DOF
IMU, capacitive touch, HR, EDA sen-
sors and haptic feedback
Teddy
A soft teddy embedding 9-DOF IMU,
capacitive touch sensor and visual
feedback
Torus
A 3D printed tours embedding HR,
EDA, 9-DOF IMU, capacitive touch
sensor and haptic feedback
Teddy
A soft teddy embedding 9-DOF IMU,
capacitive touch sensor, haptic and
visual feedback
TangToys have been presented in focus groups to teachers of
young students with mild to moderate learning disabilities to pro-
vide feedback on the design and functionality of the interfaces [
15
].
Teachers considered the methods used to interact with TangToys
suitable for children and believed the way in which children interact
with the toys will indicate their wellbeing. Additionally, teachers
liked the design of the toys as they appear similar to other toys
helping to reduce stigma. Overall, the teachers reported the design,
sensors and communication capabilities were all suitable for chil-
dren and believed TangToys would promote the communication of
wellbeing between friends.
2.1.1 Peer to Peer Communication (P2P). By utilising P2P commu-
nication it is possible for two connected devices to communicate
with one another. This method of communication helps friends
who may be nearby but socially distanced to provide physical com-
munication that is not possible with other devices. When a child
plays with a TangToy the capacitive sensor and accelerometer data
measuring touch and motion can be actuated on the paired interface
through the embedded haptic and visual feedback to simulate phys-
ical communication. The connected friend can then react to this
498
TangToys: Smart Toys to Communicate and Improve Children’s Wellbeing UbiComp/ISWC ’20 Adjunct, September 12–16, 2020, Virtual Event, Mexico
communication by interacting with their device allowing friends
to wirelessly support one another through physical interactions.
The range of feedback oered diers depending on the interac-
tions with the paired devices. For example, if a child is aggressively
shaking their TangToy or touching it harshly this can result in pro-
longed sharp haptic feedback patterns being played on the paired
device and red visual feedback. This enables friends to physically
communicate how they are feeling and provide comfort to one an-
other by replying with soft, gentle interactions to provide comfort
and a sense of presence, as shown in Figure 1.
Figure 1: Two children playing using TangToys.
2.1.2 Wireless Scanning. Each TangToy can also use its Bluetooth
capabilities to broadcast its presence to other TangToys. When a
TangToy detects another device nearby this can initiate feedback
being issued to alert the child of other nearby children. This allows
a child to nd other children who may require support when not
near their friends to facilitate peer to peer communication. These
children can then interact with the devices to form a support group
to communicate their wellbeing to each other. The feedback actu-
ated when detecting other devices can be impacted by the number
of nearby interfaces. For example, if a single child is detected nearby
more subtle haptic feedback can be issued compared with more
pronounced feedback when multiple children are nearby. Similarly,
the colour displayed on the TangToy can change dependent on the
number of users located nearby to alert the user visually. Using
this method of interaction would not enable the same capabilities
as the P2P communication, but would enable each device to inter-
act automatically with other nearby devices, and aord a sense of
’togetherness’.
3 CONCLUSION AND FUTURE WORK
We have presented TangToys, a new concept to combine tangible
user interfaces with traditional toys. Various sensors can be embed-
ded within TangToys to communicate physical interactions, such
as movement and touch, in real-time with other TangToys. Using
a peer to peer communication system enables friends to commu-
nicate their wellbeing through device interactions that can then
be actuated using haptic and visual feedback on a friend’s device.
However, there are challenges in distinguishing between emotions
that may result in similar interactions. Alternatively, TangToys can
simultaneously broadcast and scan for nearby devices allowing
for TangToys to discover other TangToys and create local support
networks.
In the future, TangToys should be trialled, potentially in schools
where children will be able to communicate with each other. The
impact of the communication networks can be measured along
with the duration in which children use the interfaces. Parental
monitoring through the use of a mobile app could also be included,
enabling parents to view interactions with the toys.
4 ACKNOWLEDGEMENT
This project has been funded by the Nurture Network (eNurture). eNurture is funded
by UK Research and Innovation (UKRI) and their support is gratefully acknowledged
(Grant reference: ES/S004467/1). Any views expressed here are those of the project
investigators and do not necessarily represent the views of eNurture or UKRI.
REFERENCES
[1]
Nouf Alajmi, Eiman Kanjo, Nour El Mawass, and Alan Chamberlain. 2013. Shop-
mobia: An Emotion-Based Shop Rating System. In 2013 Humaine Association
Conference on Aective Computing and Intelligent Interaction. IEEE, 745–750.
[2]
American Psychological Association. 2013. Stress in America Are Teens Adopting
Adults’ Stress Habits? https://www.apa.org/news/press/releases/stress/2013/
stress-report.pdf
[3]
Ruben T Azevedo, Nell Bennett, Andreas Bilicki, Jack Hooper, Fotini
Markopoulou, and Manos Tsakiris. 2017. The calming eect of a new wear-
able device during the anticipation of public speech. Sci Rep 7, 1 (2017), 2285.
[4]
Madeline Balaam, Geraldine Fitzpatrick, Judith Good, and Rosemary Luckin.
2009. Exploring Aective Technologies for the Classroom with the Subtle Stone.
Proceedings of the 28th international conference on Human factors in computing
systems - CHI ’10 (2009), 1623.
[5]
José Bravo, Ramón Hervás, and Vladimir Villarreal. 2015. Ambient intelligence
for health rst international conference, AmIHEALTH 2015 Puerto Varas, Chile,
December 1–4, 2015 proceedings. Lecture Notes in Computer Science 9456 (2015),
189–200.
[6]
Brendan Corbett, Chang S. Nam, and Takehiko Yamaguchi. 2016. The Eects of
Haptic Feedback and Visual Distraction on Pointing Task Performance. Interna-
tional Journal of Human-Computer Interaction 32, 2 (2 2016), 89–102.
[7]
Claire L Forrest, Jenny L Gibson, Sarah L Halligan, and Michelle C St Clair. 2018.
A longitudinal analysis of early language diculty and peer problems on later
emotional diculties in adolescence: Evidence from the Millennium Cohort
Study. Autism & Developmental Language Impairments (2018).
[8]
Geo Lindsay and Julie Dockrell. 2012. The relationship between speech, lan-
guage and communication needs (SLCN) and behavioural, emotional and social
diculties (BESD). Department for Education (2012).
[9]
Andreas Færøvig Olsen and Jim Torresen. 2017. Smartphone accelerometer data
used for detecting human emotions. In 3rd International Conference on Systems
and Informatics.
[10]
Public Health England. 2016. Mental health of children in England.
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/
attachment_data/le/575632/Mental_health_of_children_in_England.pdf
[11]
Juan C. Quiroz, Min Hooi Yong, and Elena Geangu. 2017. Emotion-recognition
using smart watch accelerometer data: Preliminary ndings. In 2017 ACM Inter-
national Joint Conference on Pervasive and Ubiquitous Computing and Proceedings
of the 2017 ACM International Symposium on Wearable Computers (UbiComp ’17).
[12]
Carolyn Saarni. 2011. Emotional development in childhood. Encyclopedia on
Early Childhood Develpment (2011).
[13]
Federico Sarzotti. 2018. Self-Monitoring of Emotions and Mood Using a Tangible
Approach. Computers 7, 1 (1 2018), 7.
[14]
Nandita Sharma and Tom Gedeon. 2012. Objective measures, sensors and compu-
tational techniques for stress recognition and classication: A survey. Computer
Methods and Programs in Biomedicine 108, 3 (12 2012), 1287–1301.
[15]
Kieran Woodward, Eiman Kanjo, David Brown, and T. M. McGinnity. 2019. AI-
Powered Tangible Interfaces to Transform Children ’ s Mental Well-being. In The
5th IEEE International Conference on Internet of People. Leicester.
[16]
Kieran Woodward, Eiman Kanjo, David Brown, T. M. McGinnity, Becky Inkster,
Donald J Macintyre, and Athanasios Tsanas. 2020. Beyond Mobile Apps: A Survey
of Technologies for Mental Well-being. IEEE Transactions on Aective Computing
(2020).
[17]
Kieran Woodward, Eiman Kanjo, Andreas Oikonomou, and Samuel Burton. 2018.
Emoecho: A tangible interface to convey and communicate emotions. In Ubi-
Comp/ISWC 2018 - 2018 ACM International Joint Conference on Pervasive and
Ubiquitous Computing and Proceedings of the 2018 ACM International Symposium
on Wearable Computers.
[18]
Kieran Woodward, Eiman Kanjo, Muhammad Umir, and Corina Sas. 2019. Har-
nessing Digital Phenotyping to Deliver Real-Time Interventional Bio-Feedback.
In WellComp’19: 2nd International Workshop on Computing for Well-Being - UBI-
COMP.
[19]
Zhan Zhang, Yufei Song, Liqing Cui, Xiaoqian Liu, and Tingshao Zhu. 2016. Emo-
tion recognition based on customized smart bracelet with built-in accelerometer.
PeerJ 4 (2016), e2258.
499
