Mirko Gelsomini

Mirko Gelsomini
Politecnico di Milano | Polimi · Department of Electronics, Information, and Bioengineering

PhD

About

64
Publications
27,707
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
715
Citations

Publications

Publications (64)
Article
Full-text available
Phygital interaction is a form of tangible interaction where digital and physical contents are combined in such a way that the locus of multimedia information is detached from the physical material(s) manipulated by the user. The use of phygital interaction is supported by several theoretical approaches that emphasize the development of cognitive s...
Chapter
Full-text available
This chapter explores the potential of co-creation and user innovation, investigating the physical-motor needs of children diagnosed with cerebral palsy with specific attention to the translation of movement in sound stimuli. It describes the co-design and development of BODYSOUND, a smart system that exploits a playful activity to encourage moveme...
Chapter
Full-text available
This paper presents a co-design toolkit that aims to involve people with Intellectual Disability (ID) in the ideation of smart outdoor experiences. We conducted a series of workshops with young adults with ID and special-education professionals of a social-care center, to identify the physical and digital experience that could favour reflection and...
Book
This book provides an in-depth review of the historical and state-of-the-art use of technology by and for individuals with autism. The design, development, deployment, and evaluation of interactive technologies for use by and with individuals with autism have been rapidly increasing over the last few decades. There is great promise for the use of t...
Article
Full-text available
This paper presents the design and evaluation of IMAGINE, a novel interactive immersive smart space for embodied learning. In IMAGINE children use full-body movements and gestures to interact with multimedia educational contents projected on the wall and on the floor, while synchronized light effects enhance immersivity. A controlled study performe...
Conference Paper
Full-text available
The paper introduces Reflex, a mirrored camera mobile training application for persons with Neuro-Developmental Disorder (NDD). The game, offered through a cross-platform application for smart-phones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera , physical items pla...
Conference Paper
Full-text available
Several studies explore the use of social robots in interventions for persons with cognitive disability. This paper describes ELE, a plush social robot with an elephant appearance that has been designed as a conversational companion for persons with Neuro-Developmental Disorders (NDD). ELE speaks through the live voice of a remote caregiver, enrich...
Conference Paper
Full-text available
This paper describes the result of an Usability Test over teachers' control interface in Magika, the powering engine of the Magic Room, a low-cost MultiSensory Environment (MSE) that enables new forms of playful learning for children, especially those with special needs. Magika offers a set of activities integrated in a synesthetic way; it incorpor...
Conference Paper
Full-text available
Magika is an interactive Multisensory Environment that enables new forms of playful interventions for children, especially those with Special Education Needs. Designed in cooperation with more than 30 specialists at local care centers and primary schools, Magika integrates digital worlds projected on the wall and the floor with a gamut of "smart" p...
Conference Paper
Full-text available
The demo introduces Reflex, a phy-gital game for children and adolescents with Neuro-Developmental Disorder. The game, offered through a cross-platform application for smartphones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera, physical items placed on a table. This...
Conference Paper
Full-text available
This paper describes the procedure to create an emotional data-set, named WIYE (What Is Your Emotion?), composed by semantic contents, audio and video recordings of children. Data have been collected using an interactive storytelling application, leading children aged 4 years to 12 years to discover about their emotional sphere and emotion expressi...
Conference Paper
Full-text available
Children with Autism Spectrum Disorder (ASD) often face the challenge of detecting and expressing emotions. I.e., it's hard for them to recognize happiness, sadness and anger in other people and to express their own feelings. This difficulty produces severe impairments in communication and social functioning. The paper proposes a spoken educational...
Conference Paper
Full-text available
Recent studies demonstrate how personalized and multisensory learning improves students' performances and teachers' work. Each student has different skills, aptitudes and difficulties, and the learning process requires more or less time in relation to both the topic to assimilate and the employed teaching method. Our system, called Magika, is a mul...
Chapter
Full-text available
Our research explores the potential of IoT (Internet of Things) for children with Neurodevelopmental Disorder (NDD), such as Intellectual Disability, Autism Spectrum Disorder, Down Syndrome, Attention-Deficit Hyperactivity Disorder. The paper describes an IoT empowered physical space called “Magic Room” that supports interaction with “smart” object...
Conference Paper
Full-text available
The paper describes the design of Ahù, a nomadic smart multisensory solution for children's playful learning. Ahù has a combination of features that make it unique to the current available multisensory technologies for learning: it has a totem shape, supports multimedia stimuli by communicating through voice, lights and multimedia contents projecte...
Article
Full-text available
We present in this manuscript a report on KidRec 2017 workshop, the first edition of the international workshop on Children & Recommender Systems, co-located with the 11th ACM Conference on Recommender Systems (RecSys) held in Como, Italy. While research focused on recommender systems is prolific and addresses issues pertaining to multiple domains-...
Chapter
Full-text available
“Social stories” are used in educational interventions for subjects with Neurodevelopmental Disorder (NDD) to help them gain an accurate understanding of social situations, develop autonomy and learn appropriate behavior. Traditionally, a Social Story is a short narrative that uses paper sheets, animations, or videos to describe a social situation...
Article
Full-text available
"Social stories" are used in educational interventions for subjects with Neurodevelopmental Disorders (NDD) to help them gain an accurate understanding of social situations, develop autonomy and learn appropriate behavior. Traditionally, a social story is a short narrative that uses paper sheets, animations, or videos to describe a social situation...
Conference Paper
Full-text available
This article illustrates ABBOT, a pervasive interactive game for children at the early years of primary school that aims to stimulate exploration of outdoor environments. ABBOT combines a smart tangible object to play outdoors, with a mobile app to access new content related to the discovered natural elements. The tangible object helps children cap...
Conference Paper
Full-text available
The paper introduces Reflex, a mirrored camera mobile game for children and adolescents, especially those with Neuro-Developmental Disorder. The game, offered through a cross-platform application for smart-phones and tablets, bridges the digital and the physical worlds by tracking, via a bottom-looking mirror positioned on the device camera, physic...
Conference Paper
Full-text available
Idropo is a hydroponic system for children that educates by gardening and playing. For children growing a plant can be challenging, they lose interest easily. With Idropo the cultivation time can be reduced by using an hydroponical system and the educative experience can be enhanced by interacting with a dynamic and friendly companion for gardening...
Conference Paper
Full-text available
The video describes Puffy, a friendly inflatable social robot companion for children that has been developed in cooperation with a team of psychologists and educators. Puffy has a combination of features which makes it unique with respect to existing robots.
Thesis
Full-text available
Neuro-Developmental Disorder (NDD) is an umbrella term for a group of disorders arising during the developmental period that is characterized by severe and often co-occurring deficits in the cognitive, social, communicative, motor, behavioral and emotional spheres . In this regard, the use of digital interactive technology is considered as a promis...
Article
Full-text available
The Magic Room is a smart space for use by children with neurodevelopmental disorder (NDD) and their caregivers designed in cooperation with NDD specialists and currently deployed at two therapeutic centers in Italy. It supports multimodal embodied interaction by providing controllable stimuli to the vestibular, proprioceptive, and tactile sensory...
Data
Puffy is a robotic companion that has been designed in cooperation with a team of therapists and special educators as a learning & play companion for children with Neurodevelopmental Disorder (NDD). Puffy has a combination of features that support multisensory stimuli and multimodal interaction and make this robot unique with respect to existing ro...
Conference Paper
Full-text available
Puffy is a robotic companion that has been designed in cooperation with a team of therapists and special educators as a learning & play companion for children with Neurodevelopmental Disorder (NDD). Puffy has a combination of features that support multisensory stimuli and multimodal interaction and make this robot unique with respect to existing ro...
Conference Paper
Full-text available
The 1st Workshop on Children and Recommender Systems (KidRec) is taking place in Como, Italy August 27th, 2017 in conjunction with the ACM RecSys 2017 conference. The goals of the workshop are threefold: (1) discuss and identify issues related to recommender systems used by children including specific challenges and limitations, (2) discuss possibl...
Conference Paper
Full-text available
The paper describes the design of a robot companion for children with Neurodevelopmental Disorders. This work; being informed by general design guidelines reported in the current literature, lessons learned from our own previous experience, meetings with a team of therapists and special educators, and a comparison with 21 well-known robots; shows t...
Conference Paper
Full-text available
"Social stories" are short narratives that describe everyday life situations using paper or video and are widely used in interventions for children with Neurodevelopmental Disorders (NDD), particularly in the Autistic Spectrum. Our research explores how to transform "traditional" social stories into wearable interactive immersive digital experience...
Conference Paper
Full-text available
Our research explores the potential of Wearable Immersive Virtual Reality (WIVR) as learning tool for children with disability, particularly Neurodevelopmental Disorder (NDDs). NDD is characterized by severe and often co-existing deficits in the cognitive, emotional, and motor areas. The paper discusses the learning potential of WIVR and presents t...
Conference Paper
Full-text available
XOOM is a novel interactive tool that allows non ICT-specialists to create web-based applications of Wearable Immersive Virtual Reality (WIVR) technology that use 360° realistic videos as interactive virtual tours. These applications are interesting for various domains that range from gaming, entertainment, cultural heritage, and tourism to educati...
Conference Paper
Full-text available
The paper describes the design of Puffy, a robotic companion for children with Neurodevelopmental Disorders that has been developed in cooperation with a team of therapists and special educators. Puffy has a combination of features which makes it unique with respect to existing robots for this target group. Puffy is mobile and its egg-shaped body i...
Conference Paper
Full-text available
The paper presents Polipo, a tangible toy created for children with Neurodevelopmental Disorders (NDD). Designed in collaboration with therapists, and tested with six children for five weeks, Polipo offers various opportunities for tangible interaction and a range of multisensory stimuli aimed at promoting the development of fine motor skills. One...
Conference Paper
Full-text available
While there has been a growing body of work in child-robot interaction, we still have very little knowledge regarding young children's speaking and listening dynamics and how a robot companion should decode these behaviors and encode its own in a way children can understand. In developing a backchannel prediction model based on observed nonverbal b...
Conference Paper
Full-text available
In this video, we provide an overview of the analyses, design, and evaluation of a backchannel opportunity prediction (BOP) model for a social robot listener.
Article
While there has been a growing body of work in child-robot interaction, we still have very little knowledge regarding young children's speaking and listening dynamics and how a robot companion should decode these behaviors and encode its own in a way children can understand. In developing a backchannel prediction model based on observed nonverbal b...
Conference Paper
Full-text available
Teo is a mobile robot designed for children with Neuro-Developmental Disorder (NDD). Teo's behavior can be remotely controlled by the caregiver or autonomously activated by effect of internal sensors or an external depth sensor. Teo has a soft body that can react to different types of touch (e.g., hugs, punches, or slaps); it can move freely on the...
Conference Paper
Full-text available
Our research explores wearable Immersive Virtual Reality (IVR) to support new forms of intervention for children with Neuro-Developmental Disorder (NDD). In cooperation with therapists at a local rehabilitation center, we developed and evaluated Wildcard, a system that exploits a low cost VR visor and enables the child to "feel immersed" in 3D worl...
Conference Paper
Full-text available
We propose a new emotional, huggable, mobile, and configurable robot (Teo), which can address some of the still open therapeutic needs in the treatment of Developmental Disability (DD). Teo has been designed in partnership with a team of DD specialists, and it is meant to be used as an efficient and easy-to-use tool for caregivers. Teo is integrate...
Conference Paper
Full-text available
Our research aims at supporting existing therapies for children with intellectual and developmental disorders (IDD). The personal and social autonomy is the desired end state to be achieved to enable a smooth integration in the real world. We developed and tested a framework for storytelling and learning activities that exploits an immersive virtua...
Conference Paper
Full-text available
Our research explores new forms of technology-enhanced intervention for children with Intellectual Disability (ID) that exploits our Magic K-Room, an interactive multisensory smart space designed in cooperation with therapists and special educators at a local rehabilitation center. The Magic K-Room integrates smart objects, smart lights, and (immer...
Conference Paper
Full-text available
Our research aims at helping children with intellectual disability (ID) to "learn through play" by interacting with digitally enriched physical toys. Inspired by the practice of Dolphin Therapy (a special form of Pet Therapy) and, specifically, by the activities that ID children perform at Dolphinariums, we have developed a "smart" stuffed dolphin...
Conference Paper
Full-text available
Our research aims at supporting existing therapies for children with intellectual and developmental disabilities (IDD) and with autism spectrum disorders (ASD). The personal and social autonomy is the desired end state to be achieved to enable a smooth integration in the real world. We developed and tested a framework for storytelling and learning...
Conference Paper
Our research explores new forms of technology-enhanced interventions for children with Intellectual Disability (ID). The paper presents an innovative smart space called Magic K-Room, which has been designed in cooperation with ID specialists and provides multisensory stimuli exploiting full-body interaction with various kinds of smart objects and a...
Chapter
Full-text available
In recent years we have witnessed a rapid growth of learning applications for children with different kinds of disabilities. These tools exploit different learning paradigms and employ a gamut of “beyond the desktop” interaction modes and devices, including haptic controllers, (multi)touch small and large displays, digitally augmented physical obje...
Chapter
In recent years we have witnessed a rapid growth of learning applications for children with different kinds of disabilities. These tools exploit different learning paradigms and employ a gamut of “beyond the desktop” interaction modes and devices, including haptic controllers, (multi)touch small and large displays, digitally augmented physical obje...
Conference Paper
Full-text available
We report on a nine-month-long observational study with teachers and students with autism in a classroom setting. We explore the impact of motion-based activities on students' behavior. In particular, we examine how the playful gaming activity impacted students' engagement, peer-directed social behaviors, and motor skills. We document the effective...
Conference Paper
Full-text available
The core contribution of this paper lies in exploring new spaces of interaction for children with Intellectual Developmental Disorder (IDD). In the KROG ( K inect- RO bot for G aming) Project, we blend full-body interaction, virtual worlds on large screens, motion sensing technology, and mobile robots to support game-based interventions. This paper...
Article
Full-text available
Limited studies exist that explore motionbased touchless applications for children with ASD (Autism Spectrum Disorder) and investigate their design issues and the benefits they can bring to this target group. The paper reports a structured set of design guidelines that distill our experience gained from empirical studies and collaborations with the...
Article
Full-text available
Autism spectrum disorder (ASD) has become the fastest growing disability in the United States in 2013. The disorder is characterized by a triad of symptoms related to lack of social interaction, deficits in the acquisition and expression of language, and repetitive patterns of behavior often accompanied by sensorimotor impairments. In our research,...
Article
Full-text available
Our research explores the integration of motion-based touchless interaction with human-robots interaction to support game-based learning for children with intellectual disability. The paper discusses the design challenges of this novel approach and presents the design concepts of our initial prototypes.
Article
Full-text available
With the increasingly low cost of motion-sensing technology, touchless interactive interfaces may become a new ingredient in the evolution of content intensive web applications from single-platform (desktop) to multi-platforms use. While the migration from desktop to mobile devices has been widely studied, there is limited understanding on how to i...
Thesis
Full-text available
Autism spectrum disorder (ASD) has become the fastest growing disability in the United States with current prevalence rates estimated at as many as 1 in 88 children (CDC, 2013) and an urgent need for focused and ongoing intervention. Each child with ASD behaves differently, yet the disorder is characterized by a triad of symptoms related to lack of...