Hatice Kose

Hatice Kose
Istanbul Technical University · Faculty of Computer and Informatics

Professor

About

102
Publications
18,598
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,413
Citations
Introduction
The main purpose of this project is to teach sign language to children with communication disabilities by means of interaction games with social humanoid robots.
Additional affiliations
April 2015 - present
Istanbul Technical University
Position
  • Professor (Associate)
March 2010 - present
Istanbul Technical University
Position
  • Professor (Assistant)
October 2006 - January 2010
University of Hertfordshire
Position
  • Researcher
Description
  • Research fellow working on non-verbal interaction and sensory-motor coordination @ Robotcub project. Worked on Kaspar and iCub robots

Publications

Publications (102)
Article
Full-text available
Social navigation is beneficial for mobile robots in human inhabited areas. In this paper, we focus on smooth path tracking and handling disruptions during plan execution in social navigation. We extended the social force model (SFM)-based local planner to achieve smooth and effective social path following. The SFM-based local motion planner is use...
Article
Full-text available
The automatic emotion recognition domain brings new methods and technologies that might be used to enhance therapy of children with autism. The paper aims at the exploration of methods and tools used to recognize emotions in children. It presents a literature review study that was performed using a systematic approach and PRISMA methodology for rep...
Article
Full-text available
Children with autism spectrum disorder (ASD) have deficits in the socio-communicative domain and frequently face severe difficulties in the recognition and expression of emotions. Existing literature suggested that children with ASD benefit from robot-based interventions. However, studies varied considerably in participant characteristics, applied...
Article
Full-text available
This study presents an assistive robotic system enhanced with emotion recognition capabilities for children with hearing disabilities. The system is designed and developed for the audiometry tests and rehabilitation of children in a clinical setting and includes a social humanoid robot (Pepper), an interactive interface, gamified audiometry tests,...
Chapter
This paper presents a new data collection tool for sign language recognition with Leap Motion Controller. The main motivation of this study is to develop assistive technology in order to support hearing-impaired people. This study is a part of RoboRehab project where we aim to develop an affectively aware robotic rehabilitation platform to assist h...
Chapter
Audio is one of the essential elements of game design. Even in virtual reality applications, visuals have been thought to have more importance to create impressiveness. However, using its stereo or surround capabilities, it is possible to improve the effectiveness of the game, on the players. Also, creating an audio rich game increases its accessib...
Article
Full-text available
In this paper, several physical activity-based human–computer interaction (HCI) games which are developed and implemented for the improvement of attention, emotion, and sensory–motor coordination will be presented. The interface and the difficulty levels of these games are specially designed for the use of people with different age groups and disab...
Preprint
The coronavirus disease (COVID-19) pandemic affected our lives deeply, just like everyone else, the children also suffered from the restrictions due to COVID-19 affecting their education and social interactions with others, being restricted from play areas and schools for a long time. Although social robots provide a promising solution to support c...
Chapter
This paper introduces an emotion recognition system for an affectively aware hospital robot for children, and a data labeling and processing tool called LabelFace for facial expression recognition (FER) to be employed within the presented system. The tool provides an interface for automatic/manual labeling and visual information processing for emot...
Article
Full-text available
Sensor-based human activity classification requires time and frequency domain feature extraction techniques. The set of choice in time and frequency domain features may have a significant impact on the overall classification accuracy. Another problem is to train deep learning models with sufficient dataset. The use of generative models eliminates t...
Article
This paper presents a study of an audiovisual interface-based drumming system for multimodal human–robot interaction. The interactive multimodal drumming game is used in conjunction with humanoid robots to establish an audiovisual interactive interface. This study is part of a project to design robot and avatar assistants for education and therapy,...
Article
Full-text available
In human-populated environments, robot navigation requires more than mere obstacle avoidance for safe and comfortable human-robot interaction. Socially aware navigation approaches become vital for deploying mobile service robots in human interactive environments, where the robot operates in interaction with human implicitly or explicitly. These app...
Article
Full-text available
This paper presents a sign recognition system for a sign tutoring assistive humanoid robot. In this study, a specially designed 5-fingered robot platform with expressive face (Robovie R3) is used for interaction and communication with deaf or hard of hearing children using signs and visual cues. The robot is able to recognize and generate accuratel...
Article
Full-text available
This article describes the procedures employed during the development of the first comprehensive machine-readable Turkish Sign Language (TiD) resource: a bilingual lexical database and a parallel corpus between Turkish and TiD. In addition to sign language specific annotations (such as non-manual markers, classifiers and buoys) following the recent...
Chapter
In this paper, several kinect-based games which are developed and implemented for the improvement of attention and sensory-motor coordination will be presented. The interface, and difficulty levels of these games are specially designed for the ease of different age groups. The games involve physical activities for the fulfillment of some basic task...
Conference Paper
Full-text available
Socially aware robot navigation aims to generate human-friendly paths in human-robot interactive environments. This paper focuses on maintaining humans' physical safety and mental comfort during the robot navigation in an unknown dynamic environment. To achieve this goal, we use a variant of a pedestrian model called Collision Prediction based Soci...
Article
Full-text available
In this study, an electronic chest strap prototype was designed for measuring fitness level, performance optimization, mobility and fall detection. Knitting technology is used for production by using highly elastic nylon yarn. In order to evaluate comfort performance of the garment, yarn strength and elongation, air permeability, moisture managemen...
Article
Full-text available
Sports garments with functional properties have become crucial as well as comfort properties since they improve the wearer performance. For this reason, sport vest and short having high elastic recovery with fall detection sensor, were designed and produced by using flat-bed knitting machine. Comfort properties of short and vest were tested with se...
Article
Full-text available
The current study proposes a novel cognitive architecture for a computational model of the limbic system, inspired by human brain activity, which improves interactions between a humanoid robot and preschool children using joint attention during turn-taking gameplay. Using human-robot interaction (HRI), this framework may be useful for ameliorating...
Conference Paper
In this paper, interaction between humans and robots and for rehabilitation applications in social areas is investigated. People suffering from some disorders require better nursing service for interacting socially. Additionally, utilizing human-robot interaction (HRI), this proposed architecture may be a suitable resolution for problems related to...
Article
This paper presents an assistive robotic system, which can recognize and express sign language words from a predefined set, within interactive games to communicate with and teach hearing-impaired children sign language. The robotic system uses audio, visual and tactile feedback for interaction with the children and the teacher/researcher.
Conference Paper
Full-text available
The aim of this study is to present usable educational tools of students with special needs. Hearing impaired students cannot use text and speech based technological, educational material that is becoming a crucial tool for modern education. Like videos of signers, a signing avatar for T˙ID (Turkish Sign Language) would allow communicating informat...
Research
Full-text available
In this study, a rule based heuristic method is proposed to recognize the primitive hand shapes of Turkish Sign Language (TID) which are sensed by a Leap Motion device. The hand shape data set was also tested with selected machine learning method (Random Forest), and the results of two approaches were compared. The proposed system required less dat...
Research
Full-text available
The aim of this project is to establish the communication between the hearing impaired people and non-sign-language-speaking people easier and to raise the awareness of the community about hearing impaired people. In this study, a subset of selected primitive hand shapes of Turkish Sign Language are recognized by a system based on Leap Motion devic...
Conference Paper
This paper presents a system that enables the drum strokes to be automatically detected and recognized using auditory cues. In pursuit of creating an interactive drumming game, in which a human user is able to play drums with a humanoid robot (Robotic Drum Mate) relying on turn-taking and imitation principles, in this work, we specifically focus on...
Conference Paper
This work is part of a project for sign language tutoring with imitation based interactive game, iSign1. An assistive social humanoid robot (R3) is accompanying deaf children in the interaction game. The robot interacts with children using visual modules, including sign recognition and sign generation. This paper focuses on upper torso self collisi...
Conference Paper
This paper presents an on-going project which aims to assist in Sign Language (SL) tutoring to hearing impaired/deaf children by means of interaction games with humanoid robots. The robots employed in this project are able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL). The robots also have the ability to recog...
Article
This paper presents interactive games for sign language tutoring assisted by humanoid robots. The games are specially designed for children with communication impairments. In this study, different robot platforms such as a Nao H25 and a Robovie R3 humanoid robots are used to express a set of chosen signs in Turkish Sign Language using hand and arm...
Article
This paper presents a socially interactive humanoid robot-assisted system for sign language (SL) tutoring for children with communication impairments by means of imitation-based interaction games. In this study, a five-fingered robot platform Robovie R3 is used to express a set of chosen words in Turkish sign language (TSL) using hand and body move...
Article
This study aims to spread the use of humanoid robots in teaching sign language to hearing impaired people. To track the sign language gestures, Microsoft Kinect that has a ability to record the structure of the human skeletal joints position information is used. To implement and learn sign language gestures on humanoid robot from human motion data,...
Article
Previous research on social interaction among humans suggested that interpersonal motor coordination can help to establish social rapport. Our research addresses the question of whether, in a human–humanoid interaction experiment, the human’s overall perception of a robot can be improved by realizing motor coordination behavior that allows the robo...
Chapter
This paper investigates the role of interaction and communication kinesics in human-robot interaction. It is based on a project on Sign Language (SL) tutoring through interaction games with humanoid robots. The aim of the study is to design a computational framework, which enables to motivate the children with communication problems (i.e., ASD and...
Conference Paper
This paper presents a humanoid robot assisted imitation based interactive game for Sign Language (SL) tutoring. The game is specially designed for children with communication impairments. The work presented is a part of Robot Sign Language Tutor* project. In this study, a Robovie R3 humanoid robot is used to express a set of chosen words in SL usin...
Conference Paper
This paper presents results from interactive robotic games based on a multimodal user interface. This game is intended to be used as part of children's education and therapy, within a project based on human-robot and humancomputer interaction games featuring non-verbal gestures and imitation for the purpose of teaching sign language. The humanoid r...
Conference Paper
This work aims at creating a funny and educatory application where disabled children are able to improve their memory and color recognition skills, by designing a game based on music and motion. In this application, it is decided to develop an interactive game based on drums which satisfy purposes of the work, because of ease of use and the combina...
Conference Paper
Full-text available
This work is part of a project for imitation based turn-taking and interaction games for children with communication impairments. The paper focuses on a music based interaction game mainly for children with autism. Autism Spectrum Disorder (ASD) involves communication impairments, limited social interaction, and limited imagination. Music can facil...
Conference Paper
This paper presents a system that enables the drum beats to be automatically detected using auditory cues and will be used ina drum-based human-robot interaction game. In pursuit of creating an interactive drumming game, where a human user is able to play drums with a humanoid robot relying on turn-taking and imitation principles, in this work, we...
Conference Paper
This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games with humanoid robots. The aim of the project is to motivate the children with communication problems to understand and imitate the signs implemented by the humanoid robots. In this study, we implemented selected words from Turkish...
Article
This paper investigates the role of interaction and communication kinesics in human–robot interaction. This study is part of a novel research project on sign language (SL) tutoring through interaction games with humanoid robots. The main goal is to motivate the children with communication problems to understand and imitate the signs implemented by...
Conference Paper
This work presents the preliminary results of an ongoing project which aims to use humanoid robots as sign language tutors. The study mainly focuses on children who have some problems when they are communicating with other individuals such as hearing-impaired or autistic children. In this paper, an interactive game, which is based on sign language,...
Conference Paper
Full-text available
People having disabilities (especially hearing impaired) need better interaction with their environment and humanoid robots can help in this mission. This is a challenging issue because high level linguistic capabilities are required for communication and the humanoid robot should realize sign language processing in the case of interaction with hea...
Conference Paper
This paper presents preliminary results from a socially interactive humanoid robot assisted Sign Language (SL) tutoring system for children with communication impairments, by means of imitation based interaction games. In this study, the 5 fingered robot platform Robovie R3 will be able to express a set of chosen words in SL using hand movements, b...
Conference Paper
Full-text available
Abstract—The work presented in this paper was carried out within an on-going project which aims to assist in Sign language tutoring to children with communication problems by means of interaction games with a humanoid robot. In this project, the robot is able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL), a...
Conference Paper
Full-text available
This paper presents preliminary results from a nonverbal interactive game between a humanoid robot and participants based on Sign Language. The study consists of an imitation-based learning phase and the game phase where the signs taught in the first phase are tested within an interaction game. The aim of this interactive game is to reinforce the s...
Conference Paper
Full-text available
Abstract—We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a “form-of-life”’ in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful framework for the construction of...
Conference Paper
Full-text available
This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games (iSign) with humanoid robots and children with communication impairments. The paper focuses on the extension of the game, mainly for children with autism. Autism Spektrum Disorder (ASD) involves communication impairments, limited s...
Conference Paper
Full-text available
This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games (iSign) with humanoid robots and children with communication impairments. The paper focuses on the extension of the game mainly for children with autism. Autism Spectrum Disorder (ASD) involves communication impairments, limited so...
Article
The results are from an on-going study which aims to assist in the teaching of Sign Language (SL) to hearing impaired children by means of non-verbal communication and imitation based interaction games between a humanoid robot and the child. In this study, the robot will be able to express a word in the SL among a set of chosen words using hand mov...
Conference Paper
In this paper, we describe a NAO H25 humanoid robot painter assisted by a human. The aim of this study is to reproduce the whole painting process by a humanoid robot with a vision system and fingers. The novelty of the study lies in using a human assistant in interaction with the robot and filling regions in the picture. The painting process is per...
Article
Full-text available
This work proposes a novel approach for introducing market-driven multi-agent collaboration strategy with Q-Learning based behavior assignment mechanism to the robot soccer domain in order to solve issues related to multi- agent coordination. Robot soccer difiers from many other multi-agent problems with its highly dynamic and complex nature. Marke...
Conference Paper
Full-text available
This study is a part of an ongoing project which aims to assist in teaching Sign Language (SL) to hearing-impaired children by means of non-verbal communication and imitation-based interaction games between a humanoid robot and a child. In this paper, the problem is geared towards a robot learning to imitate basic upper torso gestures (SL signs) us...
Poster
Full-text available
This study aims to assist in teaching Sign Language (SL) to hearing-impaired children by means of non-verbal communication and imitation-based interaction games between a humanoid robot and a child.