Hatice Kose

Istanbul Technical University · Faculty of Computer and Informatics

Reinforcement learning (RL) enables an agent to learn by trial and error while interacting with a dynamic environment. Traditionally, RL is used to learn and predict Euclidean robotic manipulation skills like positions, velocities, and forces. However, in robotics, it is common to have non-Euclidean data like orientation or stiffness, and neglectin...

This paper presents a contrastive learning-based facial action unit detection system for children with hearing impairments to be used on a socially assistive humanoid robot platform. The spontaneous facial data of children with hearing impairments was collected during an interaction study with Pepper humanoid robot, and tablet-based game. Since the...

There have been many studies on intelligent robotic systems for patients with motor impairments, where different sensor types and different human-machine interface (HMI) methods have been developed. However, these studies fail to achieve complex activity detection at the minimum sensing level. In this paper, exploratory approaches are adopted to in...

Social navigation is beneficial for mobile robots in human inhabited areas. In this paper, we focus on smooth path tracking and handling disruptions during plan execution in social navigation. We extended the social force model (SFM)-based local planner to achieve smooth and effective social path following. The SFM-based local motion planner is use...

The automatic emotion recognition domain brings new methods and technologies that might be used to enhance therapy of children with autism. The paper aims at the exploration of methods and tools used to recognize emotions in children. It presents a literature review study that was performed using a systematic approach and PRISMA methodology for rep...

Children with autism spectrum disorder (ASD) have deficits in the socio-communicative domain and frequently face severe difficulties in the recognition and expression of emotions. Existing literature suggested that children with ASD benefit from robot-based interventions. However, studies varied considerably in participant characteristics, applied...

This study presents an assistive robotic system enhanced with emotion recognition capabilities for children with hearing disabilities. The system is designed and developed for the audiometry tests and rehabilitation of children in a clinical setting and includes a social humanoid robot (Pepper), an interactive interface, gamified audiometry tests,...

This paper presents a new data collection tool for sign language recognition with Leap Motion Controller. The main motivation of this study is to develop assistive technology in order to support hearing-impaired people. This study is a part of RoboRehab project where we aim to develop an affectively aware robotic rehabilitation platform to assist h...

Audio is one of the essential elements of game design. Even in virtual reality applications, visuals have been thought to have more importance to create impressiveness. However, using its stereo or surround capabilities, it is possible to improve the effectiveness of the game, on the players. Also, creating an audio rich game increases its accessib...

In this paper, several physical activity-based human–computer interaction (HCI) games which are developed and implemented for the improvement of attention, emotion, and sensory–motor coordination will be presented. The interface and the difficulty levels of these games are specially designed for the use of people with different age groups and disab...

The coronavirus disease (COVID-19) pandemic affected our lives deeply, just like everyone else, the children also suffered from the restrictions due to COVID-19 affecting their education and social interactions with others, being restricted from play areas and schools for a long time. Although social robots provide a promising solution to support c...

Sensor-based human activity classification requires time and frequency domain feature extraction techniques. The set of choice in time and frequency domain features may have a significant impact on the overall classification accuracy. Another problem is to train deep learning models with sufficient dataset. The use of generative models eliminates t...

This paper introduces an emotion recognition system for an affectively aware hospital robot for children, and a data labeling and processing tool called LabelFace for facial expression recognition (FER) to be employed within the presented system. The tool provides an interface for automatic/manual labeling and visual information processing for emot...

This paper presents a study of an audiovisual interface-based drumming system for multimodal human–robot interaction. The interactive multimodal drumming game is used in conjunction with humanoid robots to establish an audiovisual interactive interface. This study is part of a project to design robot and avatar assistants for education and therapy,...

In human-populated environments, robot navigation requires more than mere obstacle avoidance for safe and comfortable human-robot interaction. Socially aware navigation approaches become vital for deploying mobile service robots in human interactive environments, where the robot operates in interaction with human implicitly or explicitly. These app...

This paper presents a sign recognition system for a sign tutoring assistive humanoid robot. In this study, a specially designed 5-fingered robot platform with expressive face (Robovie R3) is used for interaction and communication with deaf or hard of hearing children using signs and visual cues. The robot is able to recognize and generate accuratel...

This article describes the procedures employed during the development of the first comprehensive machine-readable Turkish Sign Language (TiD) resource: a bilingual lexical database and a parallel corpus between Turkish and TiD. In addition to sign language specific annotations (such as non-manual markers, classifiers and buoys) following the recent...

In this paper, several kinect-based games which are developed and implemented for the improvement of attention and sensory-motor coordination will be presented. The interface, and difficulty levels of these games are specially designed for the ease of different age groups. The games involve physical activities for the fulfillment of some basic task...

Socially aware robot navigation aims to generate human-friendly paths in human-robot interactive environments. This paper focuses on maintaining humans' physical safety and mental comfort during the robot navigation in an unknown dynamic environment. To achieve this goal, we use a variant of a pedestrian model called Collision Prediction based Soci...

In this study, an electronic chest strap prototype was designed for measuring fitness level, performance optimization, mobility and fall detection. Knitting technology is used for production by using highly elastic nylon yarn. In order to evaluate comfort performance of the garment, yarn strength and elongation, air permeability, moisture managemen...

Sports garments with functional properties have become crucial as well as comfort properties since they improve the wearer performance. For this reason, sport vest and short having high elastic recovery with fall detection sensor, were designed and produced by using flat-bed knitting machine. Comfort properties of short and vest were tested with se...

The current study proposes a novel cognitive architecture for a computational model of the limbic system, inspired by human brain activity, which improves interactions between a humanoid robot and preschool children using joint attention during turn-taking gameplay. Using human-robot interaction (HRI), this framework may be useful for ameliorating...

In this paper, interaction between humans and robots and for rehabilitation applications in social areas is investigated. People suffering from some disorders require better nursing service for interacting socially. Additionally, utilizing human-robot interaction (HRI), this proposed architecture may be a suitable resolution for problems related to...

This paper presents an assistive robotic system, which can recognize and express sign language words from a predefined set, within interactive games to communicate with and teach hearing-impaired children sign language. The robotic system uses audio, visual and tactile feedback for interaction with the children and the teacher/researcher.

The aim of this study is to present usable educational tools of students with special needs. Hearing impaired students cannot use text and speech based technological, educational material that is becoming a crucial tool for modern education. Like videos of signers, a signing avatar for T˙ID (Turkish Sign Language) would allow communicating informat...

In this study, a rule based heuristic method is proposed to recognize the primitive hand shapes of Turkish Sign Language (TID) which are sensed by a Leap Motion device. The hand shape data set was also tested with selected machine learning method (Random Forest), and the results of two approaches were compared. The proposed system required less dat...

The aim of this project is to establish the communication between the hearing impaired people and non-sign-language-speaking people easier and to raise the awareness of the community about hearing impaired people. In this study, a subset of selected primitive hand shapes of Turkish Sign Language are recognized by a system based on Leap Motion devic...

This paper presents a system that enables the drum strokes to be automatically detected and recognized using auditory cues. In pursuit of creating an interactive drumming game, in which a human user is able to play drums with a humanoid robot (Robotic Drum Mate) relying on turn-taking and imitation principles, in this work, we specifically focus on...

This work is part of a project for sign language tutoring with imitation based interactive game, iSign1. An assistive social humanoid robot (R3) is accompanying deaf children in the interaction game. The robot interacts with children using visual modules, including sign recognition and sign generation. This paper focuses on upper torso self collisi...

This paper presents an on-going project which aims to assist in Sign Language (SL) tutoring to hearing impaired/deaf children by means of interaction games with humanoid robots. The robots employed in this project are able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL). The robots also have the ability to recog...

This paper presents interactive games for sign language tutoring assisted by humanoid robots. The games are specially designed for children with communication impairments. In this study, different robot platforms such as a Nao H25 and a Robovie R3 humanoid robots are used to express a set of chosen signs in Turkish Sign Language using hand and arm...

This paper presents a socially interactive humanoid robot-assisted system for sign language (SL) tutoring for children with communication impairments by means of imitation-based interaction games. In this study, a five-fingered robot platform Robovie R3 is used to express a set of chosen words in Turkish sign language (TSL) using hand and body move...

This study aims to spread the use of humanoid robots in teaching sign language to hearing impaired people. To track the sign language gestures, Microsoft Kinect that has a ability to record the structure of the human skeletal joints position information is used. To implement and learn sign language gestures on humanoid robot from human motion data,...

Previous research on social interaction among humans suggested that interpersonal motor coordination can help to establish social rapport. Our research addresses the question of whether, in a human–humanoid interaction experiment, the human’s overall perception of a robot can be improved by realizing motor coordination behavior that allows the robo...

This paper investigates the role of interaction and communication kinesics in human-robot interaction. It is based on a project on Sign Language (SL) tutoring through interaction games with humanoid robots. The aim of the study is to design a computational framework, which enables to motivate the children with communication problems (i.e., ASD and...

This paper presents a humanoid robot assisted imitation based interactive game for Sign Language (SL) tutoring. The game is specially designed for children with communication impairments. The work presented is a part of Robot Sign Language Tutor* project. In this study, a Robovie R3 humanoid robot is used to express a set of chosen words in SL usin...

This paper presents results from interactive robotic games based on a multimodal user interface. This game is intended to be used as part of children's education and therapy, within a project based on human-robot and humancomputer interaction games featuring non-verbal gestures and imitation for the purpose of teaching sign language. The humanoid r...

This work aims at creating a funny and educatory application where disabled children are able to improve their memory and color recognition skills, by designing a game based on music and motion. In this application, it is decided to develop an interactive game based on drums which satisfy purposes of the work, because of ease of use and the combina...

This work is part of a project for imitation based turn-taking and interaction games for children with communication impairments. The paper focuses on a music based interaction game mainly for children with autism. Autism Spectrum Disorder (ASD) involves communication impairments, limited social interaction, and limited imagination. Music can facil...

This paper presents a system that enables the drum beats to be automatically detected using auditory cues and will be used ina drum-based human-robot interaction game. In pursuit of creating an interactive drumming game, where a human user is able to play drums with a humanoid robot relying on turn-taking and imitation principles, in this work, we...

This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games with humanoid robots. The aim of the project is to motivate the children with communication problems to understand and imitate the signs implemented by the humanoid robots. In this study, we implemented selected words from Turkish...

This paper investigates the role of interaction and communication kinesics in human–robot interaction. This study is part of a novel research project on sign language (SL) tutoring through interaction games with humanoid robots. The main goal is to motivate the children with communication problems to understand and imitate the signs implemented by...

This work presents the preliminary results of an ongoing project which aims to use humanoid robots as sign language tutors. The study mainly focuses on children who have some problems when they are communicating with other individuals such as hearing-impaired or autistic children. In this paper, an interactive game, which is based on sign language,...

People having disabilities (especially hearing impaired) need better interaction with their environment and humanoid robots can help in this mission. This is a challenging issue because high level linguistic capabilities are required for communication and the humanoid robot should realize sign language processing in the case of interaction with hea...

This paper presents preliminary results from a socially interactive humanoid robot assisted Sign Language (SL) tutoring system for children with communication impairments, by means of imitation based interaction games. In this study, the 5 fingered robot platform Robovie R3 will be able to express a set of chosen words in SL using hand movements, b...

Abstract—The work presented in this paper was carried out within an on-going project which aims to assist in Sign language tutoring to children with communication problems by means of interaction games with a humanoid robot. In this project, the robot is able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL), a...

This paper presents preliminary results from a nonverbal interactive game between a humanoid robot and participants based on Sign Language. The study consists of an imitation-based learning phase and the game phase where the signs taught in the first phase are tested within an interaction game. The aim of this interactive game is to reinforce the s...

Abstract—We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a “form-of-life”’ in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful framework for the construction of...

This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games (iSign) with humanoid robots and children with communication impairments. The paper focuses on the extension of the game, mainly for children with autism. Autism Spektrum Disorder (ASD) involves communication impairments, limited s...

This work is part of an ongoing work for sign language tutoring with imitation based turn-taking and interaction games (iSign) with humanoid robots and children with communication impairments. The paper focuses on the extension of the game mainly for children with autism. Autism Spectrum Disorder (ASD) involves communication impairments, limited so...

The results are from an on-going study which aims to assist in the teaching of Sign Language (SL) to hearing impaired children by means of non-verbal communication and imitation based interaction games between a humanoid robot and the child. In this study, the robot will be able to express a word in the SL among a set of chosen words using hand mov...

In this paper, we describe a NAO H25 humanoid robot painter assisted by a human. The aim of this study is to reproduce the whole painting process by a humanoid robot with a vision system and fingers. The novelty of the study lies in using a human assistant in interaction with the robot and filling regions in the picture. The painting process is per...

This work proposes a novel approach for introducing market-driven multi-agent collaboration strategy with Q-Learning based behavior assignment mechanism to the robot soccer domain in order to solve issues related to multi- agent coordination. Robot soccer difiers from many other multi-agent problems with its highly dynamic and complex nature. Marke...