Christian Wallraven

Christian Wallraven
Korea University | KU · Department of Brain and Cognitive Engineering

Dr. rer. nat.
Going AR/MR/VR!

About

252
Publications
47,621
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
3,871
Citations
Citations since 2016
74 Research Items
1755 Citations
2016201720182019202020212022050100150200250300
2016201720182019202020212022050100150200250300
2016201720182019202020212022050100150200250300
2016201720182019202020212022050100150200250300
Introduction
Our perceptual and cognitive abilities still place us humans far ahead of artificial systems. In my research I combine methods from computer vision, computer graphics, and machine learning with approaches from cognitive neuroscience to develop novel frameworks for analyzing information processing of the human cognitive system. Based on these results, I also design and improve artificial, cognitive systems as well as human-machine-interfaces.
Additional affiliations
September 2019 - present
Korea University
Position
  • Professor (Full)
March 2010 - present
Korea University
Position
  • Professor (Full)
March 2010 - present
Korea University
Position
  • Professor (Associate)
Description
  • Graduate and Undergraduate Teaching Classes on: (1) Applied Mathematics (including introduction to Matlab programming), (2) Visual Perception, (3) Cognitive and Computational Psychophysics, (4) Introduction to Brain and Cognitive Sciences
Education
August 2000 - November 2006
University of Tuebingen
Field of study
  • Physics
October 1998 - October 1999
University of Tuebingen
Field of study
  • Physics

Publications

Publications (252)
Conference Paper
Full-text available
Interaction with its environment is a key requisite for a humanoid robot. Especially the ability to recognize and manipulate unknown objects is crucial to successfully work in natural environments. Visual object recognition, however, still remains a challenging problem, as three-dimensional objects often give rise to ambiguous, two-dimensional view...
Preprint
Current supervised frameworks for facial landmark detection require a large amount of training data and due to the massive number of parameters may suffer from overfitting to the specific datasets. We introduce a semi-supervised method in which the crucial idea is to first generate implicit knowledge about the face appearance & shape from the large...
Article
Full-text available
Previous Electroencephalography (EEG) and neuroimaging studies have found differences between brain signals for subsequently remembered and forgotten items during learning of items - it has even been shown that single trial prediction of memorization success is possible with a few target items. There has been little attempt, however, in validating...
Article
What happens if we put vision and touch into conflict? Which modality “wins”? Although several previous studies have addressed this topic, they have solely focused on integration of vision and touch for low-level object properties (such as curvature, slant, or depth). In the present study, we introduce a multimodal mixed-reality setup based on real...
Preprint
Full-text available
Current powertrains in vehicles offer a broad spectrum of driving modes that shape the driving experience. One crucial factor for determining the human factors aspects of this driving experience is the perception and evaluation of acceleration-a complex, multisensory process that integrates auditory, vestibular, and visual input. Two important ques...
Article
Full-text available
Whenever we touch a surface with our fingers, we perceive distinct tactile properties that are based on the underlying dynamics of the interaction. However, little is known about how the brain aggregates the sensory information from these dynamics to form abstract representations of textures. Earlier studies in surface perception all used general s...
Chapter
The performance of predicting biological markers from brain scans has rapidly increased over the past years due to the availability of open datasets and efficient deep learning algorithms. There are two concerns with these algorithms, however: they are black-box models, and they can suffer from over-fitting to the training data due to their high ca...
Article
Full-text available
Why do some people tend to drive faster than others? Personality characteristics such as the evaluation of risk to oneself or to others, impulsivity, adherence to norms, but also other personal factors such as gender, age, or driving experience all may play a role in determining how fast people drive. Since driving speed is a critical factor underl...
Chapter
Facial expression recognition (FER) is a topic attracting significant research in both psychology and machine learning with a wide range of applications. Despite a wealth of research on human FER and considerable progress in computational FER made possible by deep neural networks (DNNs), comparatively less work has been done on comparing the degree...
Chapter
The deep learning field is growing rapidly as witnessed by the exponential growth of papers submitted to journals, conferences, and pre-print servers. To cope with the sheer number of papers, several text mining tools from natural language processing (NLP) have been proposed that enable researchers to keep track of recent findings. In this context,...
Chapter
Recent research has found that knowledge distillation can be effective in reducing the size of a network and in increasing generalization. A pre-trained, large teacher network, for example, was shown to be able to bootstrap a student model that eventually outperforms the teacher in a limited label environment. Despite these advances, it still is re...
Chapter
Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually “understand” the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. I...
Chapter
AffectNet is one of the most popular resources for facial expression recognition (FER) on relatively unconstrained in-the-wild images. Given that images were annotated by only one annotator with limited consistency checks on the data, however, label quality and consistency may be limited. Here, we take a similar approach to a study that re-labeled...
Article
Full-text available
The steady, world-wide increase in myopia prevalence in children over the past decades has raised concerns. As an early intervention for axial-length-related myopia, correcting lenses have been developed (such as Defocus Incorporated Multiple Segment (DIMS) lenses), which have been shown to be effective in slowing myopia progression. Beyond this di...
Preprint
Full-text available
Deep neural networks (DNNs) have become remarkably successful in data prediction, and have even been used to predict future actions based on limited input. This raises the question: do these systems actually "understand" the event similar to humans? Here, we address this issue using videos taken from an accident situation in a driving simulation. I...
Preprint
Full-text available
Recent research has found that knowledge distillation can be effective in reducing the size of a network and in increasing generalization. A pre-trained, large teacher network, for example, was shown to be able to bootstrap a student model that eventually outperforms the teacher in a limited label environment. Despite these advances, it still is re...
Preprint
Full-text available
Facial expression recognition (FER) is a topic attracting significant research in both psychology and machine learning with a wide range of applications. Despite a wealth of research on human FER and considerable progress in computational FER made possible by deep neural networks (DNNs), comparatively less work has been done on comparing the degree...
Preprint
Full-text available
AffectNet is one of the most popular resources for facial expression recognition (FER) on relatively unconstrained in-the-wild images. Given that images were annotated by only one annotator with limited consistency checks on the data, however, label quality and consistency may be limited. Here, we take a similar approach to a study that re-labeled...
Preprint
Full-text available
The deep learning field is growing rapidly as witnessed by the exponential growth of papers submitted to journals, conferences, and pre-print servers. To cope with the sheer number of papers, several text mining tools from natural language processing (NLP) have been proposed that enable researchers to keep track of recent findings. In this context,...
Article
Full-text available
Faces can be categorized in various ways, for example as male or female or as belonging to a specific biogeographic ancestry (race). Here we tested the importance of the main facial features for race perception. We exchanged inner facial features (eyes, mouth or nose), face contour (everything but those) or texture (surface information) between Asi...
Article
Full-text available
The operant conditioning has been less studied than the classical conditioning as a mechanism of placebo-like effect, and two distinct learning mechanisms have never been compared to each other in terms of their neural activities. Twenty-one participants completed cue-learning based pain rating tasks while their brain responses were measured using...
Article
Full-text available
We investigated whether enhanced interoceptive bodily states of fear would facilitate recognition of the fearful faces. Participants performed an emotional judgment task after a bodily imagery task inside a functional magnetic resonance imaging scanner. In the bodily imagery task, participants were instructed to imagine feeling the bodily sensation...
Preprint
Full-text available
We propose an encoder-decoder framework for the segmentation of blood vessels in retinal images that relies on the extraction of large-scale patches at multiple image-scales during training. Experiments on three fundus image datasets demonstrate that this approach achieves state-of-the-art results and can be implemented using a simple and efficient...
Article
Our work for the first time evaluates the effectiveness of visual and acoustic warning systems in an accident situation using a realistic, immersive driving simulation. In a first experiment, 70 participants were trained to complete a course at high speed. The course contained several forks where a wrong turn would lead to the car falling off a cli...
Chapter
We propose an encoder-decoder framework for the segmentation of blood vessels in retinal images that relies on the extraction of large-scale patches at multiple image-scales during training. Experiments on three fundus image datasets demonstrate that this approach achieves state-of-the-art results and can be implemented using a simple and efficient...
Conference Paper
Current supervised methods for facial landmark detection require a large amount of training data and may suffer from overfitting to specific datasets due to the massive number of parameters. We introduce a semi-supervised method in which the crucial idea is to first generate implicit face knowledge from the large amounts of unlabeled images of face...
Article
Full-text available
The question whether facial expression processing may be impaired in Parkinson’s disease (PD) patients so far has yielded equivocal results – existing studies, however, have focused on testing expression processing in recognition tasks with static images of six standard, emotional facial expressions. Given that non-verbal communication contains bot...
Article
The present study applied multi-voxel pattern analysis (MVPA) to decode spatial discrimination in pain perception to acupuncture needle from brain functional magnetic resonance image (fMRI). Fourteen participants were stimulated by acupuncture needles at two adjacent body parts on their left forearm (PC6 vs. HT7). We trained support vector machines...
Article
Full-text available
Objective: The cognitive, Affective, and Somatic Empathy Scale (CASES) suggests novel three components structure of empathy. Although CASES developed to assess the trait empathy of both children and adult, the validation in adult sample is not yet conducted. This study developed Korean version of CASES (CASES-K) and examined the psychometric prope...
Conference Paper
Full-text available
Both vision and touch contribute to the perception of real surfaces. Although there have been many studies on the individual contributions of each sense, it is still unclear how each modality’s information is processed and integrated. To fill this gap, we investigated the similarity of visual and haptic perceptual spaces, as well as how well they e...
Preprint
Full-text available
Recent progress in deep learning frameworks has enabled impressive recognition performance with discriminative networks as well as the synthesis of realistic images using generative networks. Despite these advances, the resulting solutions to both discriminative and generative tasks exist separately and often lack interpretability and explainabilit...
Article
Full-text available
A "masked face", that is, decreased facial expression is considered as one of the cardinal symptoms among individuals with Parkinson's disease (PD). Both spontaneous and voluntary mimicry toward others' emotional expressions is essential for both social communication and emotional sharing with others. Despite many studies showing impairments in fac...
Article
Emergency situations during car driving sometimes force the driver to make a sudden decision. Predicting these decisions will have important applications in updating risk analyses in insurance applications, but also can give insights for drafting autonomous vehicle guidelines. Studying such behavior in experimental settings, however, is limited by...
Article
Full-text available
Many situations require decisions to be made in very little time—in emergency or accident situations such decisions will carry potentially harmful consequences. Can we predict how people react in such situations from their personality traits alone? Since experimental tests of accident situations are not possible in the real world, existing studies...
Chapter
The use of EEG to enhance learning experience in learning environments can contribute to furthering symbiotic relationship between the user and the system. This study examines whether it is possible to predict successful memorization of previously-learned words in a language learning context from brain activity alone. Participants are tasked with l...
Article
A large number of perceptual and cognitive processes are instantiated during active gameplay, culminating in what is termed the overall “gaming experience” which encapsulates multiple, subjective dimensions of how one feels about the game. Although some research has been conducted into the neural mechanisms underlying the gaming experience, previou...
Thesis
Purpose: The aim of this study is to develop a novel deep learning system for vessel segmentation of retinal images. We present a recurrent Convolutional Neural Network (CNN) architecture and compare performance with existing CNN approaches, showing greatly reduced processing time with excellent performance. Methods: The proposed DirectNet archite...
Article
Full-text available
Viewpoint selection has been an emerging area in computer graphics for some years, and it is now getting maturity with applications in fields such as scene navigation, scientific visualization, object recognition, mesh simplification, and camera placement. In this survey, we review and compare twenty-two measures to select good views of a polygonal...
Poster
Loss of central vision - such as those from glaucoma or macular degeneration - affects peoples' ability to process objects and faces, leading to severe impairments. Here, we study how the brain may adapt to this situation using a simulated vision loss. We find that repeated training over the course of 9 sessions leads to high performance in an obje...
Article
Full-text available
Touch delivers a wealth of information already from birth, helping infants to acquire knowledge about a variety of important object properties using their hands. Despite the fact that we are touch experts as much as we are visual experts, surprisingly, little is known how our perceptual ability in touch is linked to either functional or structural...
Article
The discovery that pleasant touch is coded by C-tactile fibers has generated considerable research interest and increased understanding of the skin as a channel for social information via cutaneous senses. However, no study has differentiated between the pleasant response to visual and tactile non-human stimulations. Our study investigated pleasant...
Conference Paper
Emotions are central to all human experiences. Inducing and measuring emotions, however, has been one of the most difficult tasks, since the nature of emotions includes rapid changes and heterogeneity across different individuals. Until very recently, traditional methods for inducing emotional states mostly relied on the presentation of predefined...
Article
Background, aims and objectives: We investigated the effects of perceived trustworthiness based on doctors’ faces on economic and medical decisions in an experimental setting using expanded versions of Trust Games. Methods: First, participants indicated how much they trusted the doctor on the screen in a game in which trust was measured by whether...
Article
Full-text available
The emotion-associated bodily sensation map is composed of a specific topographical distribution of bodily sensations to categorical emotions. The present study investigated whether or not interoceptive accuracy was associated with topographical changes in this map following emotion-induced bodily sensations. This study included 31 participants who...
Data
The location of ROIs for each emotion. Colored area of each body template indicates the location of the ROI in each emotion. The bodily sensation magnitude was estimated through the average of the extracted values from the ROIs. Emotion-specific ROIs were defined as the circular area with a radius of 10 pixels from the peak points of significant bo...
Data
Visualization of emotion-specific bodily sensation map by stimuli types (GIF images and dynamic facial expressions). Cluster corrected results at p < 0.05 level are presented as in the main results of Fig 2. Statistical values were transformed into Z-scores, which indicated the significance of a sensation at the group level, and then mapped on a fr...
Article
A large number of perceptual and cognitive processes are activated during game play. Previous studies have identified seven core dimensions of such processes: competence, immersion, flow, tension, challenge, and positive and negative affect. Although research has begun to investigate neural correlates of individual dimensions, the brain patterns re...
Article
Although humans are experts for visual shape processing, several recent studies have demonstrated that haptics can also create highly detailed shape representations. Here, we investigate how vision and touch are processed when both are put in a potentially noticeable conflict in a shape similarity judgment task. To have full control over the parame...
Article
Full-text available
Research on the perception of facial emotional expressions (FEEs) often uses static images that do not capture the dynamic character of social coordination in natural settings. Recent behavioral and neuroimaging studies suggest that dynamic FEEs (videos or morphs) enhance emotion perception. To identify mechanisms associated with the perception of...
Data
Ensemble of supporting information figures and tables. It contains supporting information Figures A-H, and Tables A-G as referred to in the text. (PDF)
Data
Unnatural dynamic happy expression of one of the actresses. (MOV)
Data
Natural dynamic happy expression of one of the actresses. (MOV)
Data
Natural static happy expression of one of the actresses. (MOV)
Article
Full-text available
Placebo analgesia can be primarily explained by the Pavlovian conditioning paradigm in which a passively applied cue becomes associated with less pain. In contrast, instrumental conditioning employs an active paradigm that might be more similar to clinical settings. In the present study, an instrumental conditioning paradigm involving a modified tr...
Conference Paper
Full-text available
Peoples' risk-taking behavior varies from timid and careful, low-risk individuals to bold and careless, high-risk individuals. Can we use EEG to predict who is who? In the present study, we use the balloon analogue risk task (BART) in an EEG experiment in order to find out potential correlates in the EEG signal that allow us to distinguish high ris...