Rainer Malaka

University of Bremen | Uni Bremen · Faculty 03: Mathematics/Computer Science

With the rise of popularity and societal importance of video games, trends of grinding play for excessive amounts of time to improve proficiency becomes more and more common. This bears not only physical and mental health risks, but has already shown to be detrimental for skill acquisition, defeating its purpose in the first place. For the sake of...

As the gaming industry advances, it seeks increased immersion. Concurrently, recent advancements in speech recognition substantially enhanced its accuracy, creating opportunities for speech-based interactions in gameplay. Integrating speech into single-player games with non-player characters (NPCs) introduces a new social dimension. Yet, like toxic...

In recent years, the widespread adoption of voice user interfaces (VUIs) highlighted the growing significance of speech interaction in our everyday activities. However, researchers, designers, and developers lack a dedicated interaction model tailored to the intricacies of communication with speech agents. This paper proposes a novel interaction mo...

Zusammenfassung Digitale Gesundheitsanwendungen unterstützen Nutzer:innen unter anderem dabei, ihre physische und mentale Gesundheit durch digitale Daten besser zu verstehen, und fördern dadurch ein positives Gesundheitsverhalten. Neben den staatlich geprüften digitalen Gesundheitsanwendungen (DiGA) und digitalen Pflegeanwendungen (DiPA) besteht du...

We present the Meta-Ontology for Introspection (MOI): Inspired by fundamental processes of the human mind, cognitive architectures (CAs) explore ever more methods to leverage metacognition. Still, an ontological model to trace metacognitive experiences for learning or as input for metacognitive control routines has yet to be developed. Based on a r...

While a growing number of households contain home assistants, they mainly remain voice-only devices where the virtual agent is not represented visually. The visual representation of the agent is limited to the device’s housing and abstract light animations that signify the assistant’s state to its users. However, the audio channel is limited in con...

With rapid advances in artificial intelligence and natural language processing, voice assistants are evolving into advanced digital personal assistants capable of complex tasks. As they become more proficient at understanding people's behaviors, preferences, intentions, and surroundings, opportunities for proactive interactions emerge. However, des...

Some virtual reality (VR) applications require true-to-life object manipulation, such as for training or teleoperation. We investigate an interaction technique that replicates the variable grip strength applied to a held object when using force-feedback gloves in VR. We map the exerted finger pressure to the rotational freedom of the virtual object...

Rhythm is essential in early music education. To acquire rhythm abilities, the synchronisation between body and rhythm patterns is fundamental. Extensive research has shown the link between the auditory-motor system and rhythm acquisition; however, other sensorimotor systems also improve rhythm skills. Rhythm acquisition can be trained using new te...

Conversational agents are rapidly advancing in terms of their capabilities and human likeness-both of which are intended to enhance the user experience and engagement. One human quality that can potentially increase trust and likeability is humor. However, what is considered humorous and what is not depends on many contextual and personal factors t...

VR sports applications and exergames offer opportunities to improve users’ health playfully. Evidence from various investigations on the combinations of physiological data and VR show that visualized biofeedback supports physical training in many dimensions. Display physiological data can help improve virtual exercises’ effectiveness by observing,...

The aim of this article is to more precisely define the field of research on the automation of communication, which is still only vaguely discernible. The central thesis argues that to be able to fully grasp the transformation of the media environment associated with the automation of communication, our view must be broadened from a preoccupation w...

Zusammenfassung Die zunehmende Verbreitung von Smarthome-Geräten und ihre verschiedenen Fähigkeiten bieten viele Chancen. Allerdings sind mit dieser Entwicklung auch Risiken verbunden, die insbesondere die Privatsphäre der Nutzer betreffen. Im Folgenden wird ein Überblick über die Herausforderungen gegeben, die sich den Nutzern stellen, wenn sie de...

For HCI researchers to design inclusive and equitable conversational agents, they require conceptual frameworks to identify and address iniquitous design elements in existing CUIs. A feature of such a framework would aim to recognize non-inclusive user practices, i.e., exploitation of certain cohorts' characteristics, to avoid them in future CUI sy...

Technological advances in speech recognition have enabled us to communicate with computers using our voice. This natural form of interaction opens up a large opportunity for video game companies to utilize it as an intuitive feature and appealing game mechanic for player-NPC interaction. While communication between two human actors comes naturally,...

During the COVID-19 pandemic, online meetings became common for daily teamwork in the home office. To understand the opportunities and challenges of meeting in virtual reality (VR) compared to videoconferences, we conducted the weekly team meetings of our human-computer interaction research lab on five off-the-shelf online meeting platforms over 4...

Mit diesem Beitrag wollen wir das aktuell erst in unscharfen Konturen erkennbare Forschungsfeld der Automatisierung von Kommunikation näher bestimmen. Unsere zentrale These ist hierbei, dass wir, um die mit der Automatisierung von Kommunikation verbundene Transformation unserer Medienumgebung umfassend bearbeiten zu können, den Blick von der Perspe...

Playing video games has become a major factor of spending leisure time. As competitive and e-sports games grow in popularity and size, demands of complexity, time investment and cognitive strain to acquire proficiency and keep up with competitors grow with them. Fundamental psychological work stresses the detrimental impact of sinking extensive amo...

During the Covid-19 pandemic, online meetings became common for daily teamwork in the home office. To understand the opportunities and challenges of meeting in virtual reality (VR) compared to video conferences, we conducted the weekly team meetings of our human-computer interaction research lab on five off-the-shelf online meeting platforms over f...

With the proliferation of smart home devices, domains where users' data is processed are becoming an important issue for data protection authorities. Regulations such as the EU-GDPR obligate smart home providers to inform users about the processing of personal data. However, due to the long and complex format of the privacy policies, users often ov...

The use of procedural content generation (PCG) in the context of video games has increased over the years as it provides an economical way to generate game content whilst enhancing their variety and replayability. For city-building games, this approach is often utilized to predefine map layouts, terrains, or cityscapes for the player. One core aspe...

Metaphors are often considered to be a useful tool in user interface design. They teach a user new mechanics via transferring knowledge from one domain, where the user is experienced, to a new target domain. Specifically, this is helpful in areas with advanced technologies, such as security with its encryption and access control mechanisms. At the...

The development of cooperative serious games has specific challenges related to enabling players to achieve both gaming and learning goals in a cooperative fashion. The complexity of gameplay and learning objectives leads to the lack of guidance for the systematic development of cooperative serious games (SGs). To overcome the challenges, we propos...

Due to contact restrictions, closure of fitness centers and quarantine measures, the SARS-CoV-2 pandemic led to a considerable decline of sporting activities. The first relaxation of these restrictions allowed German citizens to mostly return to their normal training and exercise behavior, yet the long-term impact of the recurring measures (i.e. th...

Going from natural language directions to fully specified executable plans for household robots involves a challenging variety of reasoning steps. In this paper, a processing pipeline to tackle these steps for natural language directions is proposed and implemented. It uses the ontological Socio-physical Model of Activities (SOMA) as a common inter...

When playing sports in virtual reality foot interaction is crucial for many disciplines. We investigated how the visibility of the foot influences penalty shooting in soccer. In a between-group experiment, we asked 28 players to hit eight targets with a virtual ball. We measured the performance, task load, presence, ball control, and body ownership...

Purpose For complex cases, preoperative surgical planning is a standard procedure to ensure patient safety and keep the surgery time to a minimum. Based on the available information, such as MRI or CT images, and prior anatomical knowledge the surgeons create their own mental 3D model of the organ of interest. This is challenging, requires years of...

The next major evolutionary stage for voice assistants will be their capability to initiate interactions by themselves. However, to design proactive interactions, it is crucial to understand whether and when this behaviour is considered useful and how desirable it is perceived for different social contexts or ongoing activities. To investigate peop...

Providing haptic feedback in virtual reality to make the experience more realistic has become a strong focus of research in recent years. The resulting haptic feedback systems differ greatly in their technologies, feedback possibilities, and overall realism making it challenging to compare different systems. We propose the Haptic Fidelity Framework...

Video game testing has become a major investment of time, labor and expense in the game industry. Particularly the balancing of in-game units, characters and classes can cause long-lasting issues that persist years after a game’s launch. While approaches incorporating artificial intelligence have already shown successes in reducing manual effort an...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Developing cooperative serious games has specific challenges of enabling players to achieve both gaming and learning goals in a cooperative fashion. The complexity of handling game and learning requirements leads to the lack of guidance for systematically developing cooperative serious games (SGs). To overcome the challenges, we propose a systemati...

Dealing with robotic actions in uncertain environments has been demonstrated to be hard. Many classic planning approaches to robotic action make the closed world assumption, rendering them inefficient for everyday household activities, as they function without generalizability to other contexts or the ability to deal with unexpected changes. In con...

The increased availability of portable devices with high computational power gave birth to such phenomenon as Bring Your Own Device (BYOD)—a situation when an employee uses his own device for accessing enterprise sensitive resources. This situation in turn created a new conflict—an employee wants to keep his data private, and an employer want to pr...

Although smart speakers support increasingly complex multi-turn dialogues, they still play a mostly reactive role, responding to user’s questions or requests. With rapid technological advances, they are becoming more capable of initiating conversations by themselves. However, before developing such proactive features, it is important to understand...

Virtual reality (VR) is, by nature, excellent in showing spatial relationships, e.g. for viewing medical 3D data. In this work, we propose a VR system to view and manipulate medical 3D images of livers in combination with 3D printed liver models as controllers. We investigate whether users benefit from a controller in the shape of a liver and if th...

Subjective self-reports in VR user studies is a burdening and often tedious task for the participants. To minimize the disruption with the ongoing experience VR research has started to administer the surveying directly inside the virtual environments. However, due to the tedious nature of text-entry in VR, most VR surveying tools focus on closed qu...

This is a case study on using VR to plan two liver surgeries.

Zusammenfassung Smart Home-Systeme finden eine immer weitere Verbreitung. Allerdings ergeben sich hieraus Fragen der IT-Sicherheit und des Datenschutzes für EndnutzerInnen. Oftmals ist nicht transparent, welche Daten wo verarbeitet werden und wie diese vor Missbrauch geschützt sind. Dieser Aufsatz beschreibt hierfür Lösungsansätze, die in dem Verbu...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

INTRODUCTION: An important phase in the process of visual design for the narrative media is previsualization (previs). Professionals use complicated 3D software applications that are not especially designed for the purpose of previs which makes it difficult for the artists and non-technical users to create previs content.OBJECTIVES: The aim is to e...

Measure user experience in MR (i.e., AR/VR) user studies is essential. Researchers apply a wide range of measuring methods using objective (e.g., biosignals, time logging), behavioral (e.g., gaze direction, movement amplitude), and subjective (e.g., standardized questionnaires) metrics. Many of these measurement instruments were adapted from use-ca...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

While robotic proficiency excels in constrained environments, the demand for vast amounts of world knowledge to cover unforeseen circumstances, constellations and tasks prevents sufficiently robust real-world application. Human computation has shown to provide successful advances to close this reasoning gap and accumulate knowledge, yet being great...

Artificial intelligence (AI) is a frequently used term-and has seen decades of use-in the video games industry. Yet, while academic AI research recently produced notable advances both in different methods and in real-world applications, the use of modern AI techniques, such as deep learning remains curiously sparse in commercial video games. Relate...

Virtual reality exposure therapy (VRET) is a promising approach in treating phobias such as fear of heights (acrophobia). VRET provides an effective, cost-efficient, scalable and individually adaptable alternative to traditional exposure therapy. To further foster the potential of VRET, a novel concept called Playful User-generated Treatment (PUT)...

Overcoming a range of challenges that traditional therapy faces, virtual reality exposure therapy (VRET) yields great potential for the treatment of phobias such as acrophobia, the fear of heights. We investigate this potential and present playful user-generated treatment (PUT), a novel game-based approach for VRET. Based on a requirement analysis...

Verbal communication is a central component in collaborative multiplayer gaming and creates a feeling of companionship among the players. In single-player games, this aspect is often missing. Advancements in speech recognition now open new potentials for voice-activated single-player experiences. In this work, we integrated voice interaction to a s...

While smart home devices are spreading rapidly, the privacy and security of users are key concerns. Many users struggle in acquiring and applying security recommendations to protect against malicious behavior in smart home systems, which can cause users to lose interest in this topic. Game-based learning is a powerful practice to increase the motiv...

One of the key reasoning tasks of robotic agents is inferring possible actions that can be accomplished with a given object at hand. This cognitive task is commonly referred to as inferring the affordances of objects. In this paper, we propose a novel conceptualization of affordances and its realization as a description logic ontology. The key idea...

Balancing the options available to players in a way that ensures rich variety and viability is a vital factor for the success of any video game, and particularly competitive multiplayer games. Traditionally, this balancing act requires extensive periods of expert analysis, play testing and debates. While automated gameplay is able to predict outcom...

Introduction: Bühler's anastomosis (or Bühler's arcade) is an embryonic relic and represents an arterio-arterial connection between the superior mesenteric artery and the celiac trunk. It can be found as a variety in 1–2% of patients. Case Presentation: We present a case of a patient with metatastatic squamous cell carcinoma of the lung. The patien...

Many application areas ranging from serious games for health to learning by demonstration in robotics, could benefit from large body movement datasets extracted from textual instructions accompanied by images. The interpretation of instructions for the automatic generation of the corresponding motions (e.g. exercises) and the validation of these mo...