Robert Porzel

University of Bremen | Uni Bremen · Computer Science

In recent years, the widespread adoption of voice user interfaces (VUIs) highlighted the growing significance of speech interaction in our everyday activities. However, researchers, designers, and developers lack a dedicated interaction model tailored to the intricacies of communication with speech agents. This paper proposes a novel interaction mo...

A challenge for automated user access to, or presentation of, event data is the fact that such data rarely explicitly controls for appropriate narrativisation choices. This is problematic because readers nevertheless read narrativisation effects into texts regardless of whether those effects were intended or not. Gaining control of such effects, wh...

Motivational dynamics in jogging constitute a pivotal factor influencing a runner's performance, persistence, and overall engagement in the running activity. The manifestation of diminished motivation is concomitant with a cascade of physiological responses, capable of being represented through biological signals, for which biosignal monitoring, a...

We present the Meta-Ontology for Introspection (MOI): Inspired by fundamental processes of the human mind, cognitive architectures (CAs) explore ever more methods to leverage metacognition. Still, an ontological model to trace metacognitive experiences for learning or as input for metacognitive control routines has yet to be developed. Based on a r...

While a growing number of households contain home assistants, they mainly remain voice-only devices where the virtual agent is not represented visually. The visual representation of the agent is limited to the device’s housing and abstract light animations that signify the assistant’s state to its users. However, the audio channel is limited in con...

Human demonstrations of everyday activities are an important resource to learn the particularities of the corresponding control strategies that are needed to perform such activities with ease and competence. However, such demonstrations need to be annotated such that time segments get associated to the appropriate actions. Previous research in psyc...

Human beings and other biological agents appear driven by curiosity to explore the affordances of their environments. Such exploration is its own reward – children have fun when playing – but it probably also serves the practical purpose of learning theories with which to predict outcomes of actions. Cognitive robots however have yet to match the p...

Due to contact restrictions, closure of fitness centers and quarantine measures, the SARS-CoV-2 pandemic led to a considerable decline of sporting activities. The first relaxation of these restrictions allowed German citizens to mostly return to their normal training and exercise behavior, yet the long-term impact of the recurring measures (i.e. th...

We propose a five steps methodology to retrieve, reconstruct and analyse conflict related narratives in a standardized and automated way. Our methodology combines AI and network analysis techniques to build a visual representation of key agents and entities involved in a conflict and to characterize their relations. Unlike the majority of existing...

Going from natural language directions to fully specified executable plans for household robots involves a challenging variety of reasoning steps. In this paper, a processing pipeline to tackle these steps for natural language directions is proposed and implemented. It uses the ontological Socio-physical Model of Activities (SOMA) as a common inter...

When playing sports in virtual reality foot interaction is crucial for many disciplines. We investigated how the visibility of the foot influences penalty shooting in soccer. In a between-group experiment, we asked 28 players to hit eight targets with a virtual ball. We measured the performance, task load, presence, ball control, and body ownership...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Despite rapid progress, cognitive robots have yet to match the facility with which humans acquire and find ways to reuse manipulation skills. An important component of human cognition seems to be our curiosity-driven exploration of our environments, which results in generalizable theories for action outcome prediction via analogical reasoning. In t...

Dealing with robotic actions in uncertain environments has been demonstrated to be hard. Many classic planning approaches to robotic action make the closed world assumption, rendering them inefficient for everyday household activities, as they function without generalizability to other contexts or the ability to deal with unexpected changes. In con...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

In this paper, we present foundations of the Socio-physical Model of Activities (SOMA). SOMA represents both the physical as well as the social context of everyday activities. Such tasks seem to be trivial for humans, however, they pose severe problems for artificial agents. For starters, a natural language command requesting something will leave m...

Verbal communication is a central component in collaborative multiplayer gaming and creates a feeling of companionship among the players. In single-player games, this aspect is often missing. Advancements in speech recognition now open new potentials for voice-activated single-player experiences. In this work, we integrated voice interaction to a s...

One of the key reasoning tasks of robotic agents is inferring possible actions that can be accomplished with a given object at hand. This cognitive task is commonly referred to as inferring the affordances of objects. In this paper, we propose a novel conceptualization of affordances and its realization as a description logic ontology. The key idea...

Many application areas ranging from serious games for health to learning by demonstration in robotics, could benefit from large body movement datasets extracted from textual instructions accompanied by images. The interpretation of instructions for the automatic generation of the corresponding motions (e.g. exercises) and the validation of these mo...

In this paper, we introduce the framework of MEANinGS for the semi-autonomous accumulation of world knowledge for robots. Where manual aggregation is inefficient and prone to incompleteness and autonomous approaches suffer from underspecified information, we deploy the human computation game Kitchen Clash and give evidence of its efficiency, comple...

We propose an approach to facilitate adjustable grip for object interaction in virtual reality. It enables the user to handle objects with loose and firm grip using conventional controllers. Pivotal design properties were identified and evaluated in a qualitative pilot study. Two revised interaction designs with variable grip were compared to the s...

Digital home assistants have an increasing influence on our everyday lives. The media now reports how children adapt the consequential, imperious language style when talking to real people. As a response to this behavior, we considered a digital assistant rebuking impolite language. We then investigated how adult users react when being rebuked by t...

In many research areas, such as the material sciences, researchers plan and conduct numerous experiments. The corresponding findings are published, while the underlying data remains described haphazardly or not at all within the respective institutes and, therefore, is neither shared nor reused by the scientific community.

Enabling multiple-purpose robots to follow textual instructions is an important challenge on the path to automating skill acquisition. In order to contribute to this goal, we work with physical exercise instructions as an everyday activity domain where textual descriptions are usually focused on body movements. Body movements are a common element a...

Instructions for physical exercises leave many details underspecified that are taken for granted and inferred by the intended reader. For certain applications, such as generating virtual action visualizations from such textual instructions, advanced text processing is needed, requiring interpretation of both implicit and explicit information. This...

This chapter focuses on three aspects of human-computer interaction (HCI) with novel and advanced materials. First, it introduces guiding interaction paradigms, user interfaces, and interaction visions within HCI research that benefit from or demand for the integration of novel materials into user interfaces. Second, the chapter presents applicatio...

Physical exercise instruction sheets are often difficult to understand. In most cases considerable information remains implicit, which also poses a considerable hurdle regarding potential machine understanding. Major missing information types include the source and destination location of a human movement. We present a Bayesian network for extracti...

Processes in manufacturing and logistics are characterized by a high frequency of changes and fluctuations, caused by the high number of participants in logistic processes. The heterogeneous landscape of data formats for information storage further complicates efforts to automatically extract process models from this data with the tools from Proces...

In this paper, we discuss the basic constraints and objectives of a unified logistics modeling language that allows to model dynamic logistic systems on various levels of concern from the requirements definition to their design and realization. It should, furthermore, support the compositionality and interoperability among all processes and other c...

Automation always plays an important role in industry. Today, it is a basic need for industry. To develop faster manufacturing or delivery, automation is an important need. Robots always play the main role for automation in the industry. Robots are mainly designed for specific task. But, the main problem is robots are too expensive for one task. Th...

Intra-operative interaction with computer systems is a challenge because of spatial constraints and also because of the need for sterility. Therefore touchless interfaces are interesting alternatives to classical interaction devices. A device that can read and convert brain signals into control and communication commands is called brain-computer-in...

In materials sciences, a large amount of research data is generated through a broad spectrum of different experiments. As of today, experimental research data including meta-data in materials science is often stored decentralized by the researcher(s) conducting the experiments without generally accepted standards on what and how to store data. The...

This paper investigates the effects of se-mantic distance on the development of lexical entrainment. For this purpose, the authors developed a card game with three levels of semantic distance. The participants were asked to arrange the cards into a congruent sequential order. By increasing the semantic distance, more words were needed to solve the...

Enabling collaborative work on multi-touch tables comes with many challenges for the design of tabletop systems. For example, multi-touch tables have been not standardized, tabletop groupware systems are built for various purposes and the diversity of task activities constitute some of the challenges for enabling natural collaborative human compute...

Reading the Chung Tzu quote presented above one might initially get the impression that it would suffice to specify the meaning of words in some way then string those meanings together and the problem of language understanding would be solved once and for all. However, as in his paradoxical remark there is a catch which has confounded researchers f...

The early ways in which knowledge has been represented semantically in spoken dialogue systems or multi-modal dialogue systems show that individual representations with different semantics and heterogeneously structured content can be found in various formats within single natural language processing systems and applications. For example, a typical...

Many areas of artificial intelligence (AI) have had to struggle in various ways in which contextual dependencies arise, e.g. knowledge representation, natural language processing or expert systems to name a few. As the importance of context is frequently glossed over in the literature, researchers noted already in the early 80s that the denotation...

Domain-specific time limits for the execution of agent-oriented knowledge management processes constitute a significant challenge for the design of autonomous logistic control with multi-agent systems. Tailored models are needed to support the agents’ decision-making, which gives rise to questions concerning the time span agents are granted to comp...

Studies on context in human language were not predominant in the influential research direction that bases on an assumed autonomy of syntax [Chomsky, 1965] and that explicitly excluded performance related issues as out of scope of their analysis [Chomsky, 1981]. These analyses focused on linguistic competence as well as innate structures and -mecha...

Firstly, I will summarize the results of this work in the light of the specified aims and intended contributions, as presented in Section 1. Therein, I stated that the central aim of this work is: ”to present a formal approach for explicating contextual information and pragmatic knowledge that can be applied, employed and evaluated in natural langu...

Endowing embodied conversational agent with personality affords more natural modalities for their interaction with human interlocutors. To bridge the personality gap between users and agents, we designed minimal two personalities for corresponding agents i.e. an introverted and an extroverted agent. Each features a combination of different verbal a...

In this paper, we investigate how discourse context in the form of short-term memory can be exploited to automatically group consecutive strokes in digital freehand sketching. With this machine learning approach, no database of explicit object representations is used for template matching on a complete scene—instead, grouping decisions are based on...

Simulation is a commonly used method, for example in economics, engineering or computer science. In many cases, simulations become difficult to design and control, especially when the simulated actors need to posses, acquire or process knowledge in some form or the other. Today simulation systems exists that are based on the multi-agent paradigm an...

This paper presents the research activities of the Collaborative Research Centre (CRC) 637 “Autonomous Cooperating Logistic Processes—A Paradigm Shift and its Limitations” at the University of Bremen. After a motivation of autonomous logistics as an answer to current trends in increasingly dynamic markets, we sketch the structure and aims of the in...

This paper presents the research activities of the Collaborative Research Centre (CRC) 637 “Autonomous Cooperating Logistic Processes—A Paradigm Shift and its Limitations” at the University of Bremen. After a motivation of autonomous logistics as an answer to current trends in increasingly dynamic markets, we sketch the structure and aims of the in...

In multiagent-based simulation systems the agent pro-gramming paradigm is adopted for simulation. This simu-lation approach offers the promise to facilitate the design and development of complex simulations, both regarding the distinct simulation actors and the simulation envi-ronment itself. We introduce the simulation middleware PlaSMA which exte...

Herein we describe the QuickWoZ system, a Wizard-of-Oz (WoZ) tool that allows for the remote control of the behavior of animated characters in a 3D environment. The complete scene, character, behaviors and sounds can be defined in simple XML documents, which are parsed at runtime, so that setting up an experiment can be done without programming exp...

Designing user interfaces for ubiquitous computing applications is a challenging task. In this paper we discuss how to build intelligent interfaces. The foundations are usability principles that are valid on very general levels. We present a number of established methods for the design process that can help to meet these principle requirements. In...

The present work and demonstration system aims at finding an efficient and cost-effective human computation method to expand the linguistic capabilities of interactive games that need it to respond appropriately to the language based input of their users. As a showcase scenario for the experiments conducted, we took interactive fiction applications...

Increased availability of mobile computing, such as personal digital assistants (PDAs), creates the potential for constant and intelligent access to up-to-date, integrated and detailed information from the Web, regardless of one's actual geographical position. Intelligent question-answering requires the representation of knowledge from various doma...

We present three types of data collections and their experimental paradigms. The resulting data were employed to conduct a number of annotation experiments, create evaluation gold standards and train statistical models. The data, experiments and their analyses highlight the importance of data-driven empirical laboratory and field work for research...

The SmartKom multimodal dialogue system offers access to a wide range of information and planning services. A significant subset of these are constituted by external data and service providers. The work presented herein describes the challenging task of integrating such external data and service sources to make them semantically accessible to other...

The approach to knowledge representation taken in the multimodal, multidomain, and multiscenario dialogue system — SmartKom — is presented. We focus on the ontological and representational issues and choices helping to construct an ontology, which is shared by multiple components of the system and can be reused in different projects and applied to...

This chapter describes the English-language SmartKom-Mobile system and related research. We explain the work required to support a second language in SmartKom and the design of the English speech recognizer. We then discuss research carried out on signal processing methods for robust speech recognition and on language analysis using the Embodied Co...

We describe the role of context models in natural language processing systems and their implementation and evaluation in the SmartKom system. We show that contextual knowledge is needed for an ensemble of tasks, such as lexical and pragmatic disambiguation, decontextualizion of domain and common-sense knowledge that was left implicit by the user an...

This paper presents SmartKom-Mobile, the mobile version of the SmartKom system. SmartKom-Mobile brings together highly advanced user interaction and mobile computing in a novel way and allows for ubiquitous access to multidomain information. SmartKom-Mobile is device-independent and realizes multimodal interaction in cars and on mobile devices such...

In this paper we address the question of how traditional approaches to modeling world knowledge, i.e. to model shared con-ceptualizations of specific domains of interest via formal ontologies, can be enhanced by a pragmatic layer to solve the problem of ex-plicating hitherto implicit information contained in the user's utter-ances and to further th...

In this paper we describe an onto-logical model of pragmatic knowl-edge -using an example from the domain of navigation -that is based on the Descriptive Ontology for Linguistic and Cognitive Engineer-ing and employs a specific onto-logical module called Descriptions & Situations. This framework es-tablishes so-called ontological pat-terns. We empl...

Recent work on natural language processing systems is aimed at more conversational, context-adaptive systems in multiple domains.

The paper presents a system architecture for the automatic generation of interface specifications from ontologies.

The approach to knowledge representation taken in a multi-modal multi-domain dialogue system - SMARTKOM - is presented. We focus on the ontological and representational issues and choices helping to construct an ontology, which is shared by multiple components of the system, can be re-used in different projects and applied to various tasks. Finally...

In this paper we present ONTOSCORE, a system for scoring sets of concepts on the basis of an ontology. We apply our system to the task of scoring alternative speech recognition hypotheses (SRH) in terms of their semantic coherence. We conducted an annotation experiment and showed that human annotators can reliably differentiate between semantically...

End-to-end evaluations of conversational dia- logue systems with naive users are currently uncovering severe usability problems that re- sult in low task completion rates. Preliminary analyses suggest that these problems are related to the system's dialogue management and turn- taking behavior. We present the results of ex- periments designed to ta...

The need for the establishment of evaluation methods that can measure respective improvements or degradations of ontological models-e.g. yielded by a precursory ontology population stage-is undisputed. We propose an evaluation scheme that allows to employ a number of different ontologies and to measure their performance on specific tasks. In this p...

Controlled and restricted dialogue systems are reliable enough to be deployed in various real world applications. The more conversational a dialogue system becomes, the more difficult and unreliable become recognition and processing. Numerous research projects are struggling to overcome the problems arising with more- or truly conversational dialog...

The approach to knowledge representation taken in a multi-modal multi-domain dialogue system -SMARTKOM -is presented. We fo-cus on the ontological and representational is-sues and choices helping to construct an ontol-ogy, which is shared by multiple components of the system, can be re-used in different projects and applied to various tasks. Finall...

Intelligent mobile systems such as Deep Map, a mobile tourist guide, require a range of intelligent components in order to actually go beyond classical systems. These components cover smart user interfaces, designed for mobile use and incorporating the use of natural language. They also include services that are location dependent, make use of cont...

We present applications enabled via the em-ployment of a single knowledge representation in the SMARTKOMmulti-modal multi-domain dialogue system. We focus on how an rig-orously constructed ontology whose ontologi-cal and representational choices are shared by multiple components of the system, can be re-used in different projects and applied to var...

Research in cognitive linguistics has yielded valuable semantic and pragmatic insights that should be incorporated by language understanding systems. These insights have not been expressed in a rigorous enough form to be implemented.

In the tourism domain, a si