Jim Torresen

• University of Oslo

Robots have the potential to provide everyday life care and support for senior adults, but acceptance is essential for successful implementation in the domestic environment. Nonverbal social behavior can enhance this acceptance, and behavioral cues should be easy and intuitive to understand. However, which factors contribute to senior adults’ intui...

Congruence of verbal and nonverbal behavior of a robot is essential to establish a seamless and natural interaction between humans and robots. We have investigated the preference and user experience of the senior participants that a robot approaches with expressive movement that is tailored or incongruent to the message the robot brings. Twelve eld...

Robots operating in the real world will experience a range of different environments and tasks. It is essential for the robot to have the ability to adapt to its surroundings to work efficiently in changing conditions. Evolutionary robotics aims to solve this by optimizing both the control and body (morphology) of a robot, allowing adaptation to in...

In acoustic instruments, sound production relies on the interaction between physical objects. Digital musical instruments, on the other hand, are based on arbitrarily designed action-sound mappings. This paper describes the ongoing exploration of an empirically-based approach for simulating guitar playing technique when designing the mappings of 'a...

There are many different ways a robot can move in Human-Robot Interaction. One way is to use techniques from film animation to instruct the robot to move. This article is a systematic literature review of human-robot trials, pilots, and evaluations that have applied techniques from animation to move a robot. Through 27 articles, we find that animat...

The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modula...

Collaborative robots are becoming more common on factory floors as well as regular environments, however, their safety still is not a fully solved issue. Collision detection does not always perform as expected and collision avoidance is still an active research area. Collision avoidance works well for fixed robot-camera setups, however, if they are...

The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modula...

The complexity of a legged robot's environment or task can inform how specialised its gait must be to ensure success. Evolving specialised robotic gaits demands many evaluations - acceptable for computer simulations, but not for physical robots. For some tasks, a more general gait, with lower optimization costs, could be satisfactory. In this paper...

The widespread adoption of mobile devices, such as smartphones and tablets, has made touchscreens a common interface for musical performance. New mobile musical instruments have been designed that embrace collaborative creation and that explore the affordances of mobile devices, as well as their constraints. While these have been investigated from...

Despite the proven advantages of sampling-based motion planning algorithms, their inability to handle online navigation tasks and providing low-cost solutions make them less efficient in practice. In this paper, a novel sampling-based algorithm is proposed which is able to plan in an unknown environment and provides solutions with lower cost in ter...

Gaining a better understanding of how and what machine learning systems learn is important to increase confidence in their decisions and catalyze further research. In this paper, we analyze the predictions made by a specific type of recurrent neural network, mixture density RNNs (MD-RNNs). These networks learn to model predictions as a combination...

We examine how robot movement can help human-robot interaction in the context of a robot helping people over 60-years old at home. Many people are not familiar with a robot moving in their home. We present four movement conditions to classify movement between a human and robot at home. Using phenomenology and familiarity, we recognize some of these...

In order to preserve genomic stability, cells rely on various repair pathways for removing DNA damage. The mechanisms how enzymes scan DNA and recognize their target sites are incompletely understood. Here, by using high-localization precision microscopy along with 133 Hz high sampling rate, we have recorded EndoV and OGG1 interacting with 12-kbp e...

There are many different ways a robot can move in Human-Robot Interaction. One way is to use techniques from film animation to instruct the robot to move. This article is a systematic literature review of human-robot trials, pilots, and evaluations that have applied techniques from animation to move a robot. Through 27 articles, we find that animat...

Although deep neural networks have achieved state-of-the-art performance in several artificial intelligence applications in the past decade, they are still hard to understand. In particular, the features learned by deep networks when determining whether a given input belongs to a specific class are only implicitly described concerning a considerabl...

Personal and ubiquitous sensing technologies such as smartphones have allowed the continuous collection of data in an unobtrusive manner. Machine learning methods have been applied to continuous sensor data to predict user contextual information such as location, mood, physical activity, etc. Recently, there has been a growing interest in leveragin...

Wearable sensors measuring different parts of peo-ple's activity are a common technology nowadays. Data created using these devices holds a lot of potential besides measuring the quantity of daily steps or calories burned, since continuous recordings of heart rate and activity levels usually are collected. Furthermore, there is an increasing awaren...

For robots to handle the numerous factors that can affect them in the real world, they must adapt to changes and unexpected events. Evolutionary robotics tries to solve some of these issues by automatically optimizing a robot for a specific environment. Most of the research in this field, however, uses simplified representations of the robotic syst...

Elderly care at home is a matter of great concern if the elderly live alone, since unforeseen circumstances might occur that affect their well-being. Technologies that assist the elderly in independent living are essential for enhancing care in a cost-effective and reliable manner. Elderly care applications often demand real-time observation of the...

With the emergence of ubiquitous sensing technologies, it is now possible to continuously monitor users during their everyday activities in order to provide personalized feedback and interventions. For example, fitness trackers can count steps and motivate users to keep active. With their rich set of sensors, smartphones are also capable of monitor...

This paper describes the process of developing a standstill performance work using the Myo gesture control armband and the Bela embedded computing platform. The combination of Myo and Bela allows a portable and extensible version of the standstill performance concept while introducing muscle tension as an additional control parameter. We describe t...

In this paper, we present three implementations of an online evolvable hardware classifier of sonar signals on a 28 nm process technology FPGA, and compare their features using the most relevant metrics in the design of hardware: area, timing, power consumption, energy consumption, and performance. The three implementations are: one full-hardware i...

Musical performance requires prediction to operate instruments, to perform in groups and to improvise. We argue, with reference to a number of digital music instruments (DMIs), including two of our own, that predictive machine learning models can help interactive systems to understand their temporal context and ensemble behaviour. We also discuss h...

A significant problem of using deep learning techniques is the limited amount of data available for training. There are some datasets available for the popular problems like item recognition and classification or self-driving cars, however, it is very limited for the industrial robotics field. In previous work, we have trained a multi-objective Con...

For robots to handle the numerous factors that can affect them in the real world, they must adapt to changes and unexpected events. Evolutionary robotics tries to solve some of these issues by automatically optimizing a robot for a specific environment. Most of the research in this field, however, uses simplified representations of the robotic syst...

Wearable sensors measuring different parts of people's activity are a common technology nowadays. In research, data collected using these devices also draws attention. Nevertheless, datasets containing sensor data in the field of medicine are rare. Often, data is non-public and only results are published. This makes it hard for other researchers to...

Evolutionary robotics has aimed to optimize robot control and morphology to produce better and more robust robots. Most previous research only addresses optimization of control, and does this only in simulation. We have developed a four-legged mammal-inspired robot that features a self-reconfiguring morphology. In this paper, we discuss the possibi...

We introduce a new self-contained and self-aware interface for musical expression where a recurrent neural network (RNN) is integrated into a physical instrument design. The system includes levers for physical input and output, a speaker system, and an integrated single-board computer. The RNN serves as an internal model of the user's physical inpu...

The field of collaborative robotics and human-robot interaction often focuses on the prediction of human behaviour, while assuming the information about the robot setup and configuration being known. This is often the case with fixed setups, which have all the sensors fixed and calibrated in relation to the rest of the system. However, it becomes a...

Robots need to be able to adapt to complex and dynamic environments for widespread adoption, and adapting the body might yield more flexible and robust robots. Previous work on dynamic robot morphology has focused on simulation, combining simple modules, or switching between locomotion modes. This paper presents an alternative approach: automatic s...

Robots need to be able to adapt to complex and dynamic environments for widespread adoption, and adapting the body might yield more flexible and robust robots. Previous work on dynamic robot morphology has focused on simulation, combining simple modules, or switching between locomotion modes. This paper presents an alternative approach: automatic s...

In recent years, there has been increased attention on the possible impact of future robotics and AI systems. Prominent thinkers have publicly warned about the risk of a dystopian future when the complexity of these systems progresses further. These warnings stand in contrast to the current state-of-the-art of the robotics and AI technology. This a...

Many works in collaborative robotics and human-robot interaction focuses on identifying and predicting human behaviour while considering the information about the robot itself as given. This can be the case when sensors and the robot are calibrated in relation to each other and often the reconfiguration of the system is not possible, or extra manua...

RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to their short improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJa...

RoboJam is a machine-learning system for generating music that assists users of a touchscreen music app by performing responses to theirshort improvisations. This system uses a recurrent artificial neural network to generate sequences of touchscreen interactions and absolute timings, rather than high-level musical notes. To accomplish this, RoboJam...

A depth camera-based novel method is proposed here for efficient facial expression recognition. For each pixel in a depth image, eight local directional strengths are obtained and ranked. Once the rank of all pixels are obtained, eight histograms are developed for the eight surrounding directions. The histograms are then concatenated to represent f...

For many, the pursuit and enjoyment of musical performance goes hand-in-hand with collaborative creativity, whether in a choir, jazz combo, orchestra, or rock band. However, few musical interfaces use the affordances of computers to create or enhance ensemble musical experiences. One possibility for such a system would be to use an artificial neura...

In this paper we compare a selection of artificial neural networks when applied for short-term stock market price prediction. The networks are selected due to their expected relevance to the problem. Further, the work aims at covering recent advances in the field of artificial neural networks. The networks considered include: Feed forward neural ne...

Touch-screen musical performance has become commonplace since the widespread adoption of mobile devices such as smartphones and tablets. However, mobile digital musical instruments are rarely designed to emphasise collab-orative musical creation, particularly when it occurs between performers who are separated in space and time. In this article, we...

Falls in homes of the elderly, in residential care facilities and in hospitals commonly occur in close proximity to the bed. Most approaches for recognizing falls use cameras, which challenge privacy, or sensor devices attached to the bed or the body to recognize bedside events and bedside falls. We use data collected from a ceiling mounted 80 × 60...

MicroJam is a mobile app for sharing tiny touch-screen performances. Mobile applications that streamline creativity and social interaction have enabled a very broad audience to develop their own creative practices. While these apps have been very successful in visual arts (particularly photography), the idea of social music-making has not had such...

This work proposes a depth camera-based robust facial expression recognition (FER) system that can be adopted for better human machine interaction. Although video-based facial expression analysis has been focused on by many researchers, there are still various problems to be solved in this regard such as noise due to illumination variations over ti...

With 3D sensing becoming cheaper, environment-aware and visually-guided robot arms capable of safely working in collaboration with humans will become common. However, a reliable calibration is needed, both for camera internal calibration, as well as Eye-to-Hand calibration, to make sure the whole system functions correctly. We present a framework,...

The field of evolutionary robotics shows great promise, but is held back by the lack of results applicable to real world problems or other research fields. The reality gap effects present when moving from virtual to real robots makes evolution based on simulation inefficient for continuous adaption to changing morphology or environments. Evolution...

With advancing technologies, robotic manipulators and visual environment sensors are becoming cheaper and more widespread. However, robot control can be still a limiting factor for better adaptation of these technologies. Robotic manipulators are performing very well in structured workspaces, but do not adapt well to unexpected changes, like people...

This paper proposes a novel approach for human activity recognition based on body part histograms and Hidden Markov Models. From a depth video frame, body parts are segmented first using a trained random forest. Then, a histogram for each body part is combined to represent histogram features for a depth image. The depth video activity features are...

Designing and operating computing and communication systems are becoming increasingly challenging tasks, due to a multitude of reasons. First, compute nodes are evolving towards parallel and heterogeneous architectures to realise performance gains while minimising their power consumption. Progress in micro(nano)- electronics allows us to integrate...

In this book we have argued that in order to deal with the complexities of future computing systems, including size, decentralisation, uncertainty, dynamics and heterogeneity, greater levels of self-awareness on the part of such systems will be important. While this has long been agreed in principle, only now have we begun to establish a principled...

Self-aware and self-expressive technologies may be used to improve user experience in interactive music systems. This chapter presents how the concepts and techniques from the first parts of the book may be exploited to develop better technologies for active music. Nature-inspired and socially-inspired methods, as introduced in Chapter 7, are used...

With 3D sensing becoming cheaper, environment-aware robot arms capable of safely working in collaboration with humans will become common. However, a reliable calibration is needed, both for camera internal calibration, as well as Eye-to-Hand calibration, to make sure the whole system functions correctly. We present a framework, using a novel combin...

Taking inspiration from self-awareness in humans, this book introduces the new notion of computational self-awareness as a fundamental concept for designing and operating computing systems. The basic ability of such self-aware computing systems is to collect information about their state and progress, learning and maintaining models containing know...

Work on human self-awareness is the basis for a framework to develop computational systems that can adaptively manage complex dynamic tradeoffs at runtime. An architectural case study in cloud computing illustrates the framework's potential benefits.

Systems that gather unpredictable input data while responding and self-adapting in uncertain environments are transforming our relationship with and use of computers. This issue of Computer explores a variety of approaches and applications for such systems.

Run-time reconfiguration has the potential to allow reuse of resources and the reduce cost of FPGA-based systems. To compute feasible placement locations for PR modules in such systems, multiple constraints have to be evaluated. This includes unused area, placement of heterogeneous resources and communication requirements of the PR module. To impro...

A system for decentralized synchronization of musical agents is presented, inspired by Mirollo and Strogatz' pulse-coupled oscillator model of the synchronous flashing of certain species of firefly. While most previous work on pulse-coupled oscillators assume fixed and (close to) equal oscillator frequencies, the presented system tackles the challe...

This paper presents a novel methodology for generating and compressing configuration bitstreams for modules that can be executed at different positions of an FPGA. The presented methodology for bitstream generation and compression does not need deep knowledge of the bitstream format and it is independent of the target (Xilinx) FPGA family. The appr...

The facility layout planning (FLP) and the job shop scheduling problem (JSSP) are two major design issues that impact on the efficiency and productivity of manufacturing systems. The interactions between these two combinatorial optimization problems are widely known. Although, a great deal of research has been focused on solving these problems, rel...

To fully exploit the capabilities of runtime reconfigurable FPGAs in self-aware systems, design tools are required that exceed the capabilities of present vendor design tools. Such tools must allow the implementation of scalable reconfigurable systems with various different partial modules that might be loaded to different positions of the device a...

Run-time reconfiguration provides an opportunity to increase performance, reduce cost and improve energy efficiency in FPGA-based systems. However, run-time reconfigurable systems are more complex to implement than static only systems. This increases time to market, and introduces run-time overhead into the system. Our research aims to raise the ab...

In this paper, we present an open source partial reconfiguration (PR) system which is designed for portability and usability serving as a reference for engineers and students interested in using the advanced reconfiguration capabilities available in Xilinx FPGAs. This includes design aspects such as floorplanning and interfacing PR modules as well...

form only given. The design and development of digital electronic systems is mainly performed by use of a hardware description language. To prepare students in electrical engineering for a career in hardware design many universities provide courses on VHDL. The traditional approach in teaching VHDL is mainly by means of textbook examples and simula...

GOAHEAD is a tool for easily building complex run-time reconfigurable systems. The tool provides sophisticated features like module relocation, hierarchical reconfiguration, or reusing modules among different systems. This demonstration shows 1) how reconfigurable systems can be built using GOAHEAD with only a few mouse clicks. In addition, 2) we w...

A heterogeneous system with soft CPU tailored to the individual threads of the application, while still software based, offers the potential for improved performance and resource utilization over a homogeneous system. In this paper we present a method to automatically create a heterogeneous multi-core system from a multithreaded software applicatio...

Synchronization is an important part of collaborative music systems, and with such systems implemented on mobile devices, the implementation of algorithms for synchronization without central control becomes increasingly important. Decentralised synchronization has been researched in many areas, and some challenges are solved. However, some of the a...

This paper proposes an approach to representing robot morphology and control, using a two-level description linked to two different physical axes of development. The bioinspired encoding produces robots with animal-like bilateral limbed morphology with co-evolved control parameters using a central pattern generator-based modular artificial neural n...

Constraint satisfaction modeling is both an efficient, and an elegant approach to model and solve many real world problems. In this paper, we present a constraint solver targeting module placement in static and partial run-time reconfigurable systems. We use the constraint solver to compute feasible placement positions. Our placement model incorpor...

Partial run-time reconfiguration has brought forward a new dimension and many new possibilities when designing systems. However, it also leads to many new challenges that need to be addressed for partial run-time reconfiguration to be successful. One of the most significant challenges is how to perform functional verification of systems using parti...