Antti Oulasvirta

• PhD
• Professor (Associate) at Aalto University

During the early stages of interface design, designers need to produce multiple sketches to explore a design space. Design tools often fail to support this critical stage, because they insist on specifying more details than necessary. Although recent advances in generative AI have raised hopes of solving this issue, in practice they fail because ex...

Empirical evidence shows that typing on touchscreen devices is prone to errors and that correcting them poses a major detriment to users' performance. Design of text entry systems that better serve users, across their broad capability range, necessitates understanding the cognitive mechanisms that underpin these errors. However, prior models of typ...

To design data visualizations that are easy to comprehend, we need to understand how people with different interests read them. Computational models of predicting scanpaths on charts could complement empirical studies by offering estimates of user performance inexpensively; however, previous models have been limited to gaze patterns and overlooked...

We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our...

Computational rationality explains human behavior as arising due to the maximization of expected utility under the constraints imposed by the environment and limited cognitive resources. This simple assumption, when instantiated via partially observable Markov decision processes (POMDPs), gives rise to a powerful approach for modeling human adaptiv...

A large proportion of text is produced using mobile devices. However, very little research looks at the special characteristics of how this happens and, importantly, how it is affected by the design of the language model (LM). The operating systems of modern devices offer a number of LM-based intelligent text entry methods (ITEs) such as Autocorrec...

Attention management systems aim to mitigate the negative effects of multitasking. However, sophisticated real-time attention management is yet to be developed. We present a novel concept for attention management with reinforcement learning that automatically switches tasks. The system was trained with a user model based on principles of computatio...

Computational methods can potentially facilitate user interface design by complementing designer intuition, prior experience, and personal preference. Framing a user interface design task as a multi-objective optimization problem can help with operationalizing and structuring this process at the expense of designer agency and experience. While offe...

Designing interaction techniques for end-users often involves exploring vast design spaces while balancing many objectives. Bayesian optimization offers a principled human-in-the-loop method for selecting designs for evaluation to efficiently explore such design spaces. To date, the application of Bayesian optimization in a human-in-the-loop settin...

Computational rationality explains human behavior as arising due to the maximization of expected utility under the constraints imposed by the environment and limited cognitive resources. This simple assumption, when instantiated via Partially Observable Markov Decision Processes (POMDPs), gives rise to a powerful approach for modeling human adaptiv...

The ability to estimate the state of a human partner is an insufficient basis on which to build cooperative agents. Also needed is an ability to predict how people adapt their behavior in response to an agent's actions. We propose a new approach based on computational rationality, which models humans based on the idea that predictions can be derive...

Users often browse the web in an exploratory way, inspecting what they find interesting without a specific goal. However, the temporal dynamics of visual attention during such sessions, emerging when users gaze from one item to another, are not well understood. In this paper, we examine how people distribute visual attention among content items whe...

We need to rethink how we assist designers with artificial intelligence (AI). AI should aim to cooperate, not automate, by supporting and leveraging the creativity and problem‐solving capabilities of designers. The challenge for such AI is how to infer designers' goals and then help them without being needlessly disruptive. We introduce AI‐assisted...

Understanding the interaction between different road users is critical for road safety and automated vehicles (AVs). Existing mathematical models on this topic have been proposed based mostly on either cognitive or machine learning (ML) approaches. However, current cognitive models are incapable of simulating road user trajectories in general scena...

Interaction design typically involves challenging decision making that requires designers to consider multiple parameters and careful tradeoffs between various objectives. This article examines how AI can facilitate the process of interaction design by offloading some of the complex decision making required of designers. We study how multi-objectiv...

Adaptive user interfaces can improve experiences in Cross Reality (XR) applications by changing interface elements according to the user's context. Although extensive work explores diferent adaptation policies, XR creators often struggle with their implementation, which involves laborious manual scripting. The few available tools are underdeveloped...

Being able to describe any user interface (UI) screenshot in natural language can promote understanding of the main purpose of the UI, yet currently it cannot be accomplished with state-of-the-art captioning systems. We introduce XUI, a novel method inspired by the global precedence effect to create informative descriptions of UIs, starting with an...

Aesthetics is a central consideration in user interface design. It is known to affect end-user behavior and perception, in particular the first impression of graphical user interfaces. However, what users perceive as pleasant or good design is highly subjective. We contribute a computational model that estimates the visual appeal of a given webpage...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Intended for researchers and practitioners in interaction design, this book shows how Bayesian models can be brought to bear on problems of interface design and user modelling. It introduces and motivates Bayesian modelling and illustrates how powerful these ideas can be in thinking about human-computer interaction, especially in representing and m...

Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. In HCI, design optimization problems are often exceedingly complex, involving multiple objectives and expensive empirical evaluations. Model-based computational design algorithms assi...

A key objective in the design of graphical user interfaces (GUIs) is to ensure consistency across screens of the same product. However, designing a compliant layout is time-consuming and can distract designers from creative thinking. This paper studies layout recommendation methods that fulfill such consistency requirements using machine learning....

Affordance refers to the perception of possible actions allowed by an object. Despite its relevance to human-computer interaction, no existing theory explains the mechanisms that underpin affordance-formation; that is, how affordances are discovered and adapted via interaction. We propose an integrative theory of affordance-formation based on the t...

Gliding a finger on touchscreen to reach a target, that is, touch exploration, is a common selection method of blind screen-reader users. This paper investigates their gliding behavior and presents a model for their motor performance. We discovered that the gliding trajectories of blind people are a mixture of two strategies: 1) ballistic movements...

Users' engagement with pervasive displays has been extensively studied, however, determining how their content is interesting remains an open problem. Tracking of body postures and gaze has been explored as an indication of attention; still, existing works have not been able to estimate the interest of passers-by from readily available data, such a...

This paper demonstrates the learning process of typing by tracing the development of eye and finger movement strategies over time. We conducted a controlled experiment in which users typed with Qwerty and randomized keyboards on a smartphone, allowing us to induce and analyze users’ behavioral strategies with different amounts of accumulated typing...

AI for supporting designers needs to be rethought. It should aim to cooperate, not automate, by supporting and leveraging the creativity and problem-solving of designers. The challenge for such AI is how to infer designers' goals and then help them without being needlessly disruptive. We present AI-assisted design: a framework for creating such AI,...

Graphical user interfaces (GUIs) have gained primacy among the means of interacting with computing systems, thanks to the way they leverage human perceptual and motor capabilities. However, the design of GUIs has mostly been a manual activity. To design a GUI, the designer must select its visual, spatial, textual, and interaction properties such th...

Over the past decade, responsive web design (RWD) has become the de facto standard for adapting web pages to a wide range of devices used for browsing. While RWD has improved the usability of web pages, it is not without drawbacks and limitations: designers and developers must manually design the web layouts for multiple screen sizes and implement...

This essay addresses a foundational topic in applied sciences with interest in design: how do theories inform design? Previous work has attributed theory-use to abduction and deduction. However, design is about creating an intervention, a possible state that does not exist presently, and these accounts fail to explain how theories permit taking thi...

The paper presents a novel model-based method for intelligent tutoring, with particular emphasis on the problem of selecting teaching interventions in interaction with humans. Whereas previous work has focused on either personalization of teaching or optimization of teaching intervention sequences, the proposed indi- vidualized model-based planning...

Designers are increasingly using online resources for inspiration. How to best support design exploration without compromising creativity? We introduce and study Design Maps, a class of point-cloud visualizations that makes large user interface datasets explorable. Design Maps are computed using dimensionality reduction and clustering techniques, w...

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user -- for example, due to surprise or relearning effort -- or "trap" the process to a suboptimal design immaturely. However, effects on users are hard to predict a...

The problem of computational design for menu systems has been addressed in some specific cases such as the linear menu (list). The classical approach has been to model this problem as an assignment task, where commands are assigned to menu positions while optimizing for users’ selection performance and grouping of associated items. However, we show...

A new French keyboard standard is the first designed with the help of computational methods.

For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of ex...

2020 Copyright for this paper by its author(s). In the context of data visualization and analytics, this report outlines some of the challenges and emerging applications that arise in the Big Data era. In particularly, fourteen distinguished scientists from academia and industry, and diverse related communities, i.e., Information Visualization, Hum...

How do people decide how long to continue in a task, when to switch, and to which other task? It is known that task interleaving adapts situationally, showing sensitivity to changes in expected rewards, costs, and task boundaries. However, the mechanisms that underpin the decision to stay in a task versus switch away are not thoroughly understood....

Objective The objective was to better understand how people adapt multitasking behavior when circumstances in driving change and how safe versus unsafe behaviors emerge. Background Multitasking strategies in driving adapt to changes in the task environment, but the cognitive mechanisms of this adaptation are not well known. Missing is a unifying a...

To personalize a web page, case-specific rules or templates must be specified that define the visuospatial layout of elements as well as device-specific adaptation rules for an individual. This approach scales poorly. We present LaaS, a service platform for self-optimizing web layouts to improve their usability at individual, group, and population...

Computational design of menu systems has been solved in limited cases such as the linear menu (list) as an assignment task, where commands are assigned to menu positions while optimizing for for users selection performance and distance of associated items. We show that this approach falls short with larger, hierarchically organized menu systems, wh...

A central concern in an interactive intelligent system is optimization of its actions, to be maximally helpful to its human user. In recommender systems for instance, the action is to choose what to recommend, and the optimization task is to recommend items the user prefers. The optimization is done based on earlier user's feedback (e.g. "likes" an...

Relatively little is known about eye and finger movement in typing with mobile devices. Most prior studies of mobile typing rely on log data, while data on finger and eye movements in typing come from studies with physical keyboards. This paper presents new findings from a transcription task with mobile touchscreen devices. Movement strategies were...