Edward Lank

• University of Waterloo

Pointing is an elementary interaction in virtual and augmented reality environments, and, to effectively support selection, techniques must deal with the challenges of occlusion and depth specification. Most of the previous techniques require two explicit steps to handle occlusion. In this paper, we propose Conductor, an intuitive, plane-ray, inter...

Due to the proliferation of smart wearables, it is now the case that designers can explore novel ways that devices can be used in combination by end-users. In this paper, we explore the gestural input enabled by the combination of smart earbuds coupled with a proximal smartwatch. We identify a consensus set of gestures and a taxonomy of the types o...

Emerging input techniques that rely on sensing and recognition can misinterpret a user's intention, resulting in errors and, potentially, a negative user experience. To enhance the development of such input techniques, it is valuable to understand implications of these errors, but they can very costly to simulate. Through two controlled experiments...

One common task when controlling smart displays is the manipulation of video timelines. Given current examples of smart displays that support distant bare hand control, in this paper we explore CD Gain functions to support both seeking and scrubbing tasks. Through a series of experiments, we demonstrate that a linear CD Gain function provides perfo...

Virtual hand or pointer metaphors are among the key approaches for target selection in immersive environments. However, targeting moving objects is complicated by factors including target speed, direction, and depth, such that a basic implementation of these techniques might fail to optimize user performance. We present results of two empirical stu...

Target acquisition in an occluded environment is challenging given the omni-directional and first-person view in virtual reality (VR). We propose Solar-Casting, a global scene filtering technique to manage occlusion in VR. To improve target search, users control a reference sphere centered at their head through varied occlusion management modes: Hi...

BACKGROUND While in many contexts unsuccessful games targeting learning, social interaction, or behavioural change have few downsides, when covering a sensitive domain such as mental health, care must be taken to avoid harm and stigmatization of people who live with mental health conditions. As a result, evaluation of the game to identify benefits...

Background Although in many contexts unsuccessful games targeting learning, social interaction, or behavioral change have few downsides, when covering a sensitive domain such as mental health (MH), care must be taken to avoid harm and stigmatization of people who live with MH conditions. As a result, evaluation of the game to identify benefits and...

People often switch hands while holding their phones, based on task and context. Ideally, we would be able to detect which hand they are using to hold the device, and use this information to optimize the interaction. We introduce a method to use built-in orientation sensors to detect which hand is holding a smartphone prior to first interaction. Ba...

In this paper, the effect of social exposure on smartphone motion gestures is investigated. A within-subject repeated measures experiment was conducted where participants performed sets of motion gestures on a smartphone in both private and public locations. Using data from the smartphone's accelerometer, we found that the location had a significan...

Crowdworkers regularly support their work with scripts, extensions, and software to enhance their productivity. Despite their evident significance, little is understood regarding how these tools affect crowdworkers' quality of life and work. In this study, we report findings from an interview study (N=21) aimed at exploring the tooling practices us...

While mid-air gestures are an attractive modality with an extensive research history, one challenge with their usage is that the gestures are not self-revealing. Scaffolding techniques to teach these gestures are difficult to implement since the input device, e.g. a hand, wand or arm, cannot present the gestures to the user. In contrast, for touch...

Intensive exploration and navigation of hierarchical lists on smartphones can be tedious and time-consuming as it often requires users to frequently switch between multiple views. To overcome this limitation, we present PinchList, a novel interaction design that leverages pinch gestures to support seamless exploration of multi-level list items in h...

In this paper, we explore an alternative form of volunteer work, PledgeWork, where individuals, rather than working directly for a charity, make indirect donations by completing tasks provided by a third party task provider. PledgeWork poses novel research questions on issues of user acceptance of on-line volunteerism, on quality and quantity of wo...

Many touch based interactions provide limited opportunities for direct tactile feedback; examples include multi-user touch displays, augmented reality based projections on passive surfaces, and mid-air input. In this paper, we consider distal feedback, through vibrotactile stimulation on a smart-watch placed on the user's non-dominant wrist, as an...

We introduceCrushed It!, an interactive game on a sensor floor. This floor is combined with a multiple projector system to reduce occlusions from players' interactions with the floor. Individual displays, an HTC Vive to track player position, and smart watches were added to provide an extra layer of interactivity. We created this interactive experi...

In gesture recognition, one challenge that researchers and developers face is the need for recognition strategies that mediate between false positives and false negatives. In this article, we examine bi-level thresholding, a recognition strategy that uses two thresholds: a tighter threshold limits false positives and recognition errors, and a loose...

The popularity of head-worn displays (HWD) technologies such as Virtual Reality (VR) and Augmented Reality (AR) headsets is growing rapidly. To predict their commercial success, it is essential to understand the acceptability of these new technologies, along with new methods to interact with them. In this vein, the evaluation of social acceptabilit...

Many smartwatches have a digital crown as a complementary input modality to the touchscreen. While the crown eliminates issues of touchscreen occlusion and imprecision, i.e., the 'fat-finger' problem, the crown input is limited as it only supports bi-directional rotations along a single dimension. We present TiltCrown, a crown prototype that increa...

Recent research in haptic feedback is motivated by the crucial role that tactile perception plays in everyday touch interactions. In this paper, we describe psychophysical experiments to investigate the perceptual threshold of individual fingers on both the right and left hand of right-handed participants using active dynamic touch for spatial peri...

Large displays are becoming commonplace at work, at home, or in public areas. However, interaction at a distance -- anything greater than arms-length -- remains cumbersome, restricts simultaneous use, and requires specific hardware augmentations of the display: touch layers, cameras, or dedicated input devices. Yet a rapidly increasing number of pe...

Despite the ubiquity of touch-based input and the availability of increasingly computationally powerful touchscreen devices, there has been comparatively little work on enhancing basic canonical gestures such as swipe-to-pan and pinch-to-zoom. In this paper, we introduce transient pan and zoom, i.e. pan and zoom manipulation gestures that temporari...

In mobile interaction, the use of touchscreen interaction, while beneficial from the perspective of portability, has limited spatial accuracy due to the "fat finger problem". As a result, an important challenge on mobile interaction is to find solutions to balance the size of individual widgets against the number of widgets needed during interactio...

In mobile touchscreebn interaction, an important challenge is to find solutions to balance the size of individual widgets against the number of widgets needed during interaction. In this work, to address display space limitations, we explore the design of invisible off-screen toolbars (ether-toolbars) that leverage computer vision to expand applica...

Challenges confronting designers of public displays include display blindness, i.e. the propensity of people to ignore these displays as they become ever-more ubiquitous, and the "first click" problem, i.e. users not interacting with these displays due to being unaware of interactive content. To address the challenge of display blindness, explicit...

In 1997, Accot and Zhai presented seminal work analyzing the temporal cost and instantaneous speed profiles associated with movement along constrained paths. Their work posited and validated the emph{steering law}, which described the relationship between path constraint, path length and the temporal cost of path traversal using a computer input de...

We explore how users approach and define personal space on large, public displays. Our results show that users of public displays use one of two strategies for visual search tasks: minimizers create a small window and work up close to the display, and maximizers expand content to its full resolution and work at a distance. We show that these intera...

Bi-level thresholding is a motion gesture recognition technique that mediates between false positives, and false negatives by using two threshold levels: a tighter threshold that limits false positives and recognition errors, and a looser threshold that prevents repeated errors (false negatives) by analyzing movements in sequence. In this paper, we...

While machine learning is a powerful tool for the analysis and classification of complex real-world datasets, it is still challenging, particularly for developers with limited expertise, to incorporate this technology into their software systems. The first step in machine learning, data labeling, is traditionally thought of as a tedious, unavoidabl...

We describe a mid-air, watch-based 3D interaction technique called Watchcasting. The technique enables target selection and translation by mapping z-coordinate position to forearm rotation. By replicating a large display 3D selection study, we show that Watchcasting provides comparable performance to smartphone (Smartcasting) and electrical impedan...

We describe the design and evaluation of a freehand, smartwatch-based, mid-air pointing and clicking interaction technique, called Watchpoint. Watchpoint enables a user to point at a target on a nearby large display by moving their arm. It also enables target selection through a wrist rotation gesture. We validate the use of Watchpoint by comparing...

One challenge with modern smartwatches is text input. In this paper we explore the use of gestural interaction with a smartwatch to support text input. The inertial measurement unit of a smartwatch is used to capture gestural interaction by a user, and an external display is used to provide feedback. We examine two specific variants of gesture keyb...

We describe an empirical study that explores how users establish and use personal space around large public displays (LPDs). Our study complements field studies in this space by more fully characterizing interpersonal distances based on coupling and confirms the use of on-screen territories on vertical displays. Finally, we discuss implications for...

" We develop and formally evaluate a metaphor for smartphone interaction with 3D environments: Tiltcasting. Under the Tiltcasting metaphor, users interact within a rotatable 2D plane that is "cast" from their phone's interactive display into 3D space. Through an empirical validation, we show that Tiltcasting supports efficient pointing, interaction...

With current technology, 3D environments can be deployed on networked public displays, but cannot be interacted with at a distance without the use of specialized input devices that may not be feasible in public settings. We show that current mobile and wearable devices can be used in place of specialized input devices for 3D interaction with networ...

Clutching is usually assumed to be triggered by a lack of physical space and detrimental to pointing performance. We conduct a controlled experiment using a laptop trackpad where the effect of clutching on pointing performance is dissociated from the effects of control-to-display transfer functions. Participants performed a series of target acquisi...

We introduce and formally evaluate smartcasting: a smartphone-based Ray Casting implementation for 3D environments presented on large, public, autostereoscopic displays. By utilizing a smartphone as an input device, smartcasting enables "walk up and use" interaction with large displays, without the need for expensive tracking systems or specialized...

We investigate how the presence and width of interior bezels impacts visual search performance across tiled displays. In spite of a potential benefit from structured segmentation, we do not find significant differences in visual search time, and note a small effect size of less than 0.5% for bezel width. However, we find participants are more accur...

Recent work by Mandryk and Lough demonstrated that the movement time of Fitts-style pointing tasks varies based on intended use of a target, suggesting major implications for HCI research that models pointing using Fitts' Law. We replicate the study of Mandryk and Lough to determine exactly how and why observed movement times vary. We demonstrate t...

Large displays are often constructed by tiling multiple small displays, creating visual discontinuities from inner bezels that may affect human perception of data. Our work investigates how bezels impact magnitude judgement, a fundamental aspect of perception. Two studies are described which control for bezel presence, bezel width, and user-to-disp...

Handwriting recognizers occasionally misinterpret digital ink input, requiring users to compare their ink input and the recognizer output to identify and correct errors. Technologies like Anoto pens can make this error discovery and correction task more difficult, because verification of recognizer output may occur many hours after data input and m...

Designers of motion gestures for mobile devices face the difficult challenge of building a recognizer that can separate gestural input from motion noise. A threshold value is often used to classify motion and effectively balances the rates of false positives and false negatives. We present a bi-level threshold recognizer that is built to lower the...

We investigate behaviours on, and around, large vertical displays during concurrent usage. Using an observational field study, we identify fundamental patterns of how people use existing public displays: their orientation, positioning, group identification, and behaviour within and between social groups just-before, during, and just-after usage. Th...

Smartphones are frequently used in environments where the user is distracted by another task, for example by walking or by driving. While the typical interface for smartphones involves hardware and software buttons and surface gestures, researchers have recently posited that, for distracted environments, benefits may exist in using motion gestures...

Computer Algebra Systems (CAS) and matrix-based mathematics packages provide sophisticated functionality to assist with mathematical problem solving. However, despite their widespread adoption, little work in the HCI community has examined the extent to which these computational tools support expert problem solving. In this paper, we report finding...

In this work, we explore the design of multi-resolution input on multi-touch devices. We devised a refined zooming technique named Offset, where the target is set at a location offset from the non-dominant hand while the dominant hand controls the direction and magnitude of the expansion. Additionally, we explored the use of non-persistent transfor...

Despite the increasing prevalence of touch-based tablet devices, little attention has been paid to what effects, if any, this form factor has on sketch behaviours in general and on sketch recognizers in particular. We investigate the title question through an empirical study in the context of mathematical expression recognition. Using a corpus of t...

Two-dimensional geometric texture synthesis is the geometric analogue of raster-based texture synthesis. An absence of conventional evaluation procedures in recent synthesis attempts demands an inquiry into the visual significance of synthesized results. In this paper, we report on two psychophysical experiments that explore how people understand n...

Modern smartphones contain sophisticated sensors to monitor three-dimensional movement of the device. These sensors permit devices to recognize motion gestures - deliberate movements of the device by end-users to invoke commands. However, little is known about best-practices in motion gesture design for the mobile computing paradigm. To address thi...

Although publicly available, ground-truthed corpora have proven useful for training, evaluating, and comparing recognition systems in many domains, the availability of such corpora for sketch recognizers, and math recognizers in particular, is currently quite poor. This paper presents a general approach to creating large, ground-truthed corpora for...

Recognizing hand drawn mathematical matrices on tablet computers has proven to be a particularly challenging task. While individual expression recognition can be simplified by assuming the entire content is a single semantic construct, a single math expression, a matrix is composed of multiple expressions arranged in rows and columns. These express...

User interface modes are ubiquitous in both mouse-keyboard and pen-based user interfaces, but the requirement for prior setting of a user interface mode before performing an action imposes a persistent drag on system usability. This chapter reviews our research in approaches to avoiding prior deliberate mode setting while still allowing overloading...

Information kiosks often decorate large public areas to pro-vide basic information to inquisitive patrons. This paper presents an observational study examining groups interact-ing with public kiosks. We identify fundamental issues re-garding patterns in user orientation and layout, group iden-tification, and behaviour both within and between social...

Given the high value of the online search market, whether brand perception or quality of search results matters more for users is a highly salient question. This paper presents findings of the largest controlled, systematic preference elicitation study of search quality versus brand perception. We examine a total of 548 instances of sponsored and o...

Despite nearly ubiquitous access to wireless networks, many users still engage in risky behaviors, make bad choices, or are seemingly indifferent to the concerns that security and privacy researchers work diligently to address. At present, research on user attitudes toward security and privacy on public Wi-Fi networks is rare. This paper explores W...

Sketch recognition user interfaces currently treat the pen in the same manner as a mouse and keyboard. The aim of this workshop is to promote thought and discussion about how to move beyond this to create natural and intuitive pen-based interfaces. To this end, the workshop will include panel discussions, group discussions, and even an instructiona...

Both handwriting recognition systems and their users are error prone. Handwriting recognizers make recognition errors, and users may miss those errors when verifying output. As a result, it is common for recognized documents to contain residual errors. Unfortunately, in some application domains (e.g. health informatics), tolerance for residual erro...

Target expansion is a pointing facilitation technique where the user's target, typically an interface widget, is dynamically enlarged to speed pointing in interfaces. However, with densely packed (tiled) arrangements of widgets, interfaces cannot expand all potential targets; they must, instead, predict the user's desired target. As a result, mispr...

The inferred mode protocol uses contextual reasoning and local mediators to eliminate the need to access specific modes to perform draw, select, move and delete operations in a sketch interface. In this paper, we describe an observational experiment to understand the learnability, user preference and frequency of use of mode inferencing in a sketch...

In sketch recognition systems, ground-truth data sets serve to both train and test recognition algorithms. Unfortunately, generating data sets that are sufficiently large and varied is frequently a costly and time-consuming endeavour. In this paper, we present a novel technique for creating a large and varied ground-truthed corpus for hand drawn ma...

Computer Algebra Systems (CAS) provide sophisticated func- tionality to assist with mathematical problem solving. De- spite their widespread adoption, however, little work in th e HCI community has examined the extent to which these com- putational tools support domain experts. In this paper, we re- port findings from a qualitative study investigat...

This paper introduces kinematic templates, an end-user tool for defining content-specific motor space manipulations in the context of editing 2D visual compositions. As an example, a user can choose the "sandpaper" template to define areas within a drawing where cursor movement should slow down. Our current implementation provides templates that am...

Collocation preferences represent the commonly used expressions, idioms, and word pairings of a language. Because collocation preferences arise from consensus usage, rather than a set of well-defined rules, they must be learned on a case-by-case basis, making them particularly challenging for non-native speakers of a language. To assist non-native...

Many on-line (interactive) mathematics recognition systems allow the creation of typeset equations, normally in LaTeX, but they do not support mathematical problem solving. In this paper, we present MathBrush, a system that allows users to draw math input using a pen-input device on a tablet computer, recognizes the math expression, and then suppor...

Today's handheld gaming systems allow players to engage in multiplayer games via ad-hoc, wireless networking. They are also now sufficiently commonplace that it is possible to study how portability and ad-hoc wireless networking have affected the social gaming practices of owners of these systems. In this paper, we report findings from a qualitativ...

Current generations of computer algebra systems require users to transform two dimensional math expressions into one dimensional strings, to master complex sets of commands, and to analyze lengthy output strings for rel- evant information. MathBrush is a system, designed based on research in education pedagogy, that provides a pen-based interface t...

Despite the importance of pointing-device movement to efficiency in interfaces, little is known on how target shape impacts speed, ac- celeration, and other kinematic properties of motion. In this paper, we examine which kinematic characteristics of motion are impacted by amplitude and directional target constraints in Fitts-style point- ing tasks....

Effective mode-switching techniques provide users of tablet interfaces with access to a rich set of behaviors. While many researchers have studied the relative performance of mode-switching techniques in these interfaces, these metrics tell us little about the behavior of one technique in the absence of a competitor. Differing from past comparison-...

Novel key mappings, including chording, character predic- tion, and multi-tap, allow the use of fewer keys than those on a conventional keyboard to enter text. In this paper, we explore a text input method that makes use of rhythmic mappings of five keys. The keying technique averages 1.5 keystrokes per character for typical English text. In initia...

In pen-tablet input devices modes allow overloading of the electronic stylus. In the case of two modes, switching modes with the non-preferred hand is most effective (12). Further, allowing temporal overlap of mode switch and pen action boosts speed (11). We examine the effect of increasing the number of interface modes accessible via non-preferred...

In this paper we describe strategies for recognizing and using hand drawn matrices in a pen math system. This includes a new technique to recognize common short-forms of writing matrices using ellipsis (...). Ellipsis are commonly used in sketched matrices to illustrate the structure of a matrix without fully specifying the matrix. A second contrib...

Recently proposed novel interaction techniques such as cursor jumping [1] and target expansion for tiled arrangements [13] are predicated on an ability to effectively estimate the endpoint of an input gesture prior to its completion. However, current endpoint estimation techniques lack the precision to make these interaction techniques possible. To...

This paper presents the synergistic use of handheld computers and desktop computers as tools for grade school teachers to record observational data from their classrooms. The system uses handheld devices to tersely log events in situ, and allows later reflection and elaboration through annotation of the logged events on desktop computer. The terse,...

PocketPad is an information management system geared toward university students. The system is designed to support the capture, storage, browsing, editing and organization of handwritten notes via the complementary use of digital pens to capture information, handheld computers to browse and store prior information, and digital pens and handheld com...

Fitts' law, relating the time to acquire a target to the target size and the distance from the target, is an effective and widely used predictor of performance in feedback controlled human motor targeting tasks. Beyond target size, however, movement time also varies according to a subject controlled mental trade-off between speed and accuracy for a...

Pen/Stylus input systems are constrained by the limited input capacity of the electronic stylus. Stylus modes, which allow multiple interpretations of the same input, lift capacity limits, but confront the user with possible cognitive and motor costs associated with switching modes. This paper examines the costs of bimanual mode switching, in which...

Despite their many benefits, challenges exist in the creation of 3D models, particularly for individual not currently skilled with 3D modeling software. To address this, we explore the creation of 3D modeling software for non-domain experts that uses a hierarchical parts database of generic 3D models, and deforms models into specific related target...

This paper describes our experiences modifying the Rapid Application Development methodology for rapid system development to design a data gathering system for mobile fieldworkers using handheld computers in harsh environmental conditions. In our development pro- cess, we integrated User-Centred Design as an explicit stage in the Rapid Application...

This paper describes on-going work in the analysis of motion dynamics in pen-based interaction. The overall goal is the creation of a model of user motion in pen gestures where constraint and curvature vary over the length of the path. In particular, speed/curvature models of motion are used to analyze pen trajectories and infer target constraints...

The focus of this paper is on the design, implementation, and validation of asynchronous multimedia annotations designed for Web-based collaboration in educational and research settings. The two key questions we explore in this paper are: How useful are such annotations and what purpose do annotations serve? What is the ease of use of our specific...

In this paper, we describe our work in adding functional-ity to the standard favorites, or bookmarks, list typically available in modern web browsers. Our goal is to increase the rate at which those browsing the web can "re-find" in-formation they have previously bookmarked for later pe-rusal. To this end, we have developed a favorites manager, dep...

In this paper we introduce a variation on pen-input systems that uses pen orientation to enter selection mode. We compare the use of pen orientation for mode selection to a system which uses the Inferred-Mode protocol as described by Saund and Lank [7]. Our goals are better understand how users in general expect and prefer to interact with pen inpu...

Abstract Personal Digital Assistants (PDAs) have replaced notepads and paper organizers as the medium,of choice for many common,scheduling and organizational tasks. This is due, in no small part, to the pervasive nature of PDAs. However, one area where PDAs are only infrequently used is in note-taking situations, such as making notes on a talk or s...

This demonstration reports results from the EU-funded project Ambient Agoras, investigating future applications of ubiquitous and ambient computing in workspaces. Instead of presenting underlying system technologies or evaluation findings, this demonstration ...

Due to the lack of standards for D graphics on the WWW and for embedded systems, developers suffer as they are required to choose between different technologies or to support multiple technologies depending on the goals of an application. These decisions affect the final product by introducing restrictions based on the technology, or requiring addi...

The long-term goals of the recently started Biomedia project at SFSU are to provide multimedia information systems and applications for the research and education needs of several projects in the SFSU Biology Department. These applications involve a considerable amount of images and image sequence data, in addition to traditional text, genomic, and...