George Fitzmaurice

Autodesk

Skuid is a sketching tool for crafting animated illustrations that contain the exaggerated dynamics of stylized 2D animations. Skuid provides a set of motion amplifiers which implement a set of established principles of 2D animation. These amplifiers break down a complex animation effect into independent, understandable chunks. Each amplifier impos...

Within this work, a novel metaphor, haunted design, is explored to challenge the definitions of "display" used today. Haunted design draws inspiration and vision from some of the most multi-modal and sensory diverse experiences that have been reported, the paranormal and hauntings. By synthesizing and deconstructing such phenomena, four novel oppor...

We present a sketching tool for crafting animated illustrations that contain the exaggerated dynamics of stylized 2D animations. The system provides a set of motion amplifiers which implement a set of established principles of 2D animation. These amplifiers break down a complex animation effect into independent, understandable chunks. Each amplifie...

We present RetroFab, an end-to-end design and fabrication environment that allows non-experts to retrofit physical interfaces. Our approach allows for changing the layout and behavior of physical interfaces. Unlike customizing software interfaces, physical interfaces are often challenging to adapt because of their rigidity. With RetroFab, a new phy...

We present ChronoFab, a 3D modeling tool to craft motion sculptures, tangible representations of 3D animated models, visualizing an object's motion with static, transient, ephemeral visuals that are left behind. Our tool casts 3D modeling as a dynamic art-form by employing 3D animation and dynamic simulation for the modeling of motion sculptures. O...

There is a long tradition for crafting wearable objects directly on the body, such as garments, casts, and orthotics. However, these high-skill, analog practices have yet to be augmented by digital fabrication techniques. In this paper, we explore the use of hybrid fabrication workflows for on-body printing. We outline design considerations for cre...

Sliders and Visual Analogue Scales (VASs) are input mechanisms which allow users to specify a value within a predefined range. At a minimum, sliders and VASs typically consist of a line with the extreme values labeled. Additional decorations such as labels and tick marks can be added to give information about the gradations along the scale and allo...

We present a sketching tool for crafting animated illustrations that contain the exaggerated dynamics of stylized 2D animations. Our system provides a set of motion amplifiers which implement a set of established principles of 2D animation. These amplifiers break down a complex animation effect into independent, understandable chunks. Each amplifie...

We present the Smart Makerspace; a context-rich, immersive instructional workspace for novice and intermediate makers. The Smart Makerspace guides makers through the completion of a DIY task, while providing detailed contextually-relevant assistance, domain knowledge, tool location, usage cues, and safety advice. Through an initial exploratory stud...

" The growth of mobile and wearable technologies has made it often difficult to understand what people in our surroundings are doing with their technology. In this paper, we introduce the concept of candid interaction: techniques for providing awareness about our mobile and wearable device usage to others in the vicinity. We motivate and ground thi...

" In this work, we explore moveables, i.e., interactive papercraft that harness user interaction to generate visual effects. First, we present a survey of children's books that captured the state of the art of moveables. The results of this survey were synthesized into a moveable taxonomy and informed MoveableMaker, a new tool to assist users in de...

" Due to their limited input area, ultra-small devices, such as smartwatches, are even more prone to occlusion or the fat finger problem, than their larger counterparts, such as smart phones, tablets, and tabletop displays. We present NanoStylus -- a finger-mounted fine-tip stylus that enables fast and accurate pointing on a smartwatch with almost...

Text entry for smart eyewear is generally limited to speech-based input due to constraints of the input channels. However, many smart eyewear devices are now including a side touchpad making gesture-based text entry feasible. The Swipeboard technique, recently proposed for ultra-small touch screens such as smart watches, may be particularly suitabl...

A sketch-based interface within an animation engine provides an end-user with tools for creating emitter textures and oscillator textures. The end-user may create an emitter texture by sketching one or more patch elements and then sketching an emitter. The animation engine animates the sketch by generating a stream of patch elements that emanate fr...

A navigation system for navigating a three-dimensional (3D) scene that includes a model or object with which a user can interact. The system accommodates and helps both novice and advanced users. To do this, the system allows a user to designate a point on a model and the system moves the view toward that point. A perspective scale is provided that...

Scatterplots are an effective and commonly used technique to show the relationship between two variables. However, as the number of data points increases, the chart suffers from "over-plotting" which obscures data points and makes the underlying distribution of the data difficult to discern. Reducing the opacity of the data points is an effective w...

Mobile devices offer constant connectivity to the world, which can negatively affect in-person interaction. Current approaches to minimizing the social disruption and improving the subtlety of interactions tend to focus on the development of inconspicuous devices that provide basic input or output. This paper presents a more general approach to sub...

Skin-based input has become an increasingly viable interaction model for user interfaces, however it has yet to be explored outside the domain of mobile computing. In this paper, we examine skin as an interactive input surface for gestural 3D modeling-to-fabrication systems. When used as both the input surface and base canvas for digital design, sk...

A technique for integrating a multi-touch surface into a desktop environment is disclosed. One or more multi-touch display devices are placed on the horizontal surface surrounding or beneath a keyboard or mouse. At least one region on the multi-touch surface is defined to display one or more user interface tools on the multi-touch surface. The one...

A proximity-aware multi-touch tabletop is disclosed that includes both a touch screen display and proximity sensors. The proximity sensors are disposed in one or more annular groups around the touch screen display and are positioned in upward-and outward-facing directions. The proximity sensors allow the multi-touch tabletop to sense the distance o...

Task-based analysis is a common and effective way to measure expertise levels of software users. However, such assessments typically require in-person laboratory studies and inherently require knowledge of the user's task. Today, there is no accepted method for assessing a user's expertise levels outside of a lab, during a user's own home or work e...

The present invention is a system that provides a pen based computer user with a graphical user interface tool, a pen-mouse, that looks like and functions like a mouse but that is controlled by a limited input device such as a pen or stylus of the pen based computer. The pen-mouse is a tracking menu that tracks the position of the pen. A pen cursor...

One embodiment of the present invention sets forth a technique for providing an end user with a digital pen embedded with a spatially-aware miniature projector for use in a design environment. Paper documents are augmented to allow a user to access additional information and computational tools through projected interfaces. Virtual ink may be manag...

A system and technique for displaying a document's workflow history are disclosed. The system includes a graphical user interface for displaying one or more graphical representations of events generated by an application configured to edit a document. Each of the events generated by the application may be stored in a data structure that is associat...

3D printers offer extraordinary flexibility for prototyping the shape and mechanical function of objects. We investigate how 3D models can be modified to facilitate the creation of interactive objects offering dynamic input and output. We introduce a general technique to support rapidly prototyping interactivity by removing interior material from 3...

We present Kitty, a sketch-based tool for authoring dynamic and interactive illustrations. Artists can sketch animated drawings and textures to convey the living phenomena, and specify the functional relationship between its entities to characterize the dynamic behavior of systems and environments. An underlying graph model, customizable through sk...

We present Video Lens, a framework which allows users to visualize and interactively explore large collections of videos and associated metadata. The primary goal of the framework is to let users quickly find relevant sections within the videos and play them back in rapid succession. The individual UI elements are linked and highly interactive, sup...

Ultra-small smart devices, such as smart watches, have become increasingly popular in recent years. Most of these devices rely on touch as the primary input modality, which makes tasks such as text entry increasingly difficult as the devices continue to shrink. In the sole pursuit of entry speed, the ultimate solution is a shorthand technique (e.g....

Draco [Kazi et al. 2014] is a sketch-based interface that allows artists and casual users alike to add a rich set of animation effects to their drawings, seemingly bringing illustrations to life. While previous systems have introduced sketch-based animations for individual objects, our contribution is a unified framework of motion controls that all...

One embodiment of the present invention sets forth a technique for supplementing user-specified text for a search with context information to generate search results that is more closely matched to the user's needs. A context search component maintains a snapshot of the user context and additional context-based queries are generated and correspondi...

One embodiment of the invention sets forth a mechanism for recommended commands to a designer of a graphics design application. Each time a command is executed within the graphics design application, a resource module receives command use information associated with the executed command. In response, the resource module generates and transmits one...

A user data engine residing on an endpoint machine generates a current user context reflecting a specific flow of operations performed by an end-user when interacting with a software application. A context engine residing on a sever machine compares the current user context to one or more stored user contexts included in a context database and gene...

We present CADament, a gamified multiplayer tutorial system for learning AutoCAD. Compared with existing gamified software tutorial systems, CADament generates engaging learning experience through competitions. We investigate two variations of our game, where over-the-shoulder learning was simulated by providing viewports into other player's screen...

The emergence of smart devices (e.g., smart watches and smart eyewear) is redefining mobile interaction from the solo performance of a smart phone, to a symphony of multiple devices. In this paper, we present Duet -- an interactive system that explores a design space of interactions between a smart phone and a smart watch. Based on the devices' spa...

We present Draco, a sketch-based interface that allows artists and casual users alike to add a rich set of animation effects to their drawings, seemingly bringing illustrations to life. While previous systems have introduced sketch-based animations for individual objects, our contribution is a unified framework of motion controls that allows users...

3D modelers often wish to showcase their models for sharing or review purposes. This may consist of generating static viewpoints of the model or authoring animated fly-throughs. Manually creating such views is often tedious and few automatic methods are designed to interactively assist the modelers with the view authoring process. We present a view...

Short video demonstrations are effective resources for helping users to learn tools in feature-rich software. However manually creating demonstrations for the hundreds (or thousands) of individual features in these programs would be impractical. In this paper, we investigate the potential for identifying good tool demonstrations from within screen...

Draco [4] is a sketch-based interface that allows artists and casual users alike to add a rich set of animation effects to their drawings, seemingly bringing illustrations to life. While previous systems have introduced sketch-based animations for individual objects, our contribution is a unified framework of motion controls that allows users to se...

Draco is a sketch-based interface that allows artists and casual users alike to add a rich set of animation effects to their drawings, seemingly bringing illustrations to life. While previous systems have introduced sketch-based animations for individual objects, our contribution is a unified framework of motion controls that allows users to seamle...

Technologies for 3D input and output are rapidly advancing, and are becoming more common in home and workplace environments. However, viewing a stereoscopic display can cause eye strain and fully relying on 3D spatial interactions can be fatiguing and may be less efficient for common 2D tasks. In this paper we explore the design possibilities of tr...

In 2009 we presented the idea of using collaborative filtering within a complex software application to help users learn new and relevant commands (Matejka et al 2009). This project continued to evolve and we explored the design space of a contextual software command recommender system and completed a four-week user study (Li et al. 2011). We then...

YouMove is a novel system that allows users to record and learn physical movement sequences. The recording system is designed to be simple, allowing anyone to create and share training content. The training system uses recorded data to train the user using a large-scale augmented reality mirror. The system trains the user through a series of stages...

An ambient help application selects and displays learning resources to an end-user of a software application that are relevant to the current activities of the end-user in the software application. The ambient help application selects the learning resources based on computed relevance scores of each learning resource and displays the learning resou...

A system that includes a pop-up graphical user interface that includes menu bars overlapping marking menu zones. The interface pops up at the current position of the cursor when the space bar is held down. The menu bars are positioned around a central marking zone with the common menu bars located above the central zone and task specific menu bars...

Authoring tutorials for complex software applications is a time consuming process. It also highly depends on the tutorial designer's skill level and experience. This paper introduces an approach which automatically generates software tutorials using the digital artifacts produced by the users of a software program. We model this process as an optim...

One embodiment of the present invention sets forth a technique for emulating left, right, and center mouse button functionality for a multi-touch input device to provide end-users with full mouse functionality. Various finger to mouse button mapping techniques are used to specify the left, right or middle mouse buttons. A single finger is used for...

Web-based tutorials are a popular help resource for learning how to perform unfamiliar tasks in complex software. However, in their current form, web tutorials are isolated from the applications that they support. In this paper we present FollowUs, a web-tutorial system that integrates a fully-featured application into a web-based tutorial. This no...

Online streaming video systems have become extremely popular, yet navigating to target scenes of interest can be a challenge. While recent techniques have been introduced to enable real-time seeking, they break down for large videos, where scrubbing the timeline causes video frames to skip and flash too quickly to be comprehendible. We present Swif...

We present Patina, an application independent system for collecting and visualizing software application usage data. Patina requires no instrumentation of the target application, all data is collected through standard window metrics and accessibility APIs. The primary visualization is a dynamic heatmap overlay which adapts to match the content, loc...

One of the fundamental operations in today's user interfaces is pointing to targets, such as menus, buttons, and text. Making an error when selecting those targets in real-life user interfaces often results in some cost to the user. However, the existing target-directed pointing models do not consider the cost of error when predicting task completi...

We present Waken, an application-independent system that recognizes UI components and activities from screen captured videos, without any prior knowledge of that application. Waken can identify the cursors, icons, menus, and tooltips that an application contains, and when those items are used. Waken uses frame differencing to identify occurrences o...

We present Magic Finger, a small device worn on the fingertip, which supports always-available input. Magic Finger inverts the typical relationship between the finger and an interactive surface: with Magic Finger, we instrument the user's finger itself, rather than the surface it is touching. Magic Finger senses touch through an optical mouse senso...

We present GamiCAD, a gamified in-product, interactive tutorial system for first time AutoCAD users. We introduce a software event driven finite state machine to model a user's progress through a tutorial, which allows the system to provide real-time feedback and recognize success and failures. GamiCAD provides extensive real-time visual and audio...

We investigate implanted user interfaces that small devices provide when implanted underneath human skin. Such devices always stay with the user, making their implanted user interfaces available at all times. We discuss four core challenges of implanted user interfaces: how to sense input through the skin, how to produce output, how to communicate...

We first conduct a study using abstracted video content to measure the effects of latency on video scrubbing performance and find that even very small amounts of latency can significantly degrade navigation performance. Based on these results, we present Swift, a technique that supports real-time scrubbing of online videos by overlaying a small, lo...

Tutorials and sample workflows for complicated, feature-rich software packages are widely available online. As a result users must differentiate between workflows to choose the most suitable one for their task. We present Delta, an interactive workflow visualization and comparison tool that helps users identify the tradeoffs between workflows. We c...

General-purpose software applications are usually not tailored for a specific user with specific tasks, strategies or preferences. In order to achieve optimal performance with such applications, users typically need to transition to an alternative efficient behavior. Often, features of such alternative behaviors are not initially accessible and fir...

Citeology is an interactive visualization that looks at the relationships between research publications through their use of citations. The sample corpus uses all 3,502 papers published at ACM CHI and UIST between 1982 and 2010, and the 11,699 citations between them. A connection is drawn between each paper and all papers which it referenced from t...

Users of complex software applications frequently need to consult documentation, tutorials, and support resources to learn how to use the software and further their understand-ing of its capabilities. Existing online help systems provide limited context awareness through "what's this?" and simi-lar techniques. We examine the possibility of making m...

We present Medusa, a proximity-aware multi-touch tabletop. Medusa uses 138 inexpensive proximity sensors to: detect a user's presence and location, determine body and arm locations, distinguish between the right and left arms, and map touch point to specific users and specific hands. Our tracking algorithms and hardware designs are described. Explo...

We present IP-QAT, a new community-based question and answer system for software users. Unlike most community forums, IP-QAT is integrated into the actual software application, allowing users to easily post questions, answers and tips without having to leave the application. Our in-product implementation is context-aware and shows relevant posts ba...

We describe TwitApp, an enhanced micro-blogging system integrated within AutoCAD for design sharing. TwitApp integrates rich content and still keeps the sharing transaction cost low. In TwitApp, tweets are organized by their project, and users can follow or unfollow each individual project. We introduce the concept of automatic tweet drafting and o...

We describe Sketch-Sketch Revolution, a new tutorial system that allows any user to experience the success of drawing content previously created by an expert artist. Sketch-Sketch Revolution not only guides users through the application user interface, it also provides assistance with the actual sketching. In addition, the system offers an authorin...

We examine the use of modern recommender system technology to aid command awareness in complex software applications. We first describe our adaptation of traditional recommender system algorithms to meet the unique requirements presented by the domain of software commands. A user study showed that our item-based collaborative filtering algorithm ge...

Although touch-screen laptops are increasing in popularity, users still do not comfortably rely on touch in these environments, as current software interfaces were not designed for being used by the finger. In this paper, we first demonstrate the benefits of using touch as a complementary input modality along with the keyboard and mouse or touchpad...

Despite the growing research on usability in the pre-development phase, we know little about post-deployment usability activities. To characterize these activities, we surveyed 333 full-time usability professionals and consultants working in large and small corporations from a wide range of industries. Our results show that, as a whole, usability p...

In this paper we present Ambient Help, a system that supports opportunistic learning by providing automatic, context-sensitive learning resources while a user works. Multiple videos and textual help resources are presented ambiently on a secondary display. We define and examine a collection of design consideration for this type of interface. After...

Despite the prominence of multi-touch technologies, there has been little work investigating its integration into the desktop environment. Bringing multi-touch into desktop computing would give users an additional input channel to leverage, enriching the current interaction paradigm dominated by a mouse and keyboard. We provide two main contributio...

Despite being an important channel for end-user assistance, few studies have directly investigated the interactions that occur in modern-day practice of software product support. We present results from a multi-dimensional analysis of product support activities at a leading design software company. We carried out a quantitative analysis of existing...

In traditional graphical user interfaces, the majority of UI elements are hidden to the user in the default view, as application designers and users desire more space for their application data. We explore the benefits of dedicating additional screen space for presenting an alternative visualization of an application's user interface. Some potentia...

We describe Chronicle, a new system that allows users to explore document workflow histories. Chronicle captures the entire video history of a graphical document, and provides links between the content and the relevant areas of the history. Users can indicate specific content of interest, and see the workflows, tools, and settings needed to reprodu...

We investigate the use of on-line contextual video assistance to improve the learnability of software functionality. After discussing motivations and design goals for such forms of assistance, we present our new technique, ToolClips. ToolClips augment traditional tooltips to provide users with quick and contextual access to both textual and video a...

MouseLight is a spatially-aware standalone mobile projector with the form factor of a mouse that can be used in combination with digital pens on paper. By interacting with the projector and the pen bimanually, users can visualize and modify the virtually augmented contents on top of the paper, and seamlessly transition between virtual and physical...

Despite the considerable quantity of research directed towards multitouch technologies, a set of standardized UI components have not been developed. Menu systems provide a particular challenge, as traditional GUI menus require a level of pointing precision inappropriate for direct finger input. Marking menus are a promising alternative, but have ye...

Inspired by our automotive and product design customers using large displays in design centers, visualization studios, and meeting rooms around the world, we have been exploring the use and potential of large display installations for almost a decade. Our research has touched on many aspects of this rich design space, from individual tools to compl...

Digital pen systems, originally designed to digitize annotations made on physical paper, are evolving to permit a wider variety of applications. Although the type and quality of pen feedback (e.g., haptic, audio, and visual) have a huge impact on advancing the digital pen technology, dynamic visual feedback has yet to be fully investigated. In para...

It is well-accepted that learnability is an important aspect of usability, yet there is little agreement as to how learnability should be defined, measured, and evaluated. In this paper, we present a survey of the previous definitions, metrics, and evaluation methodologies which have been used for software learnability. Our survey of evaluation met...

We explore the use of multi-finger input to emulate full mouse functionality, such as the tracking state, three buttons, and chording. We first present the design space for such techniques, which serves as a guide for the systematic investigation of possible solutions. We then perform a series of pilot studies to come up with recommendations for th...

We explore the use of modern recommender system technology to address the problem of learning software applications. Before describing our new command recommender system, we first define relevant design considerations. We then discuss a 3 month user study we conducted with professional users to evaluate our algorithms which generated customized rec...

Literally hundreds of thousands of users of 2D computer-aided design (CAD) tools are in the difficult process of transitioning to 3D CAD tools. A common problem for these users is disorientation in the abstract virtual 3D environments that occur while developing new 3D scenes. To help address this problem, we present a novel in-scene 3D widget call...

Typical commercial 3D CAD tools provide modal tools such as pan, zoom, orbit, look, etc. to facilitate freeform navigation in a 3D scene. Mastering these navigation tools requires a significant amount of learning and even experienced computer users can find learning confusing and error-prone. To address this we have developed a concept called ―Safe...

We describe a new type of graphical user interface widget called the "PieCursor." The PieCursor is based on the Tracking Menu technique and consists of a radial cluster of command wedges, is roughly the size of a cursor, and replaces the traditional cursor. The PieCursor technique merges the normal cursor function of pointing with command selection...

Tangible User Interface (TUI) research has become increasingly widespread over the past 25 years. It is an essential component of Ubiquitous Computing and Augmented Reality research. It introduces many challenging problems in the theory and practice of interaction design. However much day-to-day research is concerned with the practicalities of maki...

In recent years, the CHI community has seen growth in projects that involve tangible user interfaces and tangible interaction. But, many researchers feel that this emerging field lacks in justifying research, industry adoption, and conceptual frameworks. This panel gathers pioneers and active researchers in the field, in an effort to understand the...

We describe a new interactive system for 3D design review, built to exploit the visual perception cue of motion parallax, in order to enhance shape perception and aesthetic evaluation. Traditional CAD applications typically use "bookmarked" static views for design evaluation. In our system, we replace static views with moving "shots" interspersed w...

The emergence of large displays holds the promise of basking us in rich and dynamic visual landscapes of information, art, and entertainment. How will our viewing and interaction experiences change when large displays are introduced in our workplace, home, and commercial settings? What technology is needed to implement this vision? This special iss...

Alias Visualization Studio (the VizStudio) is a unique meeting facility that makes use of large-display technology to support digital visual communication and collaboration with corporate clients, future customers, employees, and corporate partners. We discovered that using large displays in this type of corporate meeting facility produces its own...

We describe a new interaction technique, called HoverCam, for navigating around 3D objects at close proximity. When a user is closely inspecting an object, the camera motions needed to move across its surface can become complex. For tasks such as 3D painting or modeling small detail features, users will often try to keep the camera a small distance...

In this paper we investigate the relationship between bimanual (two-handed) manipulation and the cognitive aspects of task integration, divided attention and epistemic action. We explore these relationships by means of an empirical study comparing a bimanual technique versus a unimanual (one-handed) technique for a curve matching task. The bimanual...

We describe a new interaction technique, called a spotlight, for directing the visual attention of an audience when viewing data or presentations on large wall-sized displays. A spotlight is simply a region of the display where the contents are displayed normally while the remainder of the display is somewhat darkened. In this paper we define the b...

We describe a new widget and interaction technique, known as a "Frisbee," for interacting with areas of a large display that are difficult or impossible to access directly. A frisbee is simply a portal to another part of the display. It consists of a local "telescope" and a remote "target". The remote data surrounded by the target is drawn in the t...

It has been more than ten years since such "information appliances" as ATMs and grocery store UPC checkout counters were introduced. For the office environment, Mark Weiser began to articulate the notion of UbiComp and identified some of the salient features of the trends in 1991. Embedded computation is also becoming widespread. Microprocessors, f...

The elements of video and computer games to create a compelling user experience in the domains of virtual design and product marketing are discussed. Two research projects intended for use in design evaluation and product marketing applications which reflect on the game-like experiences are analyzed. The users interact with the designs and products...

We introduce the Boom Chameleon, a novel input/output device consisting of a flat-panel display mounted on a tracked mechanical boom. The display acts as a physical window into 3D virtual environments, through which a one-to-one mapping between real and virtual space is preserved. The Boom Chameleon is further augmented with a touch-screen and a mi...

We describe a new type of graphical user interface widget, known as a "tracking menu." A tracking menu consists of a cluster of graphical buttons, and as with traditional menus, the cursor can be moved within the menu to select and interact with items. However, unlike traditional menus, when the cursor hits the edge of the menu, the menu moves to c...

We introduce the Boom Chameleon, a novel input/output device consisting of a flat-panel display mounted on a tracked mechanical boom. The display acts as a physical window into 3D virtual environments, through which a one-to-one mapping between real and virtual space is preserved. The Boom Chameleon is further augmented with a touch-screen and a mi...

This paper describes StyleCam, an approach for authoring 3D viewing experiences that incorporate stylistic elements that are not available in typical 3D viewers. A key aspect of StyleCam is that it allows the author to significantly tailor what the user sees and when they see it. The resulting viewing experience can approach the visual richness and...