Min Lin’s research while affiliated with University of Maryland, Baltimore County and other places

Speech text entry can be problematic during ideal dictation conditions, but difficulties are magnified when external conditions deteriorate. Motion during speech is an extraordinary condition that might have detrimental effects on automatic speech recognition. This research examined speech text entry while mobile. Speech enrollment profiles were created by participants in both a seated and walking environment. Dictation tasks were also completed in both the seated and walking conditions. Although results from an earlier study suggested that completing the enrollment process under more challenging conditions may lead to improved recognition accuracy under both challenging and less challenging conditions, the current study provided contradictory results. A detailed review of error rates confirmed that some participants minimized errors by enrolling under more challenging conditions while others benefited by enrolling under less challenging conditions. Still others minimized errors when different enrollment models were used under the opposing condition. Leveraging these insights, we developed a decision model to minimize recognition error rates regardless of the conditions experienced while completing dictation tasks. When applying the model to existing data, error rates were reduced significantly but additional research is necessary to effectively validate the proposed solution.

When mobile devices are used on the move, a user's limited visual resources are split between interacting with the mobile devices and maintaining awareness of the surrounding environment. In this study, we examined stylus-based tapping operations on a PDA under three mobility situations: seated, walking on a treadmill, and walking through an obstacle course. The results revealed that Fitts’ Law continues to be effective even under the most challenging obstacle course condition. While target selection times did not differ between the various mobility conditions, overall task completion times, error rates, and several measures of workload differed significantly. Diminished performance under the obstacle course condition was attributed to increased demands on attention associated with navigating through the obstacle course. Results showed that the participants in the obstacle course condition were able to tap on a 6.4mm-diameter target with 90% accuracy, but they reduced their walking speed by 36% and perceived an increased workload. Extending earlier research, we found that treadmill-based conditions were able to generate representative data for task selection times, but accuracy differed significantly from the more realistic obstacle course condition.

Increased use of mobile phones and associated services in China highlights the need for effective Chinese input methods for mobile devices. Mapping thousands of characters to a standard telephone keypad is a significant challenge. Structure-based methods provide an appealing known-character/known-code solution, but assigning multiple strokes to each key forces users to learn new, often unfamiliar, mappings. Using an established stroke input method, our study revealed important effects of keypad legend on performance. Novice user performance was evaluated with several alternative keypad designs. The results confirmed that both abstract symbols and concrete examples helped improve the usability of the keypad in Chinese text-entry tasks. Further, combining abstract symbols and concrete examples resulted in performance nearly tripling as compared to the original design. The stroke-to-key mapping accuracy also increased significantly. Handwriting analysis confirmed that the reduced errors are directly associated with the keypad-based text-entry technique.

Desktop interaction solutions are often inappropriate for mobile devices due to small screen size and portability needs. Speech recognition can improve interactions by providing a relatively hands-free solution that can be used in various situations. While mobile systems are designed to be transportable, few have examined the effects of motion on mobile interactions. This paper investigates the effect of motion on automatic speech recognition (ASR) input for mobile devices. Speech recognition error rates (RER) have been examined with subjects walking or seated, while performing text input tasks and the effect of ASR enrollment conditions on RER. The obtained results suggest changes in user training of ASR systems for mobile and seated usage.

Developing more effective and efficient Chinese character input methods has the potential to help Chinese mobile phone users (currently 320 millions) input text messages. iTAP(R) supports input based on the writing structure of Chinese characters. Current keypad graphics include three items: digits (0-9), letters (A-Z), and symbols that represent the minimum writing units of Chinese characters (strokes). Our study revealed the difficulties of mapping these strokes to individual keys using the current symbols. We present a case study illustrating the user-centered redesign of these symbols. The new symbols allow for faster entry speeds and lower error rates as compared to the current commercial solution. Results with our solution were also favorable when compared to Pinyin, a popular cross-cultural solution relying on the Roman alphabet. The new design is in the process of being integrated into commercial mobile phones for users who would prefer native input methods for Chinese.

The increasing popularity of Short Message Services (SMS) in China highlights the need for effective and efficient methods for entering Chinese text on mobile phones. While stroke-based methods have potential advantages over pronunciation-based solutions, usability issues have limited the effectiveness of existing stroke-based methods. One significant usability challenge has been the ambiguous stroke-to-key mapping rules that are typically employed. We proposed a new solution that employs a combination of abstract symbols and example strokes to help users map strokes to keys more effectively. A longitudinal experiment was used to evaluate character entry performance using both objective and subjective measures for our new design as well as the existing solution. The results confirmed that a new design allows for improved performance as well as higher satisfaction levels as compared to the original design. Further, after approximately 1 h of experience with the stroke-based method, novices were able to enter Chinese text at speeds comparable to that observed with the pronunciation-based Pinyin method. Results showed that the new design provided users with a better understanding of the system throughout the study, beginning with their first exposure to the keypad. By utilizing a combination of abstract representations and concrete examples of the available strokes, the new design reduced the ambiguity that typically exists regarding stroke-to-key mappings. In this way, usability was improved without any changes to the underlying technologies. Our results demonstrate that stroke-based solutions for Chinese character entry can be effective alternatives for mobile phones, providing an effective alternative for the many individuals who can write Chinese but do not speak the Mandarin dialect that serves as the basis for Pinyin. The improved solution could also be used with a traditional numeric keypad to allow one-handed data entry for desktop or mobile computers.

Desktop interaction solutions are often inappropriate for mobile devices due to small screen size and portability needs. Speech recognition can improve interactions by providing a relatively hands-free solution that can be used in various situations. While mobile systems are designed to be transportable, few have examined the effects of motion on mobile interactions. We investigated the effect of motion on automatic speech recognition (ASR) input for mobile devices. We examined speech recognition error rates (RER) with subjects walking or seated, while performing text input tasks and the effect of ASR enrollment conditions on RER. RER were significantly lower for seated conditions. There was a significant interaction between enrollment and task conditions. When users enrolled while seated, but completed walking tasks, RER increased. In contrast, when users enrolled while walking, but completed seated tasks, RER decreased. These results suggest changes in user training of ASR systems for mobile and seated usage.

People frequently write messages to themselves. These informal, hurried personal jottings serve as temporary storage for notable information as well as reminders for future action. Many mobile technologies have been designed specifically to support this ubiquitous behavior; however, adoption has been universally problematic. Despite its clear utility, the process of taking micronotes stubbornly resists computing support. This field study examines the lifecycles of the canonical micronote forms (immediate use, temporary storage, and prospective memory aid), pinpointing the behaviors that are mismatched with current mobile support. Implications for improving the design of these systems are presented, culminating in a vision for integrated paper-digital micronote systems. This shifts the development focus away from trying to support the entire micronote lifecycle, emphasizing instead the different behaviors best supported by the different technologies.

Since Weiser introduced his vision of ubiquitous computing, computing devices have become lighter, smaller, cheaper, and more powerful. At the same time, pervasive computing and the concept of context have attracted significant attention, with the goal of supporting the use of computing devices anywhere, anytime, as computers fade into the environment. While there has been some success inferring the users' intensions, reliably understanding users' general goal remains a significant challenge. Using limited context information, such as location, can be useful, but the benefits are limited. Context is more than location. As computers are embedded into everyday things, the situations users encounter become more variable. As a result, situationally- induced impairment and disabilities (SIID) will become more common and user interfaces will play an even more important role. Recent understandings on context suggested the importance of applications themselves as parts of the whole context space. This article will explain and discuss the characteristics of SIID under a three-dimension (human, environment, and applications) context model. We suggest integrating information from all dimensions to have a whole picture of context. More studies are needed to understand the relationship among different dimensions, and to help design effective context-aware applications overcoming SIID.