Aaron Bangor’s research while affiliated with AMP and other places

This panel will discuss the System Usability Scale. Panelists all have extensive experience using the SUS within a broad range of contexts: diverse people (e.g., abilities, languages); different types of products; and different testing scenarios. Members of the audience will have the opportunity to ask questions about new research on the validity of the SUS in different environments as well as about lessons learned from practitioners using it to evaluate commercial products. Topics of specific interest to the authors are detailed within this paper.

The system usability scale (SUS; Brooke, 1996) is an instrument commonly utilized in usability testing of commercial products. The goal of this symposium is to discuss the validity of the SUS in usability tests and beyond. This article serves as an introduction to the symposium. Specifically, it provides an overview of the SUS and discusses research questions currently being pursued by the panelists. This current research includes: defining usability norms, assessing usability without performing tasks, and the use of SUS for ergonomics. In addition to this paper, there are four other papers in the symposium, which discuss the impact of experience on SUS data, the relationship between SUS and performance scores, the linkage between SUS and business metrics, as well as the potential for using SUS in test and evaluation for military systems.

As part of an effort to connect results of human factors testing with the needs of internal business clients, a usability testing program was created for smartphones to establish a link between usability metrics and business indicators of success in the marketplace. In addition to standard usability metrics of effectiveness, efficiency, and satisfaction (ISO, 1998), a measure of hedonics (Hassenzahl, Beu, and Burmester, 2001) was used. Of particular note was the use of the System Usability Scale (SUS) (Brooke, 1996) to provide a single measure of user satisfaction for all smartphones tested. Data from 54 studies involving 872 participants were analyzed to determine if usability metrics predicted marketplace outcomes. Results show that usability metrics are significantly correlated with indicators of business success and that the SUS accounted for most of usability’s contribution. Discussion of these results, the value of SUS, and implications for usability professionals are discussed.

This paper characterizes the usability of 14 common, everyday products using the System Usability Scale (SUS). Over 1,000 users were queried about the usability of these products using an online survey methodology. The study employed two novel applications of the SUS. First, participants were not asked to perform specific tasks on these products before rating their usability, but were rather asked to assess usability based on their overall integrated experience with a given product. Second, some of the evaluated products were assessed as a class of products (e.g. ‘microwaves’) rather than a specific make and model, as is typically done. The results show clear distinctions among different products and will provide practitioners and researchers with important known benchmarks as they seek to characterize and describe results from their own usability studies.

Flipping through the Human Factors and Ergonomics Society Directory and Yearbook, it quickly becomes apparent that Human Factors professionals work in a many different business sectors. These business sectors include military, commercial product development, and consulting services. Human factors experts are quite cognizant of the business sector in which they work, but they may have little to no knowledge or understanding of the work in other business sectors. In this discussion panel, experts from the business sectors of military, commercial product development, and consulting will discuss the similarities and differences of human factors in their business sectors. They will speak to the organizational factors, the time and cost factors, as well as end user population factors and how these factors impact how they work to develop systems. This discussion panel will foster greater insight and cooperation across business sectors through an understanding of similarities and differences.

The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including Web sites, cell phones, interactive voice response systems, TV applications, and more. It provides an easy-to-understand score from 0 (negative) to 100 (positive). While a 100-point scale is intuitive in many respects and allows for relative judgments, information describing how the numeric score translates into an absolute judgment of usability is not known. To help answer that question, a seven-point adjective-anchored Likert scale was added as an eleventh question to nearly 1,000 SUS surveys. Results show that the Likert scale scores correlate extremely well with the SUS scores (r=0.822). The addition of the adjective rating scale to the SUS may help practitioners interpret individual SUS scores and aid in explaining the results to non-human factors professionals.