Conference PaperPDF Available

What does the System Usability Scale (SUS) Measure?

Authors:

Abstract

The System Usability Scale (SUS) is widely used as a quick method for measuring usability; however, past research showed there is only a weak re lationship between SUS scores and one behavioral usability measure, and alternatively, SUS corresponds more strongly with user preference. This suggests that the underlying constructs of the SUS may not be well understood. In this study, participants were asked to think aloud while completing a usability test and filling out the SUS. Correlations showed no relationship between behavioral performance and SUS scores. Instead, a relationship was observed between SUS scores and perceived success. Furthermore, participants described a variety of reasons for selecting their SUS responses that were unrelated to the usability of the system, which we have termed rationalizations. This suggests that the SUS is constructed of a combination of experiential components, including attitudinal perceptions. Consequently, SUS scores may be more helpful as a tool for comparison (between competitors, iterations, etc.,) or when used in conjunction with formative usability testing methods to provide a holistic view of real and perceived user experience.
... Everyone had to perform the steps shown in Table 1 on the provided smartphone application, which merely took approximately 3-4 minutes per session. Once they finished the tasks, the overall experience was assessed by using SUS (System Usability Scale) [30], which presented us with the results described in the following sections. The brainstorming session was organised in groups of three to five people from the same age sector, in order to better understand and assess which functionalities would better fulfill their general needs. ...
... Nonetheless, the youngsters seemed to be far more receptive to the idea of allowing a smart assistant to help them in any situation they might find themselves into. To make the evaluation process more transparent, we chose to use SUS [30], in which the responses were multiplied by 2.5, thus obtaining a scale from 0 to 100 from the original 0 to 40 scores. These are considered to be percentile ranks [31]. ...
Article
Full-text available
Visually challenged people may experience certain difficulties in their daily interaction with technology. That is essentially because the main way to exchange and process information is by written text, images or videos. Since the basic purpose of innovation is to improve people’s lifestyle, in this paper we propose a system that can make technology accessible to a broader group. Our prototype is presented as a mobile application based on vocal interaction, which can help people facing visual disorders consult their personal agenda, create an event, invite other friends to attend it, check the weather in certain areas and many other day-to-day tasks. Regarding the implementation, the project consists of a mobile application that interacts with a cloud based system, which makes it reliable and low in latency due to the resource availability in multiple global regions, provided by the newly emerging platform used in building the infrastructure. The novelty of the system lays in the highly flexible serverless architecture [1] that is open to extension and closed to modification through the set of autonomous cloud processing methods that sustain the base of the functionality. This distributed processing approach guarantees that the user always receives a response from his personal assistant, either by using artificial intelligence context generated phrases, by real-time cloud function processing or by fallback to the training answers.
... We conducted an external user evaluation of GridAttackSim and GridAttackAnalyzer. Particularly, ten participants, who are Ph.D. candidates in cybersecurity or related topics, were invited to use the two frameworks and then give feedback based on the System Usability Scale (SUS) [18]. This wellknown standardized questionnaire is a widely accepted, reliable tool for measuring usability. ...
... Other researchers have developed and validated techniques like the heuristic evaluation (Nielsen & Mӧlich, 1990), the cognitive walk-through (Polson et al., 1992), and in person usability evaluations (Rubin & Chisnell, 2008) to examine interfaces in time and cost-efficient ways that provide developers with the critical information they need to improve the UX of any given system. Of course, the critical assessment of the goodness of these combined methods has also been studied extensively (Drew et al., 2018;Molich, 2018;Molich et al., 2004). There has also been work in trying to identify and use objective measures of usability, such as eye movements (Poole & Ball, 2006), and biometric measures such as GSR (Lin et al., 2005) and fNIRS (Hirshfield et al., 2009). ...
... There are some standardized usability measurement questionnaires such as Computer System Usability Questionnaire (CSUQ) [8], Questionnaire for User Interface Satisfaction (QUIS) [9], and System Usability Scale (SUS) [10], and so on. QUIS has 27 questions; each question is a rating on a ten-point scale with an appropriate anchor at the end. ...
Article
Full-text available
System usability scale (SUS) is a quick and effective approach to evaluate the usability of a product. SUS is a low expense usability scale which can be employed for the assessment of interactive systems. SUS was originally developed in English but many researchers have mentioned that participants answer more effectively when the SUS scale has been developed in their mother language instead of English. So, there is a need for its Urdu translation for native Urdu speakers. In this research, we have developed the Urdu version of the SUS and its reliability compared with the original English scale. Forward and backward translation technique is used to translate it. Data was collected by using Google Forms. A total of 170 people participated in the survey. Results are formulated by using the SPSS Statistics tool. The reliability of the English version is 0.717 and for the Urdu version, it is 0.835. High reliability of the Urdu version indicates that people understand this scale better as compared to English. It also shows that this scale can be used by usability practitioners in Pakistan.
... Nonetheless, the youngsters seemed to be far more receptive to the idea of allowing a smart application to help them in any situation they might find themselves into. To make the evaluation process more transparent, we chose to use SUS (Drew et al., 2018), in which the responses were multiplied by 2.5, thus obtaining a scale from 0 to 100 from the original 0 to 40 scores. These are considered to be percentile ranks (Rogosa, 1999). ...
Preprint
Full-text available
The habilitation thesis presents two main directions: 1. Exploiting data from social networks (Twitter, Facebook, Flickr, etc.) - creating resources for text and image processing (classification, retrieval, credibility, diversification, etc.); 2. Creating applications with new technologies : augmented reality (eLearning, games, smart museums, gastronomy, etc.), virtual reality (eLearning and games), speech processing with Amazon Alexa (eLearning, entertainment, IoT, etc.). The work was validated with good results in evaluation campaigns like CLEF (Question Answering, Image CLEF, LifeCLEF, etc.), SemEval (Sentiment and Emotion in text, Anorexia, etc.).
... Nonetheless, the youngsters seemed to be far more receptive to the idea of allowing a smart application to help them in any situation they might find themselves into. To make the evaluation process more transparent, we chose to use SUS (Drew et al., 2018), in which the responses were multiplied by 2.5, thus obtaining a scale from 0 to 100 from the original 0 to 40 scores. These are considered to be percentile ranks (Rogosa, 1999). ...
Thesis
Full-text available
This thesis presents the author's research activity after March 2009 when he defended his Ph.D. Thesis "Textual Entailment" from the artificial intelligence domain, related to natural language processing (NLP). NLP is a field of research that covers computer understanding and manipulation of human language, trying to make the machine derive meaning from human language in a smart and useful way, and performing difficult tasks such as information retrieval and extraction, question answering, exam marking, document classification, report generation, automatic summarization and translation, speech recognition, dialogs between human and machine, or other tasks currently performed by humans such as help-desk jobs. Continuing the work from this domain another great challenge of the NLP was approached, the one related to the creation of large textual resources, where the notion of credibility was introduced. When this activity is done manually by human experts, it is costly in terms of the time required to create them and in terms of the human resource to be involved. At the same time, these resources are the basic elements of NLP software applications, their quality depending on the size of the resources and their quality. For this reason, in recent years, automated methods involving social networks have proved to be a worthy method to consider, because here we have access on the one hand to many data, and on the other hand to the thoughts and feelings of users, their comments on events or products, etc.. A new direction that emerged after supporting the Ph.D. thesis is related to the use of new technologies in applications, which will come to the aid of the users who use them. Applications have been made for smartphones or tablets most of the time, and have used more natural modes of user interaction with applications. We mention here, the interpretation of user gestures, speech recognition, the exploitation of images made with the help of video cameras and the use of information taken by sensors. Augmented reality and virtual reality are two current research directions that allow the creation of applications for increasingly diverse fields, such as e-learning, games, interior design, museums, botanical gardens, medicine, etc.. There are two main directions: exploiting data from social networks and using new technologies to improve the quality of life, which corresponds to the next two sections. The last section contains conclusions and proposals for future work.
Article
Full-text available
Introduction Youth and young adults bear a disproportionate share of the HIV burden and there is a critical need for interventions to curb health disparities experienced among these age groups. The purpose of our research is to build on our theory-guided model and formative research to develop a mobile health application, called WYZ, for improved engagement in HIV care and antiretroviral therapy adherence, and pilot test it among youth and young adults living with HIV (YLWH). In this paper, we explain the design and development of WYZ for YLWH, describe the design of a forthcoming pilot trial for evaluating the feasibility and acceptability of WYZ and compare WYZ with other mobile health applications being developed to improve engagement in HIV care and antiretroviral medication adherence. Methods and analysis We used an agile methodology, shown to be useful in software development, and elicited feedback during beta testing to develop WYZ. WYZ is a modular, adaptive and personalised intervention delivered via a mobile phone. It is grounded in the information, motivation, behaviouralskills model which has been valuable for understanding and guiding the development of interventions for complex health behaviours. WYZ was created in collaboration with YLWH aged 18–29 years using a human-centred design approach that emphasises understanding the perspective of the users of the technology. WYZ is intended to improve engagement in HIV care by: (1) enhancing medication adherence self-efficacy, (2) increasing awareness and use of community resources, (3) reducing barriers to communication between youth and their healthcare team, and (4) providing a secure platform for the formation of a private online community of YLWH. We will conduct a 6-month single-arm pilot study to examine feasibility and acceptability of WYZ among 76 YLWH who live or receive care in the San Francisco Bay Area. All study activities, including recruitment, screening, enrolment, study assessments, provision of incentives and exit interviews, will be conducted remotely. We will explore feasibility and acceptability outcomes of the intervention using quantitative and qualitative methods. Ethics and dissemination Study staff will obtain written consent for study participation from all participants. This study and its protocols have been approved by the University of California San Francisco (UCSF) Institutional Review Board. Study staff will work with the UCSF Center for AIDS Prevention Studies’ Community Engagement Core and the Youth Advisory Panel to disseminate results to the participants and the community using presentations, community forums, journal publications and/or social media. Trial registration number NCT03587857 ; Pre-results.
ResearchGate has not been able to resolve any references for this publication.