No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The System Usability Scale (SUS): an Empirical evaluation
International Journal of Human-Computer Interaction
Abstract
This article presents nearly 10 year's worth of System Usability Scale (SUS) data collected on numerous products in all phases of the development lifecycle. The SUS, developed by Brooke (1996)2.
Brooke , J. 1996. “SUS: A “quick and dirty” usability scale”. In Usability evaluation in industry, Edited by:
Jordan , P. W. ,
Thomas , B. A. Weerdmeester and
McClelland , I. L. 189–194. London: Taylor & Francis. View all references, reflected a strong need in the usability community for a tool that could quickly and easily collect a user's subjective rating of a product's usability. The data in this study indicate that the SUS fulfills that need. Results from the analysis of this large number of SUS scores show that the SUS is a highly robust and versatile tool for usability professionals. The article presents these results and discusses their implications, describes nontraditional uses of the SUS, explains a proposed modification to the SUS to provide an adjective rating that correlates with a given score, and provides details of what constitutes an acceptable SUS score.
... Rapid bacterial and resistance detection in patients with sepsis in Harare, Zimbabwe System Usability Scale (SUS) is an assessment of product usability applicable to many types of technologies [15,16]. The survey questions were programmed onto ODK and were administered to the laboratory scientist by a research assistant. ...
... Statements 1,3,5,7 and 9 (positively worded statements) were derived from the average score minus 1 whilst SUS sores for statements 2,4,6,8 and 10 (negatively worded statements) were derived from 5 minus the average score. The total scores were multiplied by 2.5 to convert the range of possible values from 0 to 40 to 0 to 100 [15,16]. Existing literature suggest that a system with a SUS score above 68 is considered above average usability [15,16]. ...
... The total scores were multiplied by 2.5 to convert the range of possible values from 0 to 40 to 0 to 100 [15,16]. Existing literature suggest that a system with a SUS score above 68 is considered above average usability [15,16]. SUS yields a single number representing a composite measure of the overall usability of the system under study. ...
Background
Sepsis is a major cause of mortality in low-resource settings. Effective microbiological culture services are a bottleneck in diagnosis and surveillance.
Aim
We aimed to evaluate the performance of the BIOFIRE FILMARRAY Blood Culture Identification 2 (BCID2, bioMérieux) assay in a low-resource setting laboratory in comparison to standard practice.
Methods
This five month prospective validation study included all positive blood cultures collected at Sally Mugabe Central Hospital, Harare, Zimbabwe. BCID2 testing was done in parallel to standard phenotypic procedures and resistance testing. Reference identification was performed using mass spectrometry or whole genome sequencing. Only samples with available reference standard results were included in the analysis. Data captured on paper-based forms was entered into electronic case report forms (ODK Collect). Specificity and sensitivity for BCID2 were calculated in comparison to the reference standards, with performance measures calculated using the Wilson score. Biomedical scientists using BCID2 completed a system usability survey (SUS).
Results
Positive results were recorded in 780/2,023 (38.5%) blood cultures, within which 377 (48.3%) had reference results and so were included in analysis. Neonatal samples were most frequent (182, 48.3%), then paediatric (150, 39.8%), then adults (18, 4.8%) and unknown (27, 7.2%). Specificity exceeded 95% throughout. Sensitivity ranged from 50% (A. calcoaceticus-baumanii complex, Proteus spp.) to 100% (S. pneumoniae, Salmonella spp). Using BCID2, CTX-M was detected in 111/175 (74.5%) Enterobacterales, from which 5/111 also had NDM and VIM detected. NDM-5 was detected in 2/5 NDM samples using sequencing. In total 3/23 S. aureus isolates were methicillin resistant, from which one was confirmed using phenotypic antimicrobial susceptibility testing. Usability was good (SUS score = 79.5).
Conclusion
Rapid molecular tests have potential to improve turn-around time and quality of sepsis diagnostics. However, specific work-flows are critical to supplement molecular tests with minimal phenotypic tests for optimal clinical decision-making.
... The System Usability Scale questionnaire is a reliable and widely recognized tool for assessing the usability of various technologies and innovations (Supplemental Digital Content, Supplementary Data 1, http://links.lww.com/JCN/A344). [25][26][27][28] The research received approval from the Ethics Committee for Human Sciences at the University of Turku. The researchers then obtained permission from Turku University of Applied Sciences to collect data. ...
... Brooke 31 initially developed System Usability Scale to evaluate product usability, which was later modified for various technological products. [25][26][27][28] The System Usability Scale questionnaire is a reliable and widely recognized tool for assessing the usability of various technologies and innovations. [25][26][27][28] It is quick and easy to administer and provides a single score on a scale that most people can understand. ...
... [25][26][27][28] The System Usability Scale questionnaire is a reliable and widely recognized tool for assessing the usability of various technologies and innovations. [25][26][27][28] It is quick and easy to administer and provides a single score on a scale that most people can understand. The System Usability Scale overall score ranges from 0 to 100, with higher scores indicating better usability. ...
Background
Several audiovisual feedback (AVF) devices have been developed to monitor chest compression quality during cardiopulmonary resuscitation (CPR). However, most marketed stand-alone AVF devices are inflexible and rigid, causing discomfort and sometimes pain to the rescuers' hands.
Objective
The objective of this study was to evaluate the effectiveness and usability of a newly developed soft and flexible resuscitation glove designed to improve the quality of chest compressions during CPR.
Methods
We conducted a manikin-based randomized crossover study to compare the effectiveness of a newly developed AVF device (ResuGlove CPR Group) and standard CPR (Standard CPR Group) in improving the quality of chest compressions in simulated cardiac arrest cases. The usability of the newly developed ResuGlove was assessed using a System Usability Scale questionnaire.
Results
There were no significant differences in compression depth (mean, 53.69 vs 53.28; P = .70) and compression rate (mean, 111.48 vs 113.38; P = .23) between the ResuGlove CPR and Standard CPR groups. However, the group using ResuGlove had a higher percentage of complete chest releases between compressions ( P = .008). Furthermore, the ResuGlove CPR Group had a significantly higher percentage of participants who performed chest compressions with adequate compression depth (82.8% vs 41.4%, P = .001) and compression rate (96.6% vs 72.4%, P = .012) compared with the Standard CPR Group. The ResuGlove usability score was calculated to be 70.4.
Conclusions
The newly developed ResuGlove significantly improved the quality of certain chest compression parameters, and the device's usability score was within the acceptable range.
... Expectations and opinion regarding the intervention were assessed through Expectations and opinion of treatment scales (Borkovec and Nau, 1972), satisfaction with the group sessions was evaluated using an ad hoc opinion questionnaire on group sessions. In addition, The System Usability Scale (SUS; Bangor et al., 2008;Castilla et al., 2023) was used to assess the usability as well as the acceptance of the online platform. (Mira et al., 2019) were used to evaluate the symptoms of anxiety and depression, respectively. ...
... Regarding system usability after the first use of the web platform (post-module 1), participants had a mean score of 84.02 (SD = 12.34) in the SUS. The global score can be expressed as a percentage, with values ranging from 0 to 100, so the results show an excellent usability and acceptability of the platform according to Bangor et al. (2008). Usability was reassessed after having used the online platform to complete the treatment (post-treatment evaluation), and the patients had a mean score of 80.98 (SD = 15.86) in the SUS, so usability remained high. ...
... They are also in line with studies on treatments delivered via videoconference, which show high patient satisfaction (Steel et al., 2011). With respect to usability, patients' means were above desirable (≥70) and around 'excellent' (Bangor et al., 2008). In short, it seems feasible to administer transdiagnostic treatment in a blended and group format, and this is in line with previous literature (Nunes-Zlotkowski et al., 2024;Peris-Baquero and Osma, 2023). ...
... The SUS includes 10 statements rated on a 5-point Likert scale (from 1=strongly disagree to 5=strongly agree). The score ranges from 0 to 100 with a score higher than 70 being considered acceptable [35]. We also collected the c-PDS data from the app, which were based on participants' logged entries into the app. ...
... Participants logged into the app an average of 18 (SD 14) days per month over the 12-week intervention period with the number of log-ins trending down over the duration of the intervention but these results were not statistically significant (Table S1 in Multimedia Appendix 5). The average SUS score was 80.9 (SD 17.3), which surpasses the usability quality benchmark threshold of 70, indicating a high level of usability [35]. Table S2 in Multimedia Appendix 5 shows the scores for individual SUS items. ...
... The PortfolioDiet.app was rated as usable, with the app surpassing the usability quality benchmark threshold [35]. While participants engaged often with the app over the 12 weeks, use gradually declined. ...
Background
The Portfolio Diet is a dietary pattern for cardiovascular disease (CVD) risk reduction with 5 key categories including nuts and seeds; plant protein from specific food sources; viscous fiber sources; plant sterols; and plant-derived monounsaturated fatty acid sources. To enhance implementation of the Portfolio Diet, we developed the PortfolioDiet.app, an automated, web-based, multicomponent, patient-facing health app that was developed with psychological theory.
Objective
We aimed to evaluate the effect of the PortfolioDiet.app on dietary adherence and its acceptability among adults with a high risk of CVD over 12 weeks.
Methods
Potential participants with evidence of atherosclerosis and a minimum of one additional CVD risk factor in an ongoing trial were invited to participate in a remote web-based ancillary study by email. Eligible participants were randomized in a 1:1 ratio using a concealed computer-generated allocation sequence to the PortfolioDiet.app group or a control group for 12 weeks. Adherence to the Portfolio Diet was assessed by weighed 7-day diet records at baseline and 12 weeks using the clinical Portfolio Diet Score, ranging from 0 to 25. Acceptability of the app was evaluated using a multifaceted approach, including usability through the System Usability Scale ranging from 0 to 100, with a score >70 being considered acceptable, and a qualitative analysis of open-ended questions using NVivo 12.
Results
In total, 41 participants were invited from the main trial to join the ancillary study by email, of which 15 agreed, and 14 were randomized (8 in the intervention group and 6 in the control group) and completed the ancillary study. At baseline, adherence to the Portfolio Diet was high in both groups with a mean clinical Portfolio Diet Score of 13.2 (SD 3.7; 13.2/25, 53%) and 13.7 (SD 5.8; 13.7/25, 55%) in the app and control groups, respectively. After the 12 weeks, there was a tendency for a mean increase in adherence to the Portfolio Diet by 1.25 (SD 2.8; 1.25/25, 5%) and 0.19 (SD 4.4; 0.19/25, 0.8%) points in the app and control group, respectively, with no difference between groups (P=.62). Participants used the app on average for 18 (SD 14) days per month and rated the app as usable (System Usability Scale of mean 80.9, SD 17.3). Qualitative analyses identified 4 main themes (user engagement, usability, external factors, and added components), which complemented the quantitative data obtained.
Conclusions
Although adherence was higher for the PortfolioDiet.app group, no difference in adherence was found between the groups in this small ancillary study. However, this study demonstrates that the PortfolioDiet.app is considered usable by high-risk adults and may reinforce dietitian advice to follow the Portfolio Diet when it is a part of a trial for CVD management.
Trial Registration
ClinicalTrials.gov NCT02481466; https://clinicaltrials.gov/study/NCT02481466
... Na análise da usabilidade, foi seguido a classificação proposta por Bangor Bangor et al. [2008], que categoriza a usabilidade de um sistema em uma escala que vai de "Muito Ruim"a "Melhor Imaginável". Os resultados da avaliação do RODI revelaram uma avaliação positiva da usabilidade do sistema, a média geral de satisfação foi de 94%. ...
... Os resultados da avaliação do RODI revelaram uma avaliação positiva da usabilidade do sistema, a média geral de satisfação foi de 94%. Ao compararmos esses resultados com a escala proposta por Bangor Bangor et al. [2008], o RODI se enquadra na categoria um sistema considerado Melhor Imaginável. ...
... A avaliação dos usuários revelou uma média de usabilidade de 94%. Essa pontuação coloca o RODI na categoria "excelente"da escala proposta por Bangor Bangor et al. [2008], refletindo a percepção positiva dos usuários em relação à eficácia e facilidade de uso do sistema. ...
Este artigo propõe o desenvolvimento do RODI (Robô da Organização Didática Inteligente), um chatbot destinado a auxiliar os estudantes do Instituto Federal do Ceará, campus Tianguá. As orientações desse chatbot são em relação aos processos acadêmicos, com foco específico nos procedimentos estabelecidos pelo Regulamento de Organização Didática (ROD). O chatbot foi criado para facilitar o acesso às informações acadêmicas por meio de tecnologias de conversação, eliminando a necessidade de deslocamento até a coordenação ou busca em grupos nas redes sociais. Adicionalmente, foi realizada uma avaliação de usabilidade do RODI, que revelou uma percepção positiva por parte dos usuários em relação à eficácia e facilidade de uso do chatbot.
... Since its introduction by Brooke, the SUS has been widely recognized for assessing the usability of various products and systems. It is considered a highly robust, popular, valid, and reliable instrument for usability evaluation [68], [69], [70], [71], [72]. Furthermore, several studies have highlighted the application of SUS testing by teachers to evaluate usability [50], [51], [52]. ...
... Where, X the average score, ∑x the total SUS score and n number of respondents Fig. 9. A comparison of the overall SUS score [68] As shown in Figure 9, a higher SUS score indicates a more acceptable and userfriendly system. Additionally, SUS scores can be categorized into letter grades ranging from A to F, with A representing the highest level of usability and F the lowest [79]. ...
... As shown in Table 7, the SUS score calculation results indicate an average score of 79.43. According to the SUS score assessment [68], the TUI model, which is used by teachers as an educational tool for teaching geometric shapes in kindergarten, falls within the "good" and "acceptable" categories and ranks in the 4th quartile (highest). In addition to the manual calculations, we also employed the SUS analysis toolkit for further analysis [82]. ...
Tangible user interfaces (TUIs) offer significant educational benefits, but their use in kindergarten, particularly for teaching geometric shapes, remains underexplored, with research largely focused on elementary education or higher levels. Moreover, usability evaluations assessing the ease of use of such tools by teachers as end-users are rarely conducted. This study aims to address these gaps by designing and evaluating TUI interactive surfaces as an aid for teaching geometric shapes in kindergarten. The proposed TUI design was evaluated for usability using the System Usability Scale with kindergarten teachers, achieving a score of 97.43, categorized as ‘Good’ (Grade A). This score indicates superior and acceptable performance, placing it in the highest quartile (4th quartile). Furthermore, functionality testing of the TUI system demonstrated a 100% success rate in its application as a geometric shape learning media. These findings suggest that the proposed TUI design is a viable and effective tool for teaching geometric shapes in kindergarten, offering a practical and user-friendly solution for educators. The study’s limited sample size, subject focus, and lack of teacher diversity affect generalizability, highlighting the need for future research to expand TUI applications, assess long-term impacts, and include diverse teacher demographics.
... 30 Our study highlighted the usability of TBA as an acceptable product for hearing screening. 31 There was however a negative usability score association with older patients, as found by Bangor et al, with verbal feedback indicating less familiarity with technology. 31 This emphasises the importance of assessing digital literacy when considering novel OMP integration and using a flexible approach. ...
... 31 There was however a negative usability score association with older patients, as found by Bangor et al, with verbal feedback indicating less familiarity with technology. 31 This emphasises the importance of assessing digital literacy when considering novel OMP integration and using a flexible approach. As a secondary aim, our OMP implementation resulted in the detection of hearing loss at pre/early-clinical stages enabling earlier diagnosis of ototoxicity and treatment changes. ...
Aminoglycoside antibiotics cause ototoxicity for which baseline audiometric testing is recommended but often not done. Barriers to successful implementation include limited availability of sound-booths and audiologists. An ototoxicity monitoring programme (OMP) was implemented using tablet-based audiometry (TBA) by non-audiologists.
A quality improvement project conducted over 1 year (19 April 2021 to 18 April 2022), using Plan Do Study Act (PDSA) cycles, monitored the adherence to the OMP using Shoebox Standard Edition application on iPads. Barriers to adoption were identified to determine potential solutions for improved adherence. Adult respiratory patients (cystic fibrosis (CF), bronchiectasis, non-tuberculosis mycobacteria (NTM) infection) aged 17–82 years receiving >1 day of intravenous aminoglycosides (IVAGs) at a single tertiary-referral hospital were included. Other reported outcomes were patient characteristics, risk factors associated with abnormal hearing and ototoxic shift.
73 patients were tested in the OMP (46 received ≥2 hearing tests) giving an overall adherence rate of 69% after 12 months. Patient identification using referral and reporting systems initially improved adherence from 36% to 88% (p=0.03) during PDSA 1. Barriers to successful adherence were staff availability and COVID-19 infection outbreaks (p=0.057). Older age (p<0.001), higher Body Mass Index (p=0.041), non-CF bronchiectasis (p=0.01), non-CF NTM (p=0.028) and higher Hearing Handicap Inventory for Adults scores (p=0.002) were significantly associated with abnormal baseline hearing. 78% with hearing loss were asymptomatic. Ototoxic shift was associated with gentamicin use compared to amikacin/tobramycin (p=0.027). TBA was associated with high usability in patients ≤50 years old.
TBA by non-audiologists was feasible and demonstrated good patient usability, permitting screening of patients within 72 hours of starting IVAG and earlier referrals for formal audiometry. Hearing loss and ototoxicity were detected at earlier stages, enabling more rapid decision-making and treatment modification.
... African swine fever (ASF) is a highly contagious and deadly disease affecting pigs, with no cure or limited vaccine available, posing a significant global challenge [1]. Since its arrival in the Philippines in 2019, ASF has devastated small-scale farms across 12 regions, leading to financial ruin for farmers due to pig culling and lost income [2]. ...
... It also highlights opportunities for targeted improvements to further enhance usability. This outcome aligns with previous research [1,14,15], emphasizing the importance of high SUS scores in fostering user satisfaction, adoption, efficiency, and positive brand perception. Overall, the elevated SUS score signifies the system's effectiveness in ASF management and its broader implications for user engagement and digital presence. ...
African swine fever poses a tough challenge to the swine industry, particularly in regions such as the Philippines, where its impact has been devastating. This paper introduces the development of a web-based GIS simulator tailored to model real-time African swine fever spread dynamics. Through iterative processes within the system development life cycle, the system was meticulously designed to address the specific needs of small-scale backyard farmers and government agencies. A thorough requirement analysis and stakeholder engagement defined the system’s core functionalities, including realtime disease monitoring, interactive mapping, and customized best practices dissemination. Implementation involved integrating these features into a user-friendly platform, ensuring usability and effectiveness. Rigorous testing confirmed the reliability and functionality of the system, with high system usability scale scores reflecting positive user perception. Deployment in a pilot municipality showcased the system’s potential to empower stakeholders with timely information and proactive measures against African swine fever outbreaks. The newly developed system for simulating African swine fever spread dynamics and mapping confirmed cases has received positive feedback and a high system usability scale score. It is recommended to further explore and integrate the newly developed system for simulating African swine fever spread dynamics and mapping confirmed cases. To build on this success, stakeholders are encouraged to consider gradually expanding the use of the system, gathering ongoing feedback for improvement, offering user support and training, nurturing partnerships with research institutions and system usability scale control agencies, and raising awareness of the system’s benefits.
... Secondary endpoints are DiGA use after 16 weeks, change in depressive symptoms (measured by PHQ-9 [26]), and change in treatment of mental disorder after 8 and 16 weeks. The usability of the DiGA will be assessed by the System Usability Scale (SUS) [27] and the User Experience Questionnaire (UEQ) [28] at week eight and 16. ...
... The participant timeline is shown in the table below [26][27][28][29][30][31]. ...
Background
In the face of extensive waiting times for outpatient psychotherapy, prescriptible digital health applications (DiGA) are a useful and effective addition to the range of available therapy options for patients with mild to moderate depression. However, older adults face a particular challenge in implementing DiGA since higher age is a decisive predictor of lower digital health literacy. The necessity of an independent use of the prescribed DiGA is therefore associated with challenges for older patients and providers. In practice, it is crucial not to leave patients, especially older adults, alone after prescribing, but to maintain close contact to overcome technical and motivational barriers and to ensure that the novel application is used. However, this is difficult for physicians and psychotherapists due to the critical healthcare system situation in Germany described above. Another support system is needed. Hence, the main hypothesis of this study is that the additional implementation of digital nurses leads to a higher percentage of older patients with depressive symptoms starting DiGA use compared to a prescription and information alone.
Methods
Two DiGA for mild to moderate depression in older patients were available and permanently approved at the time of the funding application. Using the most suitable one of them, as shown in a pilot study, the feasibility of implementation will be examined within a randomized proof of concept study. In our study, a digital nurse is trained to support patients with depression in using a DiGA. The main outcome is DiGA use (first session started: yes/no) after 8 weeks. Major secondary outcomes are patient-relevant outcomes, feasibility of recruitment and intervention, and factors moderating the effect or predicting DiGA use in the target group. Best practice guidelines will be elaborated on how to support and improve DiGA prescription and successful use in this population.
Discussion
In Germany, the approved DiGA are currently little used, especially by people with a low digital affinity. This proof of concept study will use the example of older people with depressive disorders to show whether it is possible to increase the usage rate of a DiGA with the support of a digital nurse so that a DiGA can become a serious therapy option.
Trial registration
DRKS: DRKS00033535. Registered on February 2, 2024.
... The usability and acceptability of the VR intervention were assessed using an adapted version of the original System Usability Scale (SUS), developed by Brooke. 15,16 This widely used tool evaluates the usability of a range of systems on a scale from 0 to 100, where scores above 68 are considered above average. The adaptation to an 8-item format was made to better suit the older adult population with MCI, reducing respondent fatigue while retaining the scale's robustness and core usability evaluation framework. ...
... • SUS: Scores for patients and professionals were normalized to a 0-100 scale following Brooke's methodology. 15,16 Scores above 68 were interpreted as above average, and scores closer to 100 reflect higher levels of usability and user satisfaction. Started with passive experiences (e.g., mindfulness) and progressed to interactive, immersive tasks. ...
With the global population rapidly aging, cognitive decline, including mild cognitive impairment (MCI), is becoming increasingly common. MCI impacts essential cognitive functions like memory and attention, significantly reducing the quality of life for those affected and increasing the burden on healthcare systems. Virtual reality (VR) has emerged as a promising tool for enhancing cognitive rehabilitation by creating immersive and engaging environments. This pilot study aimed to evaluate the usability, acceptability, and preliminary cognitive benefits of VR-based cognitive stimulation exercises for older adults with MCI. The intervention consisted of eight VR and augmented reality sessions over 4 weeks, integrating interactive tasks and 360-degree videos tailored to enhance cog-nitive flexibility, visuospatial processing, and memory. Forty-five participants enrolled, with 40 completing the full intervention and final postevaluation, and usability feedback was collected from 42 participants, including two who completed partial sessions. Usability was assessed using an adapted System Usability Scale (SUS). Patients rated the VR content (81.9) and headset usability (86.5) highly reflecting strong engagement and comfort. Four healthcare professionals also provided a high SUS score (85.2), emphasizing the program's clinical relevance and feasibility for integration into rehabilitation workflows. Cognitive assessments indicated a modest improvement in Trail Making Test A scores (p = 0.037), which may suggest enhanced cognitive flexibility. Other measures, including Mini-Mental State Examination, Montreal Cognitive Assessment, Digit Span, and Symbol Digit Modalities Test, showed trends toward improvement but did not reach statistical significance. These exploratory findings align with the short duration of the intervention and the study's feasibility focus. The iterative, user-centered design process ensured the intervention was accessible, safe, and adaptable to participants' needs and clinical workflow. These findings suggest that VR-based cognitive stimulation programs are highly usable and acceptable and show potential for improving cognitive performance in older adults with MCI. Further research with larger sample sizes and longer intervention periods is warranted to evaluate clinical impact and functional outcomes.
... The odd-numbered items (1, 3, 5, 7, and 9) are positively worded and are scored minus 1 by the respondent, whereas the even-numbered items (2, 4, 6, 8, and 10) are negatively worded and are scored 5 minus. The total SUS score was obtained by summing all item scores and multiplying by 2.5, resulting in a range of 0 to 100 with higher scores indicating better system usability(Bangor et al. 2008). ...
Labour care involving clinical decision-making and pain management is challenging for nursing students. Existing virtual teaching methods, such as spherical video-based virtual reality (SVVR), predominantly rely on visual and auditory stimuli, thus they provide no haptic feedback or interactivity. This study aimed to develop and evaluate a virtual reality synchronous haptic feedback system (VR-SHFS) to improve nursing students’ skills, knowledge, and empathy development regarding labour care. A randomised controlled trial was conducted with nursing students randomly assigned to experimental (VR-SHFS, n = 40) and control (SVVR, n = 40) groups. The experimental group engaged in haptic feedback and dual-perspective learning, whereas the control group learnt through 360° panoramic videos. Learning outcomes were assessed at the baseline (T0), immediately (T1) and two months (T2) after the intervention. Learning satisfaction was evaluated using the system usability scale (SUS) and a focus group. The data analysis revealed that the VR-SHFS group demonstrated significantly improved labour care skills and empathy at T1 and T2, as well as knowledge application at T2 compared to the SVVR group. From the students’ perspective, VR-SHFS can enhance clinical realism through haptic feedback and interactive learning, facilitating skill transfer, although there is a need for additional instructions and more time to become familiar with the system. VR learning approaches that integrate haptic feedback and dual perspectives effectively improve nursing students’ learning outcomes in labour care and outperform those based on SVVR. Improved VR technology and its interactive mechanism may further enhance learning outcomes and broaden clinical application.
... In iterative testing, the number of users is usually controlled between 5-10. There is an insignificant correlation between age and SUS scores (SUS scores decrease with increasing age), and no gender effect [33,34]. Crouch and McKenzie proposed that less than 20 participants in a qualitative study can help mitigate some of the bias and validity threats inherent in qualitative research [35]. ...
A-pillars can obstruct the driver's view, creating a blind spot that affects driving safety. Therefore, it is necessary to design a new device that allows drivers to quickly identify traffic objects despite the visual obstruction of the A-pillars. This study proposes a new alternative solution. The design consists of two parts: a monitor and a surveillance camera. The monitor is located on the inside of the A-pillar and displays real-time traffic information captured by the surveillance camera outside the car, covering the driver's blind spot. The surveillance camera is mounted on the outside of the A-pillar and monitors the traffic information in the driver's blind spot via video. In a static simulated driving environment, researchers conducted experiments using this device to detect traffic objects, identify traffic objects, and measure detection difficulty. Finally, the experimental data were analyzed using SPSS V26 software. The results show that the proposed monitor system significantly outperforms the A-pillar in detecting traffic objects, detection difficulty, and the accuracy of identifying traffic objects. The proposed monitoring system can help drivers overcome the visual obstruction of the A-pillars, improve driving safety, provide better protection for pedestrians, other drivers, and animals, and assist automotive developers in overcoming the limitations of traditional A-pillars to develop car designs with fewer limitations.
... The mean SUS score for the CCT was 83.51 out of 100, indicating an 'above average' usability rating and an 'excellent' level of acceptability. 21 This high usability score suggests that participants found the system intuitive and easy to use. Furthermore, the participants achieved a mean compliance rate of 92.7%, which reflects a high level of adherence to the intervention. ...
Background
Schizophrenia is characterised by pervasive cognitive deficits that significantly impair daily functioning and quality of life. Pharmacological treatments have limited efficacy in addressing these deficits, highlighting the need for adjunctive interventions like computerised cognitive training (CCT).
Aims
This study aimed to evaluate the effects of a 30-session CCT programme on mental well-being and cognitive performance in individuals with schizophrenia. Additionally, it assessed the usability and acceptability of CCT in this population.
Methods
A double-blind, randomised clinical trial was conducted with 54 participants assigned to intervention and control groups. Cognitive and mental health outcomes were assessed using validated tools such as the Depression Anxiety Stress Scale 21, the Warwick-Edinburgh Mental Wellbeing Scale and the Cambridge Neuropsychological Test Automated Battery. Usability was measured with the System Usability Scale (SUS). Assessments were conducted at baseline, post-intervention and 3 months post-follow-up.
Results
The CCT intervention significantly improved mental well-being, reduced stress and enhanced working memory (paired associate learning, spatial working memory and spatial span) compared with controls. However, no significant effects were observed for anxiety, depression or executive function. Usability scores were high (SUS=83.51), and compliance rates were strong (92.7%), indicating favourable participant engagement.
Conclusion
CCT demonstrated potential as an adjunctive treatment for schizophrenia, with significant improvements in targeted cognitive and mental health domains. The high usability and compliance rates support its feasibility for broader implementation. Further research is needed to optimise protocols and explore long-term benefits. CCT offers a promising approach to addressing mental health and cognitive challenges in schizophrenia, particularly for stress and working memory. Its usability and acceptability suggest it could be seamlessly integrated into clinical practice.
... To evaluate UX, selecting the appropriate measurement items for surveys is an important yet challenging [12,13,20,21]. Consequently, evaluation has often relied on long-established questionnaires [1,2,5,20] such as SUS [6], NASA-TLX [7], UEQ [10], QUIS [9], UBS [8], and more. Although these surveys are credible and widely adopted, they primarily focus on general constructs such as usability, utility, and ease of use [4,18]. ...
Researchers often struggle to develop measurement items and lack a standardized process. To support the design process, we present UX Remix, a system to help researchers develop constructs and measurement items using large language models (LLMs). UX Remix leverages a database of constructs and associated measurement items from previous papers. Based on the data, UX Remix recommends constructs relevant to the research context. The researchers then select appropriate constructs based on the recommendations. Afterward, selected constructs are used to generate a custom construct, and UX Remix recommends measurement items. UX Remix streamlines the process of selecting constructs, developing measurement items, and adapting them to research contexts, addressing challenges in the selection and reuse of measurement items. This paper describes the implementation of the system, the potential benefits, and future directions to improve the rigor and efficiency of measurement design in human-computer interaction (HCI) research.
... The order of the conditions was counterbalanced. The different trials had randomized sources of uncertainty (four starting locations for the workpiece, three rotations for the jig, and the color of the [28] and Systems and Usability Scale (SUS) [29] questionnaires. We also recorded the total task time and amount of user input. ...
There are a variety of mechanisms (i.e., input types) for real-time human interaction that can facilitate effective human-robot teaming. For example, previous works have shown how teleoperation, corrective, and discrete (i.e., preference over a small number of choices) input can enable robots to complete complex tasks. However, few previous works have looked at combining different methods, and in particular, opportunities for a robot to estimate and elicit the most effective form of assistance given its understanding of a task. In this paper, we propose a method for estimating the value of different human assistance mechanisms based on the action uncertainty of a robot policy. Our key idea is to construct mathematical expressions for the expected post-interaction differential entropy (i.e., uncertainty) of a stochastic robot policy to compare the expected value of different interactions. As each type of human input imposes a different requirement for human involvement, we demonstrate how differential entropy estimates can be combined with a likelihood penalization approach to effectively balance feedback informational needs with the level of required input. We demonstrate evidence of how our approach interfaces with emergent learning models (e.g., a diffusion model) to produce accurate assistance value estimates through both simulation and a robot user study. Our user study results indicate that the proposed approach can enable task completion with minimal human feedback for uncertain robot behaviors.
... SUS is one of the most widely used testing instruments. SUS was developed by Brooke in 1986 to measure the usability value of a system "quick and dirty," but still reliable (Bangor et al., 2008;U.S. Dept. of Health and Human Services, 2012). We use the Indonesian version (Sharfina & Santoso, 2016) for the SUS questions in this study. ...
This study aims to provide recommendations for interaction design to enhance students' task interpretation , as one of the crucial aspects of Self-Regulated Learning (SRL) in online learning environments. Utilizing the Engineering Design Metacognitive Questionnaire (EDMQ), open-ended questions, and in-depth interviews, this study examines the implementation of task interpretation among students in software development courses at a reputable public university in Indonesia. The interaction design was developed considering Schneiderman's Eight Golden Rules, identifying students' task interpretation implementation and user needs. Evaluation results revealed an Acceptable SUS Score. The prototype is expected to be applied to e-learning systems to help improve students' task interpretation.
... Usability and acceptablility of the electronic depth gauge was assessed with the validated System Usability Scale (SUS) containing 10 items [9,10] (Fig. 3). Each item is rated on a 5-point Likert scale with anchors: 1-strongly disagree and 5-strongly agree. ...
Introduction
Orthopedic drilling and screw placement require precision to obtain the effect of bicortical screwing and avoid complications such as soft tissue damage. Traditional manual depth gauges, while effective, are prone to human error. This study evaluates the usability and acceptability of an automated electronic depth gauge compared to a conventional manual depth gauge in a preclinical setting.
Methods
A randomized controlled cross-over trial was conducted with 45 participants of varying clinical experience (12 medical students, 11 resident doctors, and 22 consultants, 14/22 were senior consultants) at Aarhus University Hospital. According to randomization, participants used either the “electronic -> manual” or “manual -> electronic” depth gauge and subsequently the sequence “with -> without” or “without -> with” soft tissue protector to measure screw length in sawbones. Efficiency was assessed by measuring time (savings), and usability was evaluated using the System Usability Scale (SUS).
Results
The electronic depth gauge significantly reduced the time required to measure screw length, with medical students saving the most time, p < 0.001. 65/331 (20%) of screws were changed, with no difference in the rate of screw changes between manual and electronic depth gauges, p = 0.76. Level of experience was significantly correlated with the rate of screw changes, p = 0.03. Resident doctors changed fewer screws than medical students, with no difference between the remaining groups. The median SUS score was 87.5, indicating excellent usability. Most participants rated the tool as “best imaginable” or “excellent.”
Conclusion
The electronic depth gauge was time-efficient and surgeons rated it as highly usable. However, screw exchanges were not significantly minimized in this study, which might be mitigated by proper training and awareness of device-specific recommendations. The effect of electronic depth gauges on screw exchange rates and its clinical applicability warrents further research.
... The items are rated on a 5-point Likert scale with values between 0 (= strongly disagree) and 4 (= strongly agree). Previous research by Bangor et al. [46] demonstrated good internal consistency for the SUS, with Cronbach's = .91. Furthermore, the SUS has been shown to correlate highly (r = 0.91) with the Usability Metric for User Experience (UMUX) [47], which itself closely aligns with ISO 9241-11 usability standards, showing high sensitivity for slightly differences between systems. ...
Virtual Reality (VR) has paved its way into experimental psychology due to its capacity to realistically simulate real-world experiences in a controlled way. Theoretically, this technology opens the possibility to conduct experiments anywhere in the world using consumer hardware (e.g. mobile-VR). This would allow researchers to access large scale, heterogeneous samples and to conduct experiments in the field in cases where social distancing is required – e.g. during the COVID-19 pandemic. Here, we investigate the feasibility of carrying VR experiments in the field using mobile-VR through a stress inductive (public speaking task) and a relaxation (nature) task and contrast them with results in the laboratory (HTC Vive and mobile-VR). The first experiment employed a 2 (device: HTC Vive Pro (HMD) versus Wearality Sky VR smartphone adapter) x 3 (audience: ‘none’, ‘attentive’, ‘inattentive’) between-subjects design. Thirty-four participants took part in the experiment and completed a public speaking task. No significant difference was detected in participants’ sense of presence, cybersickness, or stress levels. In the second experiment, using an inexpensive Google Cardboard smartphone adapter a 3 (between: device setting) x 2 (within: task) mixed-design was employed. Sixty participants joined the experiment, and completed a public speaking and a nature observation task. No significant difference in participants’ sense of presence, cybersickness, perceived stress and relaxation were detected. Taken together, our results provide initial evidence supporting the feasibility and validity of using mobile VR in specific psychological field experiments, such as stress induction and relaxation tasks, conducted in the field. We discuss challenges and concrete recommendations for using VR in field experiments. Future research is needed to evaluate its applicability across a broader range of experimental paradigms.
... Moreover, the "Smart Responder" utilizes review content to filter and prioritize relevant venue information. To evaluate how well the solution achieves the objective, we measure the usability score of the entire system (Bangor et al., 2008). ...
Online customer feedback management (CFM) is becoming increasingly important for businesses. Providing timely and effective responses to guest reviews can be challenging, especially as the volume of reviews grows. This paper explores the response process and the potential for artificial intelligence (AI) augmentation in response formulation. We propose an orchestration concept for human–AI collaboration in co-writing within the hospitality industry, supported by a novel NLP-based solution that combines the strengths of both human and AI. Although complete automation of the response process remains out of reach, our findings offer practical implications for improving response speed and quality through human–AI collaboration. Additionally, we formulate policy recommendations for businesses and regulators in CFM. Our study provides transferable design knowledge for developing future CFM products.
... First we tested the usability of the system by means of the System Usability Scale 8 (Brooke, 1995;Sauro, 2011;Brooke, 2013). The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including web sites, cell phones, interactive voice response systems, TV applications, and more (Bangor et al., 2008). SUS provides a quick and dirty, reliable tool for measuring the usability. ...
Chatbots serve as valuable instruments for enhancing students' educational experience and aiding them in their day-to-day academic tasks. Advances in Generative AI (GAI) have ushered in increasingly sophisticated and adaptive chatbots, with ChatGPT and DALL⋅E being prime examples. ChatGPT excels at generating text-based answers across diverse areas of knowledge, while DALL⋅E is adept at converting text-based concepts into visual imagery. These technologies are increasingly used by students across various levels of education. In this study, we introduce AI4Design, a web-based system designed to assist design students with their course projects by acting as an intelligent chatbot. The field of design is propitious for such work because of the increasing use of technology and the necessity of introducing its critical use during study. Comprising two integrated modules, the system is based on a two-step workflow. The first step is anchored on ChatGPT, enabling students to prompt questions and receive answers. The second step allows for the generation of one or more images based on the system's answer to the initial question. Our research assesses whether our system can offer valuable insights and inspiration to students in their design work. We conducted an exploratory study in the Design domain involving 31 students from the Lebanese American University. Over a two- to three-day period, participants used the AI4Design system to enhance their projects. A subsequent evaluation of their work indicated improvements in conceptual clarity and visual outputs that highlighted a measurable increase in creativity, supporting the efficacy of both the system and its foundational learning model, which will be confirmed in the future through a large-scale experimental study. Meanwhile, our study suggests that in the iterative design process, GAI can assist students in making better decisions by giving them just-in-time access to a broader palette of possibilities.
... To get systematic feedback from the patients concerning the app's usability, content, and acceptability, the project group developed a self-report evaluation form (Supplementary material 2), which was handed out to all patients after completing their project period. The form contained questions pertaining specifically to MinDag, in addition to more general questions adapted from the System Usability Scale (SUS; Bangor et al. 2008). There were twenty items in total concerning app use; the first ten items were scored on a Likert scale from one to five (completely disagree-completely agree), whereas the remaining ten had varying types of scoring instructions (please refer to supplementary material for specifics). ...
Background
The app “MinDag” (MyDay) was developed as a tool for monitoring mood, symptoms and illness-relevant behaviour in the initial treatment of bipolar disorder. Digital self-monitoring may provide patients and clinicians with valuable data for tailoring treatment interventions. This study aims to evaluate the practical use and clinical implications of integrating MinDag in the early treatment of bipolar disorder from the perspectives of both patients and clinicians.
Methods
The MinDag app includes six content modules covering mood, sleep, functioning/activities, substance use, emotional reactivity, and psychotic symptoms. Patients were asked to use the app for six months, and automated feedback based on the app registrations was delivered to the patients’ clinicians biweekly. The study involved quantitative evaluations completed by patients (n = 20), as well as interviews with patients (n = 7) and clinicians (n = 2).
Results
Overall, the patients reported that they felt that MinDag was safe, relevant and easy to use, although technical difficulties such as too many automated reminders and need for reinstallations were reported. The patients appreciated the potential for increased awareness of their mental health, but expressed a desire for direct access to their data. Clinicians found the visual reports and feedback useful for tailoring treatment, even though the alert system for high-risk variables needed refinement. The quantitative evaluations indicated a positive general reception, with suggestions for improvement in usability and accessibility.
Conclusion
The MinDag app shows promise as a tool for enhancing the treatment of bipolar disorder by facilitating self-monitoring and providing actionable data to clinicians. However, technical issues and the need for direct patient access to data must be addressed. Development of digital tools to support the treatment of bipolar disorder and other mental health conditions is resource demanding, and there is a need to clarify criteria to establish proof of concept to guide the selection of tools for upscaling and implementation.
... The System Usability Scale (SUS) test [Brooke et al. 1996;Bangor et al. 2008] was included in the user study to quantify how easy and intuitive it is to use VolcanoVR. A SUS score is a value out of 100 that can be compared to other SUS surveys on a distribution curve. ...
With the increasing size and complexity of geological datasets relating to volcano monitoring and research, effective visualisation can be challenging. Here, we demonstrate the possibilities of volcanic data visualisation utilising virtual reality (VR) and 3D game engine technology to create a robust and adaptable program called VolcanoVR. VolcanoVR can display multiple complex datasets and has been developed to investigate volcanoes in the Taupō Volcanic Zone, New Zealand. To assess the usability and effectiveness of VolcanoVR a survey was conducted, involving 33 participants, ranging in education level and previous experience with volcanic data and VR. Results indicate VolcanoVR is easy to use with high immersion ratings and acceptable system useability and mental demand scores, with minor improvements made following the survey. Limited variability across user experience levels indicates the program is useable for a broad range of geoscientists. We have made the source code for VolcanoVR freely available so that it can be easily adapted and applied worldwide to a range of different volcanoes and geological settings.
... SUS produced a median score of 77.5, which is indicative of "good" usability and suggests that the design choices, such as the large fonts and the simple navigation, were effective. Scores in the 70 s and 80 s indicate that the game's acceptability is promising [7,52]. The participants' responses to positively worded items (e.g., ease of use) and the lower scores on items assessing perceived complexity (e.g., unnecessarily complex) indicate that the design approaches were successful and appropriately tailored for the target group. ...
Highlights
What are the main findings? MarketMind AR showed good usability and high engagement, with participants reporting competence, immersion, and low frustration.
Players improved performance over time, with faster completion, higher scores, and better PIN recall, suggesting cognitive benefits.
What is the implication of the main finding? AR-based cognitive training is cost-effective and accessible, using smartphones/tablets and built-in sensors instead of expensive VR headsets.
The familiar supermarket setting enhances engagement and usability, supporting intuitive and long-term use.
Abstract
The aging population increases the need for accessible interventions for cognitive training of the elderly to preserve cognitive health. Serious games have been widely used for this purpose, with many existing applications leveraging virtual reality (VR) technology. In contrast, this study explores the potential of augmented reality (AR) for cognitive training. The literature review shows that cognitive training interventions typically employ supermarket-themed serious games that are used extensively in such interventions. MarketMind AR is a supermarket-themed serious game that was created to train memory, attention, and executive function using mobile phone sensors such as cameras, accelerometers, and gyroscopes to interact and recognize the environment. Fifteen older adults participated in a three-attempt trial and completed the System Usability Scale (SUS), the in-game Game Experience Questionnaire (iGEQ), and an adapted version of the Unified Theory of Acceptance and Use of Technology (UTAUT) questionnaires. Qualitative interviews and in-game data (e.g., completion times, PIN recall) were also examined. The results indicated that participants had a positive experience, confirming ease of use, immersive appeal, and perceived cognitive benefits, despite some difficulties with environment scanning and object detection. The results provide evidence that an AR supermarket game leveraging mobile sensors has the potential to be an effective cognitive training tool for older adults.
... Based on adaptations of the work [16], a concept test was carried out with 15 people to find out how they felt about interacting with the robot and various aspects of integration between the systems. The scores ranged from 1 (strongly disagree with the statement) to 5 (strongly agree with the statement), and the results are shown in Table I. ...
With the evolution of Artificial Intelligence (AI) technologies, there is a growing demand for Intelligent Personal Assistants (IPAs) that allow fluid communication with humans. However, there are still several challenges to overcome, such as difficulty in understanding nuances in some contexts. Therefore, this study seeks to mitigate these gaps by presenting a modular and scalable architecture for integrating IPAs into autonomous service robots. The architecture allows communication between various robot systems and peripherals with AI mechanisms, including Large Language Model (LLM) and Natural Language Processing, which seek to improve the user experience. The proposed architecture was applied to developing a service robot designed to guide and interact with people in a university environment, incorporating the RASA framework with an LLM for natural language processing and response generation. The paper discusses the adopted technologies, the current state of development, the difficulties encountered, and the analysis of the first feedback from volunteers.
... Scores were calculated by summing the converted values and multiplying by 2.5. Based on Bangor et al's [28] cutoffs, SUS scores were categorized as follows: excellent (85.58-100), good (72.75-85.57), OK (52.01-72.74), ...
Background: Physical activity (PA) is vital for physical and mental health, but many college students fail to meet recommended levels. Artificial intelligence (AI)-powered gamification interventions through mobile app have the potential to improve PA levels among Chinese college students. Objective: This study aimed to assess the feasibility and usability of an AI-powered gamification intervention.
... with a standard deviation of approximately 10.01, indicating that participants generally provided high and consistent usability ratings. Previous research has examined the interpretation of SUS scores in usability assessments, highlighting that higher scores are associated with superior usability experiences [62]. Bangor et al. suggest that products with scores above 70 are generally considered usable, while those in the high 70s to upper 80s indicate better usability. ...
Episodic Future Thinking (EFT) is an intervention that involves vividly imagining personal future events and experiences in detail. It has shown promise as an intervention to reduce delay discounting - the tendency to devalue delayed rewards in favor of immediate gratification - and to promote behavior change in a range of maladaptive health behaviors. We present EFTeacher, an AI chatbot powered by the GPT-4-Turbo large language model, designed to generate EFT cues for users with lifestyle-related conditions. To evaluate the chatbot, we conducted a user study that included usability assessments and user evaluations based on content characteristics questionnaires, followed by semi-structured interviews. The study provides qualitative insights into participants' experiences and interactions with the chatbot and its usability. Our findings highlight the potential application of AI chatbots based on Large Language Models (LLMs) in EFT interventions, and offer design guidelines for future behavior-oriented applications.
... The study's SUS scores ranged from awful to excellent. Using the adjective rating scale proposed by Bangor et al. (2008), we found that the aggregate SUS score, determined by participant evaluations, was good, with a mean SUS score of 69.25 (Table 3). Several factors were compared with SUS scores, but the most significant finding was the strong negative correlation between SUS scores and the number of clicks (r = -0.737, ...
In the era of digital transformation, the need for more efficient self-service technologies has increased, particularly after the COVID-19 pandemic, which highlighted the importance of reducing physical interactions. Although there is research on commercial self-service technologies, there is a shortage in studies focusing on self-service portals designed exclusively for internal use within an academic organizational environment. This study aims to address this gap by examining the usability of a self-service portal in a university in Sweden. The study adopts a mixed-methods approach, incorporating data collection techniques such as cognitive walkthrough, usability testing and semi-structured interviews. In addition, a system usability scale and interaction log file data were used to understand user behavior and satisfaction. Findings reveal important factors influencing the user experience, which are further materialized into design guidelines aiming to enhance the usability of the self-service portals.
... We begin our evaluation by measuring each participant's level of creativity and experience with creative processes, as these variables can influence interaction with the interface and understanding of the information presented. We then evaluate the usability performance of the prototype in terms of effectiveness, efficiency, and ease of use using the System Usability Scale (SUS) questionnaire [33] to ensure that the interface is intuitive and easy to use. In the second questionnaire, User Engagement Scale (UES) [34], we analyzed the participants' level of engagement with the prototype, as an engaging interface can increase users' attention and improve understanding of the information presented. ...
Background
Young adults living with HIV are less likely to engage in care and achieve viral suppression, compared to other age groups. Young adults living with HIV also have a high degree of self-efficacy and willingness to adopt novel care modalities, including mobile health (mHealth) interventions. Interventions to increase care engagement could aid young adults living with HIV in overcoming structural and social barriers and leveraging youth assets to improve their health outcomes.
Objective
The objective of the paper was to use an assets-based framework, positive youth development, and human-centered design principles to adapt an existing mHealth intervention, PositiveLinks (PL), to support care engagement for 18‐ to 29-year-olds with HIV.
Methods
We conducted a formative evaluation including semistructured interviews with 14 young adults with HIV and focus groups with 26 stakeholders (providers, nurses, case managers, and clinic staff). Interviews covered barriers to care, provider communication, and concerns or suggestions about mHealth interventions. The research team used thematic analysis to review interview transcripts. In the second phase, human-centered design processes informed adaptation of the existing PL platform using data from real-time use suggestions of 3 young adults with HIV. Throughout the formative evaluation and adaptation, a Youth Advisory Board (YAB) provided input.
Results
Young adults with HIV and stakeholders identified common elements of an mHealth intervention that would support care engagement including: the convenience of addressing needs through the app, online support groups to support interconnection, short videos or live chats with other young adults with HIV or providers, appointment and medication reminders, and medical information from a trustworthy source. Stakeholders also mentioned the need for youth empowerment. Concerns included worries about confidentiality, unintentional disclosures of status, urgent content in an unmoderated forum, and the impersonality of online platforms. Design suggestions from young adults with HIV included suggestions on appearance, new formatting for usability of the online support group, and prioritization of local content. Based on the feedback received, iterative changes were made to transform PL into Positive Links for Youth (PL4Y). Final votes on adaptations were made by the YAB. The overall appearance of the platform was changed, including logo, color, and font. The online support group was divided into 3 channels which support hashtags and content searches. The “Resources” and “Frequently Asked Questions” sections were condensed and revised to prioritize South Texas–specific content.
Conclusions
Our assets-based framework supported young adults with HIV and stakeholder input in the transformation of an mHealth intervention to meet the needs of 18- to 29-year-olds in South Texas. The human-centered design approach allowed young adults with HIV to suggest specific changes to the intervention’s design to support usability and acceptability. This adapted version, PL4Y, is now ready for pilot testing in the final phase of this implementation science project.
One ongoing challenge in human-robot interaction design is minimizing user misunderstandings and confusion. While engineers constantly improve the reliability of robots, the user’s mental model about robots and their limitations have to be addressed as well. In this work, we investigate ways to improve the human understanding about robots. For this, we propose FAMILIAR – FunctionAl user Mental model by Increased LegIbility ARchitecture, a transparent robot architecture with regard to the robot behavior and decision-making process. We conducted an exploratory online simulation user study (N=81) to evaluate two complementary approaches to convey and increase the knowledge about this architecture to non-expert users: a dynamic visualization of the system’s processes as well as an interface for defining the sequence of user and robot actions for teaching the robot, the interaction protocol. The experimental scenario consisted of teaching a robot about a simulated indoor environment. The results of this study reveal that the definition of an interaction protocol improves knowledge about the architecture measured via a questionnaire on knowledge of the different conceptual elements of the system (Sensors, Interaction Protocol, Behaviors, Preconditions, Actions, and how these interact: the Process). Furthermore, we show that with increased knowledge about the control architecture of the robot, users were significantly better in reaching the interaction goal. Moreover, we interestingly found that anthropomorphism may actually reduce interaction success. Our results support the crucial role of considering user mental models in robot architecture design.
Na última década, tem-se verificado um crescente interesse pelo desenvolvimento das tecnologias da informação e dos jogos sérios, aplicados em diferentes contextos, nomeadamente na educação e na saúde. A gamificação surge como uma abordagem inovadora cada vez mais utilizada para promover mudanças positivas nos processos cognitivos, tanto em cenários educacionais como clínicos, ao redefinir métodos tradicionais. Dessa forma, constitui-se simultaneamente como ferramenta pedagógica e terapêutica, favorecendo a motivação, o envolvimento ativo e a eficácia na aprendizagem, bem como na reabilitação cognitiva. Este capítulo aprofunda o modo como a gamificação pode influenciar as funções cognitivas e explora os mecanismos subjacentes ao seu potencial, ilustrando aplicações práticas e mostrando evidências científicas que sustentam a adoção desse recurso no desenvolvimento de novas competências e na intervenção em quadros de alterações cognitivas.
Os jogos desempenham um papel crucial no desenvolvimento humano, moldando e adaptando-se às diversas fases da vida e pelos interesses individuais. Desde as primeiras brincadeiras infantis, que estimulam a criatividade e a coordenação motora, até os jogos de tabuleiro estratégicos, que promovem o pensamento crítico e a socialização na adolescência e na idade adulta. Além disso, os videojogos ajudam os idosos a manterem-se mentalmente ativos e socialmente envolvidos. A essência dos jogos evolui ao longo do ciclo vital, adaptando-se às necessidades e capacidades de cada etapa da vida e ao contexto social, bem como ao desenvolvimento da tecnociência. Este capítulo explora detalhadamente como os jogos contribuem para o desenvolvimento e manutenção das capacidades motoras e cognitivas, destacando sua importância em cada etapa da vida. Analisaremos evidências científicas e exemplos práticos para ilustrar como diferentes tipos de jogos podem beneficiar o desenvolvimento infantil, a capacidade de resolução de problemas na juventude, a manutenção das habilidades cognitivas na idade adulta e a prevenção do declínio cognitivo na velhice.
Introduction
There is a limited understanding of therapist acceptance and use of robot‐assisted upper limb therapy (RT‐ULT) in routine practice. The aim of this study was to explore the factors that influence Australian therapist acceptance and use of RT‐ULT.
Methods
Two discipline‐specific focus groups were conducted involving occupational therapists ( n = 5) and physiotherapists ( n = 4) who had used RT‐ULT. Focus group questions were developed, and transcriptions analysed using the Theoretical Domains Framework (TDF). Additionally, participants scored the overall usability of the RT‐ULT device with the System Usability Scale (SUS).
Consumer and Community Involvement
There was no direct involvement from consumers or community in this study.
Findings
Nine of the 14 domains of the TDF were covered in depth by participants during the focus groups: environmental context and resources, beliefs about consequences, knowledge, skills, decision‐making, reinforcement, social influences, social/professional role and identity (single domain), and beliefs about capabilities. Physiotherapists recorded higher scores of the device on the SUS than the occupational therapists.
Conclusion
Both disciplines were accepting of RT‐ULT, but it was physiotherapists who predominantly used RT‐ULT in part due to the device being located in the physiotherapy rehabilitation gym. Other factors facilitating RT‐ULT acceptance in practice included (1) increase in repetitive, intensive independent practice for stroke survivors, (2) ease of use, (3) strong patient acceptance, and (4) implementation process being clinician‐led. Functional‐based UL practice took priority over RT‐ULT once stroke survivors demonstrated sufficient active movement and RT‐ULT was not used in isolation but part of a combination of UL interventions.
PLAIN LANGUAGE SUMMARY
There is a little known about what therapists think about using robot‐assisted upper limb therapy in their daily practice. The aim of this study was to explore Australian therapist perceptions of the use of robotics.
Focus groups were conducted separately with five occupational therapists and four physiotherapists who had used robotics at their rehabilitation facility. In addition to the focus groups, each therapist scored the user‐friendliness of the robotic device by completing a short survey.
Both occupational therapists and physiotherapists believed the robotics was a beneficial addition to the rehabilitation facility. Physiotherapists used the device more than the occupational therapists with the device being located in the physiotherapy area of the rehabilitation facility. Therapists explained that robotics increased the amount of practice stroke survivors could do, was easy to use, and was motivating for stroke survivors. However, once stroke survivors had gained enough arm movement, the focus moved to practicing actual daily tasks rather than robotics. Also, robotics was not the only form of upper limb therapy offered to stroke survivors with multiple upper limb treatments used to aid their recovery.
Background
Detrimental smoking-related health outcomes warrant the investigation of novel smoking cessation interventions; the cessation program nuumi integrates digital behavioral therapy and an electronic cigarette (EC).
Objective
The relationship between program participation and smoking cessation among adults who smoke and are motivated to quit was investigated, as well as program acceptability, changes in smoking-related outcomes, including cigarettes per day (CPD), urges to smoke and psychophysiological health variables (perceived stress, mindfulness, cessation-related self-efficacy, life satisfaction, subjective psychophysiological health) and their associations with smoking cessation.
Methods
A prospective 6-month single-arm pilot study was conducted; 71 adults who smoked and were motivated to quit received a cognitive behavioral therapy (CBT) app, a closed-system EC, and pods containing decreasing nicotine concentrations. Online surveys were issued at baseline, and at 4, 8, 12, and 24 weeks post-baseline. Intention-to-treat (ITT) and complete-case analyses were conducted to assess self-reported 7-day point prevalence of smoking abstinence (PPA; primary outcome), 30-day PPA, and repeated PPA. T-tests and logistic regressions were used to assess changes in secondary outcomes CPD, urges to smoke, and psychophysiological health variables by smoking status at 12 and 24 weeks, and their relationship with cessation.
Results
Per ITT, self-reported abstinence rates were high at 12 weeks (39.4%), and 24 weeks (32.4%), as was 30-day PPA of 32.4% at both 12 and 24 weeks. Repeated PPA per ITT was 22.5% at both 12 and 24 weeks. Non-abstinent participants significantly reduced their CPD at 12 weeks ( t (34) = 6.12, p < 0.001), and at 24 weeks ( t (30) = 6.38, p < 0.001). Urges to smoke and perceived stress decreased, and mindfulness, cessation-related self-efficacy, life satisfaction and subjective psychophysiological health increased significantly (all p s < 0.05), predominantly in individuals who reported abstinence. Lower urges to smoke, lower perceived stress, and higher self-efficacy and subjective mental health were related to greater odds of cessation at 24 weeks (all p s < 0.05). Most participants rated the program as highly (43%) or moderately (54%) acceptable.
Discussion
Program participation seems to support cessation and improvements in smoking-related outcomes, but adjustments to the program may be needed to improve engagement and acceptability. Findings may inform the development of future trials and cessation programs.
Trial Registration
German Clinical Trials Register DRKS00032652, registered prospectively 09/15/2023, https://drks.de/search/de/trial/DRKS00032652
Background
School-partnered interventions may improve health outcomes for children with type 1 diabetes, though there is limited evidence to support their effectiveness and sustainability. Family, school, or health system factors may interfere with intervention usability and implementation.
Objective
To identify and address potential implementation barriers during intervention development, we combined methods in user-centered design and implementation science to adapt an evidence-based psychosocial intervention, the collaborative care model, to a virtual school-partnered collaborative care (SPACE) model for type 1 diabetes between schools and diabetes medical teams.
Methods
We recruited patient, family, school, and health system partners (n=20) to cocreate SPACE through iterative, web-based design sessions using a digital whiteboard (phase 1). User-centered design methods included independent and group activities for idea generation, visual voting, and structured critique of the evolving SPACE prototype. In phase 2, the prototype was evaluated with the usability evaluation for evidence-based psychosocial interventions methods. School nurses reviewed the prototype and tasks in cognitive walkthroughs and completed the Intervention Usability Scale (IUS). Two members of the research team independently identified and prioritized (1-3 rating) discrete usability concerns. We evaluated the relationship between prioritization and the percentage of nurses reporting each usability issue with Spearman correlation. Differences in IUS scores by school nurse characteristics were assessed with ANOVA.
Results
In the design phase, the partners generated over 90 unique ideas for SPACE, prioritizing elements pertaining to intervention adaptability, team-based communication, and multidimensional outcome tracking. Following three iterations of prototype development, cognitive walkthroughs were completed with 10 school nurses (n=10, 100% female; mean age 48.5, SD 9.5 years) representing different districts and years of experience. Nurses identified 16 discrete usability issues (each reported by 10%-60% of participants). Two issues receiving the highest priority (3.0): ability to access a virtual platform (n=3, 30% of participants) and data-sharing mechanisms between nurses and providers (n=6, 60% of participants). There was a moderate correlation between priority rating and the percentage of nurses reporting each issue (ρ=0.63; P=.01). Average IUS ratings (77.8, SD 11.1; 100-point scale) indicated appropriate usability. There was no difference in IUS ratings by school nurse experience (P=.54), student caseload (P=.12), number of schools covered (P=.90), or prior experience with type 1 diabetes (P=.83), suggesting that other factors may influence usability. The design team recommended strategies for SPACE implementation to overcome high-priority issues, including training users on videoconferencing applications, establishing secure forms for school data reporting, and sharing glucose data in real-time during SPACE meetings.
Conclusions
Cross-sector interventions are complex, and perceived usability is a potential barrier to implementation. Using web-based cocreation methods with community partners promoted high-quality intervention design that is aligned with end-user priorities. Quantitative and qualitative assessments indicated appropriate degree of usability to move forward with pilot-testing.
Objectives
To develop an optimal outline for an online dance injury surveillance system (ODISS) based on expert consensus from a Delphi survey and evaluate the system’s usability and feasibility within a fictional folk-dance academy setting.
Background
Dance is characterized by intensive training and performance, which can elevate susceptibility to injuries. Robust surveillance systems are required to understand and mitigate this injury risk. Current systems exhibit limitations and lack uniformity.
Methods
A Delphi panel of 35 stage 1 experts were recruited through snowball sampling from online professional networks, professional dance associations, organizations, and academies. Inclusion criteria were ≥18 years of age, fluent in English, and worked as dance academics or researchers, clinicians, instructors, managers, or competitive or professional dancers. These stage 1 experts were provided with a preliminary outline of a dance injury surveillance system that covered 6 domains: dancer demographics and screening, exposure monitoring, injury identification and classification, injury management, dancer recovery and system access. Stage 1 experts then voted on the importance of and provided feedback on different design elements within each domain across 2 Delphi survey rounds to determine a consensus system design outline. A pilot system was then developed and evaluated by stage 2 experts, who were end-users and included dance instructors and administrators from various dance academies/institutions recruited through direct invitations. These stage 2 experts utilized the System Usability Scale (SUS) and System Feasibility Measure (SFM) based on application to a fictional folk-dance injury scenario.
Results
In survey round 1, the stage 1 experts reached consensus to include 30 elements and exclude 3 elements and demonstrated mixed opinion on 30 elements that were revised into 14 elements for further consideration in round 2. All but 1 element reached consensus to include after round 2. System testing demonstrated low marginal usability (SUS score: 58.2% ± 11.6%). Stage 2 experts agreed or strongly agreed the system was feasible for 86.7% ± 10.3% of SFM statement ratings. Key areas for improvement identified from stage 2 expert feedback were a need for an improved user interface and strategies to reduce data entry time burdens. A common suggestion was to integrate more dropdown and checkbox response options within the interface to increase efficiency of data entry.
Conclusion
This study established a Delphi-consensus on the essential design elements for an ODISS. Expert evaluation resulted in a usable and feasible online system that can be used to improve future dance injury surveillance research across dance populations.
Level of Evidence
The usability and feasibility portion of the study falls under Level 3 evidence according to the Centre for Evidence-Based Medicine (CEBM) hierarchy. Whereas the Delphi portion of the study is at Level 5 Evidence on the CEBM hierarchy.
Exoskeletons for rehabilitation can help enhance motor recovery in individuals suffering from neurological disorders. Precision in movement execution, especially in arm rehabilitation, is crucial to prevent maladaptive plasticity. However, current exoskeletons, while providing arm support, often lack the necessary 3D feedback capabilities to show how well rehabilitation exercises are being performed. This reduces therapist acceptance and patients’ performance. Augmented Reality technologies offer promising solutions for feedback and gaming systems in rehabilitation. In this work, we leverage HoloLens 2 with its advanced hand-tracking system to develop an application for personalized rehabilitation. Our application generates custom holographic trajectories based on existing databases or therapists’ demonstrations, represented as 3D tunnels. Such trajectories can be superimposed on the real training environment. They serve as a guide to the users and, thanks to colour-coded real-time feedback, indicate their performance. To assess the efficacy of the application in improving kinematic precision, we conducted a feasibility study with 15 healthy subjects. Comparing user tracking capabilities with and without the use of our feedback system in executing 4 different exercises, we observed significant differences, demonstrating that our application leads to improved kinematic performance. 12 clinicians tested our system and positively evaluated its usability (System Usability Scale score of 67.7) and acceptability (4.4 out of 5 in the ’Willingness to Use’ category in the relative Technology Acceptance Model). The results from the tests on healthy participants and the feedback from clinicians encourage further exploration of our framework, to verify its potential in supporting arm rehabilitation for individuals with neurological disorders.
Usability comprises the aspects effectiveness, efficiency, and satisfaction. The correlations between these aspects are not well understood for complex tasks. We present data from an experiment where 87 subjects solved 20 information retrieval tasks concerning programming problems. The correlation between efficiency, as indicated by task completion time, and effectiveness, as indicated by quality of solution, was negligible. Generally, the correlations among the usability aspects depend in a complex way on the application domain, the user's experience, and the use context. Going through three years of CHI Proceedings, we find that 11 out of 19 experimental studies involving complex tasks account for only one or two aspects of usability. When these studies make claims concerning overall usability, they rely on risky assumptions about correlations between usability aspects. Unless domain specific studies suggest otherwise, effectiveness, efficiency, and satisfaction should be considered independent aspect of usability and all be included in usability testing.
Efficiency is an important consideration in the design of industrial usability studies. One way
to reduce the cost of a usability study is to reduce its sample size. Small samples are not
always appropriate, but in this paper I will describe a way to use binomial confidence intervals to determine rapidly if a usability defect rate exceeds a criterion. In these situations, relatively small samples are often adequate to meet the goals of a usability evaluator. I will also discuss the risks of using small samples in these situations.
The Help America Vote Act (HAVA) of 2002 secured funding for improvements to election administration. Improvements include upgrading older voting systems to meet new guidelines. To determine whether the new voting systems are improvements over existing voting systems, information is needed on the usability of the older, traditional systems. This study was designed as a first step in addressing the need for usability data on existing voting systems. Three traditional paper ballots were empirically evaluated to collect baseline data that can later be compared to newer, electronic voting systems. Usability was evaluated using the thee International Organization for Standardization (ISO) metrics suggested by the National Institute of Standards and Technology (NIST): effectiveness, efficiency, and satisfaction. All three ballot types (bubble, arrow, and open response) produced reasonable levels of efficiency. The three ballot types did not produce different levels of effectiveness, but the overall error rate was higher than would be expected. On satisfaction, voters were clearly more satisfied with their experience with the bubble ballot.
The Software Usability Measurement Inventory is a rigorously tested and proven method of measuring software quality from the end user's point of view.SUMI is a consistent method for assessing the quality of use of a software product or prototype, and can assist with the detection of usability flaws before a product is shipped.It is backed by an extensive reference database embedded in an effective analysis and report generation tool.
Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts. This, in turn, means that there are no absolute measures of usability, since, if the usability of an artefact is defined by the context in which that artefact is used, measures of usability must of necessity be defined by that context too. Despite this, there is a need for broad general measures which can be used to compare usability across a range of contexts. In addition, there is a need for "quick and dirty" methods to allow low cost assessments of usability in industrial systems evaluation. This chapter describes the System Usability Scale (SUS) a reliable, low-cost usability scale that can be used for global assessments of systems usability.
Factor analysis of Post Study System Usability Questionnaire (PSSUQ) data from 5 years of usability studies (with a heavy emphasis on speech dictation systems) indicated a 3-factor structure consistent with that initially described 10 years ago: factors for System Usefulness, Information Quality, and Interface Quality. Estimated reliabilities (ranging from .83-.96) were also consistent with earlier estimates. Analyses of variance indicated that variables such as the study, developer, stage of development, type of product, and type of evaluation significantly affected PSSUQ scores. Other variables, such as gender and completeness of responses to the questionnaire, did not. Norms derived from this data correlated strongly with norms derived from the original PSSUQ data. The similarity of psychometric properties between the original and this PSSUQ data, despite the passage of time and differences in the types of systems studied, provide evidence of significant generalizability for the questionnaire, supporting its use by practitioners for measuring participant satisfaction with the usability of tested systems.
Usability comprises the aspects effectiveness, efficiency, and satisfaction. The correlations between these aspects are not well understood for complex tasks. We present data from an experiment where 87 subjects solved 20 information retrieval tasks concerning programming problems. The correlation between efficiency, as indicated by task completion time, and effectiveness, as indicated by quality of solution, was negligible. Generally, the correlations among the usability aspects depend in a complex way on the application domain, the user's experience, and the use context. Going through three years of CHI Proceedings, we find that 11 out of 19 experimental studies involving complex tasks account for only one or two aspects of usability. When these studies make claims concerning overall usability, they rely on risky assumptions about correlations between usability aspects. Unless domain specific studies suggest otherwise, effectiveness, efficiency, and satisfaction should be considered independent aspect of usability and all be included in usability testing.
This paper describes recent research in subjective usability measurement at IBM. The focus of the research was the application of psychometric methods to the development and evaluation of questionnaires that measure user satisfaction with system usability. The primary goals of this paper are to (1) discuss the psychometric characteristics of four IBM questionnaires that measure user satisfaction with computer system usability, and (2) provide the questionnaires, with administration and scoring instructions. Usability practitioners can use these questionnaires with confidence to help them measure users' satisfaction with the usability of computer systems.
Magnitude estimation is a measurement method that is very useful for measuring multi-faceted constructs that do not have a physical analog (e.g., usability) and are produced from multidimensional stimuli (e.g., user interfaces). Traditional usability metrics have shown limitations in usefulness and validity. Usability magnitude estimation is an assessment method whereby participants assign usability ‘values’ to tasks, conditions, or other user interface targets according to ratio-based number assignments. The resultant ratio-scale data is appropriate for parametric statistical analysis. This method has been used successfully in a variety of usability activities at Oracle. It has proven to be efficient, sensitive, and highly effective for comparisons using usability as a differentiator.
The System Usability Scale (SUS) was administered verbally to native English and non-native English speakers for several internally deployed applications. It was found that a significant proportion of non-native English speakers failed to understand the word "cumbersome" in Item 8 of the SUS (that is, "I found the system to be very cumbersome to use.") This finding has implications for reliability and validity when the questionnaire is distributed electronically in multinational usability efforts.
Background
Sexual function can be altered in patients by many neurological disorders affecting the cerebrum, the brain stem, the spinal cord, the spinal roots or the peripheral nerves. It can also be prominently affected in patients with an underlying undiagnosed neurological disease. Traditionally, practising neurologists have not paid much attention to sexual dysfunction in their patients, partly because therapeutic possibilities were scant. Sexual dysfunction, however, is observed in many neurological disorders, and may arise as a primary neurogenic disorder. Although not as obviously limiting as, for instance, paresis and pain, it is most disruptive for patients' lives. With emerging awareness of the primary importance of quality of life as the most important indicator of good patient management, and with the advent of more effective treatment of sexual dysfunction, it is no longer acceptable to ignore this very important dimension of life.In this review we will present the anatomical and physiological basis of the sexual response, the clinical approach to patients with sexual dysfunction, the characteristics of symptoms in neurological disorders, how to take a case history, diagnostic procedures and the treatment possibilities.
The growth of end user computing (EUC) has been a phenomenon in the information management world of the 1980's. Our study, based on data from 104 end users in six large companies, examines the individual, organizational, and system factors affecting the success of EUC, as reflected in end user satisfaction. A key result is that the leadership style of information system managers is positively and significantly related to user satisfaction. Other findings of the study indicate that significant positive relationship exist between user satisfaction and hardware/software accessibility and availability, computer background of users, user attitudes toward end user computing, and system utilization. The results also demonstrate that computer anxiety and user age are negatively related with end user satisfaction. No significant relationships were found between user satisfaction and gender, education, and organizational level. Implications of the findings for practicing information systems managers are discussed.
Current methods to represent system or task usability in a single metric do not include all the ANSI and ISO defined usability aspects: effectiveness, efficiency & satisfaction. We propose a method to simplify all the ANSI and ISO aspects of usability into a single, standardized and summated usability metric (SUM). In four data sets, totaling 1860 task observations, we show that these aspects of usability are correlated and equally weighted and present a quantitative model for usability. Using standardization techniques from Six Sigma, we propose a scalable process for standardizing disparate usability metrics and show how Principal Components Analysis can be used to establish appropriate weighting for a summated model. SUM provides one continuous variable for summative usability evaluations that can be used in regression analysis, hypothesis testing and usability reporting.
This paper explores the emotional and the functional aspects of a new type of wearable notification interface. Three wearable devices with different shape, color and size have been developed: tangible mascot, box-shaped notifier and PDA virtual agent. They have a common function: to assist the user in various situations in everyday life. Likert scale questionnaires have been used to analyze the users' attitudes and preferences towards the interfaces in two basic modes: passive by visual observation and active through use-case scenarios. A separate study was conducted to correlate the visual effect patterns with the notification-message context. The usability of the proposed interfaces was measured by SUS (system usability scale). The relevant factors for appropriate information notification of each wearable interface were obtained and analyzed. The mascot was shown to be a preferred interface suitable for developing a personal relation with the user, while the box-shaped interface was found to be most helpful and suitable for business use.
The use of questionnaire methods for usability assessment. Unpublished manuscript Human centered measures of success in Web design SUMI: The Software Measurement Inventory
- J Kirakowski
- J Kirakowski
- N Claridge
- R Whitehand
Kirakowski, J. (1994). The use of questionnaire methods for usability assessment. Unpublished manuscript. Retreived from http://sumi.ucc.ie/sumipapp.html Kirakowski, J., Claridge, N., & Whitehand, R. (1998). Human centered measures of success in Web design. Conference Proceedings of the 4th Conference on Human Factors and the Web. Retrieved from http://www.research.att.com/conf/hfweb/proceedings/kirakowski/ index.html Kirakowski, J., & Corbett, M. (1993). SUMI: The Software Measurement Inventory. British Journal of Educational Technology, 24, 210–212.
Display technology and ambient illumination influences on visual fatigue at VDT workstations. Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University SUS: A " quick and dirty " usability scale
- A W Bangor
- J Brooke
- B A Thomas
Bangor, A. W. (2000). Display technology and ambient illumination influences on visual fatigue at VDT workstations. Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University, Blacksburg, VA. Brooke, J. (1996). SUS: A " quick and dirty " usability scale. In P. W. Jordan, B. Thomas, B. A.
Measuring usability with the USE Questionnaire Usability Interface: The usability SIG newsletter of the Society for Technical Communications
- A Lund
Lund, A. (2001). Measuring usability with the USE Questionnaire. Usability Interface: The
usability SIG newsletter of the Society for Technical Communications, 8(2). Retrieved from
http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html
Human centered measures of success in Web design
- J Kirakowski
- N Claridge
- R Whitehand
Kirakowski, J., Claridge, N., & Whitehand, R. (1998). Human centered measures of success
in Web design. Conference Proceedings of the 4th Conference on Human Factors and the Web.
Retrieved from http://www.research.att.com/conf/hfweb/proceedings/kirakowski/
index.html
The use of questionnaire methods for usability assessment
- J Kirakowski
Kirakowski, J. (1994). The use of questionnaire methods for usability assessment. Unpublished
manuscript. Retreived from http://sumi.ucc.ie/sumipapp.html
Usability Interface: The usability SIG newsletter of the Society for Technical Communications
- A Lund
Lund, A. (2001). Measuring usability with the USE Questionnaire. Usability Interface: The
usability SIG newsletter of the Society for Technical Communications, 8(2). Retrieved from
http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html
The British national corpus (version 2)
- A Mcgee
Oxford University Computing Service. (2001). The British national corpus (version 2).
Oxford, UK: Author. Available from http://www.natcorp.ox.ac.uk/
Rich, A., & McGee, M. (2004). Expected usability magnitude estimation. In Proceedings of the
Human Factors and Ergonomics Society 48 th Annual Meeting (pp. 912-916). Santa Monica,
CA: Human Factors and Ergonomics Society.
- Lewis J.