ArticlePDF Available

On the dimensionality of the System Usability Scale: A test of alternative measurement models

Authors:

Abstract and Figures

The System Usability Scale (SUS), developed by Brooke (Usability evaluation in industry, Taylor & Francis, London, pp 189-194, 1996), had a great success among usability practitioners since it is a quick and easy to use measure for collecting users' usability evaluation of a system. Recently, Lewis and Sauro (Proceedings of the human computer interaction international conference (HCII 2009), San Diego CA, USA, 2009) have proposed a two-factor structure-Usability (8 items) and Learnability (2 items)-suggesting that practitioners might take advantage of these new factors to extract additional information from SUS data. In order to verify the dimensionality in the SUS' two-component structure, we estimated the parameters and tested with a structural equation model the SUS structure on a sample of 196 university users. Our data indicated that both the unidimensional model and the two-factor model with uncorrelated factors proposed by Lewis and Sauro (Proceedings of the human computer interaction international conference (HCII 2009), San Diego CA, USA, 2009) had a not satisfactory fit to the data. We thus released the hypothesis that Usability and Learnability are independent components of SUS ratings and tested a less restrictive model with correlated factors. This model not only yielded a good fit to the data, but it was also significantly more appropriate to represent the structure of SUS ratings.
Content may be subject to copyright.
LETTER TO THE EDITOR
On the dimensionality of the System Usability Scale:
a test of alternative measurement models
Simone Borsci ÆStefano Federici ÆMarco Lauriola
Received: 30 May 2009 / Revised: 12 June 2009 / Accepted: 15 June 2009 / Published online: 30 June 2009
ÓMarta Olivetti Belardinelli and Springer-Verlag 2009
Abstract The System Usability Scale (SUS), developed by
Brooke (Usability evaluation in industry, Taylor & Francis,
London, pp 189–194, 1996), had a great success among
usability practitioners since it is a quick and easy to use
measure for collecting users’ usability evaluation of a system.
Recently, Lewis and Sauro (Proceedings of the human
computer interaction international conference (HCII 2009),
San Diego CA, USA, 2009) have proposed a two-factor
structure—Usability (8 items) and Learnability (2 items)—
suggesting that practitioners might take advantage of these
new factors to extract additional information from SUS data.
In order to verify the dimensionality in the SUS’ two-com-
ponent structure, we estimated the parameters and tested with
a structural equation model the SUS structure on a sample of
196 university users. Our data indicated that both the unidi-
mensional model and the two-factor model with uncorrelated
factors proposed by Lewis and Sauro (Proceedings of the
human computer interaction international conference (HCII
2009), San Diego CA, USA, 2009) had a not satisfactory fit to
the data. We thus released the hypothesis that Usability and
Learnability are independent components of SUS ratings and
tested a less restrictive model with correlated factors. This
model not only yielded a good fit to the data, but it was also
significantly more appropriate to represent the structure of
SUS ratings.
Keywords Questionnaire Usability evaluation
System Usability Scale
Introduction
The System Usability Scale (SUS) developed in 1986 by Digital
Equipment CorporationÓis a ten-item scale giving a global
assessment of Usability, operatively defined as the subjective
perception of interaction with a system (Brooke 1996). The SUS
items have been developed according to the three usability
criteria defined by the ISO 9241-11: (1) the ability of users to
complete tasks using the system, and the quality of the output of
those tasks (i.e., effectiveness), (2) the level of resource con-
sumed in performing tasks (i.e., efficiency), and (3) the users’
subjective reactions using the system (i.e., satisfaction).
Practitioners have considered the SUS as unidimensional
(Brooke 1996;Kirakowski1994) since the scoring system of
this scale results in a single summated rating of overall
usability. Such scoring procedure is strongly based on the
assumption that a single latent factor loads on all items. So far
this assumption has been tested with inconsistent results.
Whereas Bangor et al. (2008) retrieved a single principal
component of SUS items, Lewis and Sauro (2009) suggested a
two-factor orthogonal structure, which practitioners may use
to score the SUS on independent Usability and Learnability
dimensions. This latter finding is very inconsistent with the
unidimensional SUS scoring system as items loading on
independent factors of Usability and Learnability cannot be
summated according to the classical test theory (Carmines and
Zeller 1992). Furthermore, these factor analyses of the SUS
have been carried out by exploratory techniques, nevertheless
S. Borsci (&)
ECoNA, Interuniversity Centre for Research on Cognitive
Processing in Natural and Artificial Systems,
University of Rome ‘La Sapienza’, Rome, Italy
e-mail: simone.borsci@uniroma1.it; siomone.bo21@alice.it
S. Federici
Department of Human and Educational Sciences,
University of Perugia, Perugia, Italy
M. Lauriola
Department of Psychology of Socialization and Development
Processes, University of Rome ‘La Sapienza’, Rome, Italy
123
Cogn Process (2009) 10:193–197
DOI 10.1007/s10339-009-0268-9
these techniques lack of the necessary formal developments to
test which of the two proposed factor solutions is the best
account of collected data.
Unlike exploratory factor analysis, confirmatory factor
analysis (CFA) is a theory-driven approach who needs a pri-
ori specification of the number of latent variables (i.e., the
factors), of the observed-latent variables correlations (i.e., the
factor loadings) as well as of the correlations among latent
variables (Fabrigar et al. 1999). Once the model’s parameters
have been estimated, the hypothesized model is evaluated
according to its ability to replicate sample’s data. These
features make the CFA approach the state of the art most
accurate methodology to compare alternative factorial
structures and eventually decide which is the best one.
Purpose
In the present study, we aim at comparing three alternative factor
models of the SUS items: the one-factor solution with an overall
usability factor (overall SUS) resulting from Bangor et al. (2008)
(Fig. 1a); the two-factor solution resulting from Lewis and
Sauro (2009) with uncorrelated Usability and Learnability fac-
tors (Fig. 1b) and its less restrictive alternative assuming
Usability and Learnability as correlated factors (Fig. 1c).
Methods
Procedure
One hundred and ninety-six Italian students of University
of Rome ‘‘La Sapienza’’ (28 males, 168 females, age
mean =21) were asked to navigate a website (http://www.
serviziocivile.it) in three consecutive sections (all the stu-
dents declared they never had previous surfing experience
with the website):
1. In the first 20-min pre-experimental training section,
the participants were asked to navigate the website
freely in order to learn features, graphic layouts,
information structures and lays of the interface.
2. Afterwards, in the second no-time-limit-scenario-
based navigation section, the participants were asked
to navigate the website following four scenario targets.
3. Finally, in the third usability evaluation section, the
SUS-Italian version was administered to the partici-
pants (Table 1).
Statistical analyses
All models were estimated by the Maximum Likelihood
Robust Method as the data were not normally distributed
(Mardia’s normalized coefficient =10.72). This method
provided us with the Satorra–Bentler scaled chi-square
statistic (S–Bv
2
), which is an adjusted measure of fit for
non-normal data that is more accurate than the standard
ML statistic (Satorra and Bentler 2001). According to the
inspection of the model’s v
2
, virtually any factor model can
be rejected if the sample size is large enough, therefore
many authors (McDonald and Ho 2002; Widaman and
Thompson 2003) recommended to supplement the evalu-
ation of the model’s fit by some more ‘‘practical’’ indices.
The so-called Comparative Fit Index (Bentler 1990) was
purposefully designed to take sample size into account, as
Fig. 1 SUS models tested: one-factor (a), two uncorrelated factors (b), two correlated factors (c)
194 Cogn Process (2009) 10:193–197
123
it compares the hypothesized model’s v
2
with the null
model’s v
2
. By convention (Hu and Bentler 2004), a CFI
greater than 0.90 indicates an acceptable fit to the data,
with values greater 0.95 being strongly recommended. A
second suggested index is the Root Mean Square Error of
Approximation (Browne and Cudeck 1993). Like the CFI,
the RMSEA is relatively insensitive to sample size, as it
measures the difference between the reproduced covari-
ance matrix and the population covariance matrix. Unlike
the CFI, the RMSEA is a ‘‘badness of fit’’ index as a value
of 0 indicates perfect fit and the greater the RMSEA the
worse the model’s fit. By convention (Hu and Bentler
2004), a RMSEA less than 0.05 corresponds to a ‘‘good’’ fit
and an RMSEA less than 0.08 corresponds to an ‘‘accept-
able’’ fit.
Results
Table 2shows that the S–Bv
2
was statistically significant
for all the models we tested regardless of the number of
factors and of whether the factors were correlated or not
(Bentler 2004). The inspection of the CFI and RMSEA fit
indexes indicated, however, that the less restrictive model
assuming Usability and Learnability as correlated factors
(Fig. 1c) resulted in a good fit (i.e., CFI [0.95 and
RMSEA \0.06), whereas the unidimensional factor model
(Fig. 1a) proposed by Bangor et al. (2008) resulted only in
an acceptable fit (i.e., CFI [0.90 and RMSEA \0.00).
Differently, the two-factor model proposed by Lewis and
Sauro (2009) with uncorrelated factors (Fig. 1b) did not
meet with any of the recommended fit indexes.
Since both the Bangor’s and the Lewis and Sauro’s
factor models are nested within the less restrictive and best
fitting model (i.e., the model with Usability and Learna-
bility as correlated factors) we could formally compare the
fit of each of the model proposed in the literature to the fit
of the model which they were nested in. Nevertheless,
given that we used the Satorra–Bentler scaled v
2
measure
for not multivariate normal data, we could not merely
assess the v
2
difference of two nested models. Rather we
have assessed the scaled S–Bv
2
difference according to the
procedures devised by Satorra and Bentler (2001). The first
contrast, which involved the comparison of the Lewis and
Table 1 Synoptical table of the
English and Italian versions of
the SUS
Original English version Italian version
1. I think I would like to use this system
frequently
1. Penso che mi piacerebbe utilizzare questo
sistema frequentemente
2. I found the system unnecessarily complex. 2. Ho trovato il sistema complesso senza che ce
ne fosse bisogno
3. I thought the system was easy to use 3. Ho trovato il sistema molto semplice da usare
4. I think I would need the support of a
technical person to be able to use this
system
4. Penso che avrei bisogno del supporto di una
persona gia
`in grado di utilizzare il sistema
5. I found the various functions in this system
were well integrated
5. Ho trovato le varie funzionalita
`del sistema
bene integrate
6. I thought there was too much inconsistency
in this system
6. Ho trovato incoerenze tra le varie funzionalita
`
del sistema
7. I would imagine that most people would
learn to use this system very quickly
7. Penso che la maggior parte delle persone
potrebbero imparare ad utilizzare il sistema
facilmente
8. I found the system very cumbersome to use 8. Ho trovato il sistema molto macchinoso da
utilizzare
9. I felt very confident using the system 9. Ho avuto molta confidenza con il sistema
durante l’uso
10. I needed to learn a lot of things before I
could get going with this system
10. Ho avuto bisogno di imparare molti processi
prima di riuscire ad utilizzare al meglio il
sistema
Table 2 Exact and close fit confirmatory factor analysis statistics/indices maximum likelihood estimation for the system usability scale
Model S–Bv
2
(df) CFI RMSEA RMSEA CI
One-factor, overall usability 76.50 (35) 0.921 0.079 0.054–0.103
Two-factor, usability and learnability, uncorrelated 108.58 (35) 0.857 0.105 0.083–0.127
Two-factor, usability and learnability, correlated 54.81 (34) 0.959 0.057 0.026–0.083
All v
2
measures were statistically significant at the 0.001 level
Cogn Process (2009) 10:193–197 195
123
Sauro’s (2009) model (Fig. 1b) to the less restrictive two-
factor model with correlated factors (Fig. 1c), was statis-
tically significant (DS–Bv
2
=30.17; df =1; p\0.001).
Likewise, the second contrast, which involved the com-
parison of the unidimensional model (Bangor et al. 2008)
(Fig. 1a) to the less restrictive two-factor model with cor-
related factors (Fig. 1c), was also statistically significant
(DS–Bv
2
=28.54; df =1; p\0.001). Based on the
inspection of absolute and relative fit indexes as well as on
the results of formal tests of v
2
differences, we may con-
clude that the two-factor model with correlated factors
outperformed both the factor models proposed in the lit-
erature to account for the measurement model of the SUS.
The inspection of model parameters assessed for the best
fitting model (Table 3) indicated that all the SUS items
significantly loaded on the appropriate factor, with factor
loadings ranging from |0.44| to |0.74| for Usability and
greater than 0.70 for Learnability. Accordingly, the factor
reliability assessed by the xcoefficient
1
yielded fairly high
values, such as 0.81 and 0.76, respectively, for Usability
and Learnability factors. The correlation of Usability and
Learnability was positive and significant (r=0.70) thus
showing that the greater the perceived Usability the greater
the perceived Learnability.
Conclusions
Despite the SUS is one of the most used questionnaires to
evaluate usability of systems, recent contributions have
provided inconsistent results regarding the factorial struc-
ture of its items, which in turn has important consequences
in determining the most appropriate scoring system of this
scale for practitioners and researchers. The traditional
unidimensional structure (Brooke 1996; Kirakowski 1994;
Bangor et al. 2008) has been challenged by the more recent
view of Lewis and Sauro (2009), assuming Learnability
and Usability as independent factors. Based on a relatively
large sample of users’ evaluations of an existing website,
we tested which of the two alternative models was the best
for SUS ratings. Our data indicated that both the proposed
models had a not satisfactory fit to the data with the uni-
dimensional model—being too narrow to represent the
contents of all SUS items—and with the two-factor model
with uncorrelated factors—being too restrictive for its
psychometric assumptions. We thus released the hypothe-
sis that Usability and Learnability are independent com-
ponents of SUS ratings and tested a less restrictive model
with correlated factors. This model not only yielded a good
fit to the data, but it was also significantly more appropriate
to represent the structure of SUS ratings. Albeit the liter-
ature reported greater reliability coefficients (e.g., [0.80)
of the Overall SUS scale, the reliability of the two Lear-
nability and Usability factors was in keeping with required
psychometric standards for short scales (Carmines and
Zeller 1992). Thus, we propose that future usability studies
may evaluate systems according to the scoring rule sug-
gested by Lewis and Sauro (2009) which is very consistent
with the bidimensional and best fitting model we have
retrieved in this study. However, since we have found a
relative correlation of Usability factors with Learnability
ones, future studies should clarify under which circum-
stances researchers may expect to obtain Usability scores
dissociated from Learnability (e.g., systems with high
Learnability but low Usability). In the present study, users
evaluated a single system (i.e., the serviziocivile.it website)
and this might have boosted up the association of the two
factors. Alternatively, our sample of users, who is com-
prised of college students, might be considered a sample
with high computer skills compared to the general popu-
lation and this might have also boosted up the factor cor-
relation. Other studies of the SUS should, then, consider
different combinations of systems and users to test the
generality of the correlation of the two factors.
References
Bangor A, Kortum PT, Miller JT (2008) An empirical evaluation of
the system usability scale. Int J Hum Comp Interact 24:574–594
Bentler PM (1990) Comparative fit indexes in structural models.
Psychol Bull 107:238–246
Bentler PM (2004) EQS structural equations modeling software
(Version 6.1) (Computer software). Multivariate Software,
Encino
Table 3 Maximum likelihood standardized solution for the two-
factor model of the system usability scale
Item k
Usability
k
Learnability
Var e
Q1 0.440 0.898
Q2 -0.737 0.676
Q3 0.750 0.662
Q4 0.752 0.660
Q5 0.629 0.777
Q6 -0.578 0.816
Q7 0.670 0.742
Q8 -0.600 0.800
Q9 0.681 0.732
Q10 0.712 0.702
1
x¼Pki
ðÞ
2
Pki
ðÞ
2þPVar ei
ðÞ
where k
i
the standardized factor loadings for
the factor and Var(e
i
) the error variance associated with the individual
indicator variables (both reported in Table 3).
196 Cogn Process (2009) 10:193–197
123
Brooke J (1996) SUS: a ’quick and dirty’ usability scale. In: Jordan
PW, Thomas B, Weerdmeester BA, McClelland IL (eds)
Usability evaluation in industry. Taylor & Francis, London,
pp 189–194
Browne MW, Cudeck R (1993) Alternative ways of assessing model
fit. In: Bollen KA, Long JS (eds) Testing structural equation
models. Sage, Beverly Hills, pp 136–162
Carmines EG, Zeller RA (1992) Reliability and validity assessment.
SAGE, Beverly Hills
Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ (1999)
Evaluating the use of exploratory factor analysis in psycholog-
ical research. Psychol Meth 4:272–299
Hu L, Bentler PM (2004) Cutoff criteria for fit indexes in covariance
structure analysis: conventional criteria versus new alternatives.
Struct Equ Model 6:1–55
Kirakowski J (1994) The use of questionnaire methods for usabil-
ity assessment (unpublished manuscript). http://sumi.ucc.ie/
sumipapp.html
Lewis JR, Sauro J (2009) The factor structure of the system usability
scale. In: Proceedings of the human computer interaction
international conference (HCII 2009), San Diego CA, USA
McDonald RP, Ho MR (2002) Principles and practice in reporting
structural equation analyses. Psychol Meth 7:64–82
Satorra A, Bentler PM (2001) A scaled difference chi-square test
statistic for moment structure analysis. Psychometrika 66:507–
514
Widaman KF, Thompson JS (2003) On specifying the null model for
incremental fit indices in structural equation modeling. Psychol
Meth 8:16–37
Cogn Process (2009) 10:193–197 197
123
... Further secondary outcomes are shown in Table 1, together with the tools, timing, and users involved in the data collection. Among the secondary outcomes, the SCNs over time (until the end of the study) [18,19], quality of life [20,21], patient satisfaction [22,23], therapeutic adherence [24,25] and performance status [26] will be assessed and compared in both groups. Moreover, the incidence and severity of the following vital signs and adverse events will be compared to highlight the effectiveness of the telenursing intervention with tools based on the Common Terminology Criteria for Adverse Events (CTCAE) [27]: body temperature, blood pressure, heart rate, body weight, pain, stomatitis and mucositis, dysgeusia, nausea and vomiting, diarrhoea, loss of appetite, fatigue, headache, dermatologic reactions, cough and dyspnoea, pain, palmar-plantar erythrodysesthesia, bleeding events, photosensitivity, ophthalmological reactions, sleep, mood and voice disorders. ...
... • Data collection will be carried out using the chosen tools, according to the timing described in Table 1, at baseline, daily, after 30, 90 and 180 days, to allow comparison between the intervention arm and control arm. In particular, patients and nurses will monitor the burden and needs related to care through the Supportive Care Needs scale [18,19]; therapeutic adherence through the MARS-5I [24,25]; QoL through the QoL SF-36 [20,21]; satisfaction and usability of the system through the SUS [22,23]; performance status through the Karnofsky performance status [26]; and the AEs described in Section 1.2 through the CTCAE tools [27]. Patients enrolled in the control group will use paper questionnaires, while patients in the intervention group will use electronic tools that nurses will check daily to implement any coaching interventions. ...
... To the best of the authors' knowledge, this is the first RCT to assess the impact of telenursing interventions on the clinical pathway of patients with melanoma and lung cancer on TT. In particular, it could be helpful to assess the effects of a broader nursing contribution on the SCNs [18,19] at one month (primary outcome), the main AEs, therapeutic adherence [24,25], QoL [20,21], system usability and satisfaction [22,23], performance status [26] and PROs at one, three and six months (secondary outcomes). ...
Article
Full-text available
Background: Telenursing comprises a set of tools and interventions enabling nurses to provide remote care. This study aims to assess the impact of telenursing interventions on the supportive care needs of patients with melanoma and lung cancer who are receiving targeted therapies. Methods: This six-month monocentric, double-arm, randomised, controlled trial study protocol will assess the effect of telenursing on the supportive care needs (primary outcome) in 40 patients (20 in each group) after one month. The secondary outcomes will be monitored at baseline, one, three and six months: supportive care needs (at three and six months), therapeutic adherence, quality of life, usability and satisfaction, performance status, patient-reported outcomes and main adverse events. The SPIRIT guidelines will be used for the reporting. Results: The results from this trial will assess the impact of a telenursing intervention on cancer care. Conclusions: This trial could be a starting point for more extensive studies on telenursing interventions to promote nurses’ skills, as well as the quality and safety of care in patients with cancer, highlighting the impact of more outstanding nursing contributions on cancer care. Trial and Protocol Registration: The study protocol was approved by the relevant Italian Ethics Committee Lazio Area 5 (RS1851/23, 2773; 6 September 2023) and was registered on ClinicalTrials.gov (trial registry number NCT06254196).
... It assesses different cognitive domains: attention and concentration, executive functions, memory, language, visuospatial skills, conceptual reasoning, and orientation. Based on the MoCA outcome, we proceeded to use tests to assess specific cognitive domains as follows: Standard Progressive Matrices (SPM) which measures logical reasoning and visuospatial organization (Dubois et al., 2000); the Rey-Osterrieth Complex Figure Test (ROCFT; Lawton and Brody, 1969) to assess visuospatial memory; Frontal Assessment Battery (FAB; Borsci et al., 2009) to examine executive functions such as categorization, cognitive flexibility, and inhibition. Finally, we assessed autonomies in the daily activity functioning with the Activities of Daily Living (ADL) scale and Instrumental ADL (IADL; Maggio et al., 2018). ...
Article
Full-text available
Introduction Mild cognitive impairment (MCI) or “mild neurocognitive disorder” represents an intermediate status between normality and dementia. It is characterized by cognitive decline that does not significantly interfere with normal daily living activities. Virtual reality (VR) is the new frontier of rehabilitation. Methods We enrolled 50 MCI patients who underwent a neuropsychological evaluation and participated in 40 sessions of cognitive treatment using the Virtual Environment for a Superior Neuro-Psychiatry, Second Generation (VESPA 2.0) System. This preliminary study highlights the role of VR tools for cognitive rehabilitation (CR) for the recovery of cognitive functions and consequent better management of MCI condition. Our study demonstrated that the VESPA 2.0 System is a valuable tool in a context that closely resembles real-life situations rather than controlled, artificial environments as traditional cognitive training methods. Results The results showed that the patient group had significant improvements between T0 and T1 (assessment), in particular, in the global cognitive profile, visuospatial skills, and executive functions after treatment with the VESPA 2.0 System. Discussion Our findings contribute with new evidence of understanding the impact of using simulations of the Activities of Daily Living (ADL) scale in the CR.
... A normalised score above 80 indicates excellent usability, suggesting that the system or product is highly user-friendly and intuitive. 20 Compliance data, including the number of uses over the 3-month period and the thoroughness of data completion, were extracted from the WebApp. ...
Article
Full-text available
Objectives To develop and validate a web-based ecological momentary assessment (EMA) tool to enhance symptoms monitoring among patients with Sjögren’s disease (SjD). Methods Consecutive adults with SjD were enrolled in this pilot observational study. Participants used the WebApp over a 3-month period, for the daily collection of individual EULAR Sjögren’s Syndrome Patient Reported Index (ESSPRI) scales and separate assessment of eyes and mouth dryness, using 0–10 numerical scales. Primary outcome was the measure of the interdaily variability of symptoms. Data collected through the WebApp were compared with those obtained with paper-based questionnaires administered during a final visit, using distinct approaches (predicted error, maximum negative error and maximum positive error). User experience was assessed using the System Usability Scale (SUS) score. Results Among the 45 participants, 41 (91.1%) were women. Median age was 57 years (IQR: 49–66). Daily variability of symptoms ranged between 0.5 and 0.8 points across the scales. Over the 3-month period, the predicted error ranged between −1.2 and −0.3 points of the numerical scales. The greatest differences were found for fatigue (−1.2 points (IQR: −2.3 to −0.2)) and ESSPRI score (−1.2 points (IQR: −1.7 to −0.3)). Over the last 2 weeks, the predicted error ranged between – 1.2 and 0.0 points. Maximum negative error ranged between −2.0 and −1.0 points, and maximum positive error between −0.3 and 0.0 points. Median SUS score was 90 (IQR: 85–95). Conclusion Our results demonstrate the usability and the relevance of our web-based EMA tool for capturing data that closely reflects daily experiences of patients with SjD.
... The International Standardization Organization (ISO)'s 9241-11 international standard defines usability as learnability, efficiency, and user satisfaction when specific users use products for specific purposes in specific usage environments. 31 SUS provides a comprehensive reflection of users' subjective evaluation of system usability quickly and easily, 32 with results immediately scored on a single reliable ranging from 0 to 100. Kubra et al. [33][34][35][36] also utilized SUS to evaluate the usability of various systems such as rehabilitation systems and breast cancer systems. ...
Article
Full-text available
Objective Ophthalmic ward nursing work is onerous and busy, and many researchers have tried to introduce artificial intelligence (AI) technology to assist nurses in performing nursing tasks. This study aims to use augmented reality (AR) and AI technology to develop an intelligent assistant system for ophthalmic ward nurses and evaluate the usability and acceptability of the system in assisting clinical work for nurses. Methods Based on AR technology, under the framework of deep learning, the system management, functions, and interfaces were completed using acoustic recognition, voice interaction, and image recognition technologies. Finally, an intelligent assistance system with functions such as patient face recognition, automatic information matching, and nursing work management was developed. Ophthalmic day ward nurses were invited to participate in filling out the System Usability Scale (SUS). Using the AR-based intelligent assistance system (AR-IAS) as the experimental group and the existing personal digital assistant (PDA) system as the control group. The experimental results of the three subscales of learnability, efficiency, and satisfaction of the usability scale were compared, and the clinical usability score of the AR-IAS system was calculated. Results This study showed that the AR-IAS and the PDA systems had learnability subscale scores of 22.50/30.00 and 21.00/30.00, respectively; efficiency subscale scores of 29.67/40.00 and 28.67/40.00, respectively; and satisfaction subscale scores of 23.67/30.00 and 23.17/30.00, respectively. The overall usability score of the AR-IAS system was 75.83/100.00. Conclusion Based on the analysis results of the System Usability Scale, the AR-IAS system developed using AR and AI technology has good overall usability and can be accepted by clinical nurses. It is suitable for use in ophthalmic nursing tasks and has clinical promotion and further research value.
... There were no significant differences between completers compared to non-completers on age, annual income, insurance type, employment, education, or living with a partner. We used evidence-based approaches including Mother-Baby Interaction Therapy [34], developed by one of the investigators (June Andrews Horowitz), to address deficits in the mother-infant relationship, in addition to a combination of Interpersonal Psychotherapy, Cognitive Behavioral Therapy, and Dialectal Behavioral Therapy. This combination of therapeutic modalities addresses significant problematic relationships, dysfunctional thinking patterns, and emotional dysregulation contributing to depressive symptoms and sub-optimal maternal functioning. ...
Article
Full-text available
The PRECEDE-PROCEED model is a comprehensive planning and theoretical framework that incorporates epidemiological, environmental, behavioral, and social factors systematically to design, implement, and evaluate health promotion programs. As such, PRECEDE-PROCEED is a highly effective tool for addressing complex and significant public health concerns like postpartum depression (PPD). PPD negatively impacts mothers and their infants, with studies showing that approximately one in eight mothers experience PPD, leading to adverse effects on maternal functioning and infant development. However, access to specialized evidence-based treatment remains significantly limited due to barriers including social determinants of health. This paper explores the application of the PRECEDE-PROCEED model as a planning and theoretical framework for the design and development of MommaConnect, an innovative digital healthcare platform aimed at reducing PPD symptoms and improving maternal-infant interaction while overcoming barriers to treatment. Key components of the MommaConnect design and development process are mapped onto the steps of the PRECEDE-PROCEED model. MommaConnect features are aligned with specific stages of the model, from assessing, predisposing, enabling, and reinforcing factors to designing, implementing, and evaluating the intervention. By leveraging this model, MommaConnect represents a promising innovative approach to address PPD to improve maternal functioning and infant health in a digitally-enabled era. This paper underscores the importance of utilizing a framework like the PRECEDE-PROCEED model in the design and development of innovative healthcare solutions.
... The test has 10 questions with five ranking scale from "strongly disagree" to "strongly agree" as shown in Table 1. The SUS items have been developed according to the three usability criteria defined by the ISO 9241-11 (Borsci et al. 2009). These criteria are: ...
Article
Personalised e-learning aims at providing a personalisation effect based on the learner’s characteristics such as knowledge level, preferences, and learning style. The support derived from using social collaboration tools like social media reflects the discovery of these characteristics from content generated during collaboration. The collaboration process is guided using different annotations equipped with the collaboration tool. This type of system needs to be evaluated in terms of usability factors including usefulness, ease of use, and System Usability Scale (SUS). These evaluation factors reflect the objectives of the system based on the different functionalities provided. Thus, the correlation between these factors and how they are related to the system objectives is needed to be validated. This validation is performed using Principal Component Analysis (PCA) utilising PerLCol framework as discussed in this paper. PerLCol is a framework that aims at providing personalisation effects by utilising the generated information during social collaboration and interaction. The result reveals the strength items as indicated by the selected components (PC1, PC2, and PC3). These components are related to three evaluated factors which are personalisation, social collaboration, and seamless design which ultimately reflect the objectives of the framework.
... Usability was measured with the system usability scale (SUS) [34] (Italian version [35]), which is composed of 10 items on a 5-point Likert scale where 1 stands for "strongly disagree" and 5 stands for "strongly agree." Items 2, 4, 6, 8, and 10 need to be reversed. ...
Article
Full-text available
Background Considering the growing population of older adults, addressing the influence of loneliness among this demographic group has become imperative, especially due to the link between social isolation and deterioration of mental and physical well-being. Technology has the potential to be used to create innovative solutions to increase socialization and potentially promote healthy aging. Objective This 6-month study examined the usability and acceptability of a technology-based socialization service and explored how stress and living situation affect older adults’ and their ecosystem’s perceptions of technology, investigating cross-sectional and longitudinal differences among and across user groups. Methods Participants were recruited in Tuscany and Apulia (Italy) through a network of social cooperatives and a research hospital, respectively. A total of 20 older adults were provided with the same technology installed on a tablet and on a smart television. The technology has three functionalities: video calling, playing games, and sharing news. Additionally, 20 informal caregivers (IC) and 13 formal caregivers (FC) connected to the older adults were included in the study. After both initial training in the use of the system (T0) and 6 months of using the system (T6), questionnaires on usability, acceptability, and technostress were filled in by older adults, IC, and FC. Nonparametric or parametric tests were conducted to investigate group differences at both time points and changes over time. Additional analyses on older adults were done to assess whether differences in usability and acceptability were related to living situation (ie, alone or with someone). Furthermore, correlation analyses were performed between usability, acceptability, and stress toward technology at T0 and T6. Results At both T0 and T6, older adults had lower usability scores than IC and FC and higher anxiety than IC. Over time, there was a significant decrease in older adults’ attitudes toward technology score, depicting a negative attitude over time (T0 median 4.2, IQR 0.5; T6 median 3.7, IQR 0.8; Cohen d =0.7), while there was no change for IC and FC. At T0, those living alone had lower acceptability than those living with someone but this difference disappeared at T6. People or participants living with someone had a decline in anxiety, attitudes toward technology, enjoyment, and perceived usefulness. Stress toward technology affected usability and acceptability in the older adult group entering the study (ρ=−.85) but this was not observed after 6 months. In the IC group, stress affected trust at T0 (ρ=−.23) but not at T6. Conclusions At the start of the study, older adults judged the system to be less usable and more stressful than did the caregivers. Indeed, at first, technostress was correlated with usability and acceptability; however, with repeated use, technostress did not influence the perception of technology. Overall, getting accustomed to technology decreased anxiety and stress toward technology.
Article
Full-text available
Despite the widespread use of exploratory factor analysis in psychological research, researchers often make questionable decisions when conducting these analyses. This article reviews the major design and analytical decisions that must be made when conducting a factor analysis and notes that each of these decisions has important consequences for the obtained results. Recommendations that have been made in the methodological literature are discussed. Analyses of 3 existing empirical data sets are used to illustrate how questionable decisions in conducting factor analyses can yield problematic results. The article presents a survey of 2 prominent journals that suggests that researchers routinely conduct analyses using such questionable methods. The implications of these practices for psychological research are discussed, and the reasons for current practices are reviewed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts. This, in turn, means that there are no absolute measures of usability, since, if the usability of an artefact is defined by the context in which that artefact is used, measures of usability must of necessity be defined by that context too. Despite this, there is a need for broad general measures which can be used to compare usability across a range of contexts. In addition, there is a need for "quick and dirty" methods to allow low cost assessments of usability in industrial systems evaluation. This chapter describes the System Usability Scale (SUS) a reliable, low-cost usability scale that can be used for global assessments of systems usability.
Conference Paper
Full-text available
Since its introduction in 1986, the 10-item System Usability Scale (SUS) has been assumed to be unidimensional. Factor analysis of two independent SUS data sets reveals that the SUS actually has two factors - Usability (8 items) and Learnability (2 items). These new scales have reasonable reliability (coefficient alpha of .91 and .70, respectively). They correlate highly with the overall SUS ( r = .985 and .784, respectively) and correlate significantly with one another ( r = .664), but at a low enough level to use as separate scales. A sensitivity analysis using data from 19 tests had a significant Test by Scale interaction, providing additional evidence of the differential utility of the new scales. Practitioners can continue to use the current SUS as is, but, at no extra cost, can also take advantage of these new scales to extract additional information from their SUS data.
Article
This article examines the adequacy of the “rules of thumb” conventional cutoff criteria and several new alternatives for various fit indexes used to evaluate model fit in practice. Using a 2‐index presentation strategy, which includes using the maximum likelihood (ML)‐based standardized root mean squared residual (SRMR) and supplementing it with either Tucker‐Lewis Index (TLI), Bollen's (1989) Fit Index (BL89), Relative Noncentrality Index (RNI), Comparative Fit Index (CFI), Gamma Hat, McDonald's Centrality Index (Mc), or root mean squared error of approximation (RMSEA), various combinations of cutoff values from selected ranges of cutoff criteria for the ML‐based SRMR and a given supplemental fit index were used to calculate rejection rates for various types of true‐population and misspecified models; that is, models with misspecified factor covariance(s) and models with misspecified factor loading(s). The results suggest that, for the ML method, a cutoff value close to .95 for TLI, BL89, CFI, RNI, and Gamma Hat; a cutoff value close to .90 for Mc; a cutoff value close to .08 for SRMR; and a cutoff value close to .06 for RMSEA are needed before we can conclude that there is a relatively good fit between the hypothesized model and the observed data. Furthermore, the 2‐index presentation strategy is required to reject reasonable proportions of various types of true‐population and misspecified models. Finally, using the proposed cutoff criteria, the ML‐based TLI, Mc, and RMSEA tend to overreject true‐population models at small sample size and thus are less preferable when sample size is small.