No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Perceived Usability and the Modified Technology Acceptance Model
Abstract
In response to recent criticism of the usefulness of the construct of usability, we investigated the relationships between measures of perceived usability and the components of a modified version of the Technology Acceptance Model (mTAM) – Perceived Usefulness (PU) and Perceived Ease-of-Use (PEU). In three surveys, respondents used SUS, UMUX-LITE and mTAM to rate their actual (as opposed to expected) experience with three software products. As expected, the correlations between PEU and other measures of perceived usability tended to be significantly stronger than those with PU. Additional findings support the use of the UMUX-LITE as a compact measure of perceived usability that has a strong relationship to the mTAM and strong correspondence with concurrently collected SUS scores. The main theoretical result of this research were regression results providing evidence that the PEU component of the mTAM appears to be another measure of the construct of perceived usability, connecting the TAM to the construct of perceived usability through the mTAM and providing evidence against the claim that the construct of usability is a theoretical dead end.
... Overall experience is the global responses or evaluation outcomes of users of a product after using it, which have positive correlations with usability, pragmatic quality, and hedonic quality [22,23]. Similarly, some studies [24,25] suggest that affect and usability can significantly predict overall experience. ...
... Overall experience [14,23,25,26,40] OE1 I feel motivated to continue to use the wearable robot. OE2 ...
... In addition, the survey on the UX of wearable robots showed clear effects of usability, hedonic quality, and attitude on the overall user experience when using WRs. In contrast to the findings of [22,23,25], the largest direct effect was found for hedonic quality rather than usability. Nevertheless, users' perceived usability plays an important role in predicting their overall experience of using WRs. ...
Wearable robots (WRs) might interact with humans in a similar manner to teammates to accomplish specific tasks together. However, the available data on WR user experience (UX) studies are limited, especially during the prototyping phase. Therefore, this study aims to examine the overall experience of WRs during the prototyping phase based on an exploratory research model. This theoretical model considered usability, hedonic quality, and attitude toward using WRs as key factors in explaining and predicting overall experience. To test the hypotheses inherent in the research model, quantitative empirical research was conducted and the data were analyzed by partial least squares structural equation modeling (PLS-SEM). The results from the PLS-SEM analysis revealed the significance level of correlations between the latent variables in the research model. The exploratory research model was able to explain up to 53.2% of the variance in the overall experience of using WRs, indicating medium predictive power. This research develops a new quantitative empirical research model that can be used to explain and predict the overall experience of interactive products such as WRs. Meanwhile, the model is needed during WR testing in the prototype phase.
... In this study, we assessed 45 different UMUX variants based on the combinations of 12 candidate items, which are obtained by translations from versions in English (Finstad, 2010), Italian (Borsci et al., 2015), Slovene (Lah et al., 2020), Arabic (Al-Tahat, 2020) and Chinese (Wang et al., 2022). ...
... Another reason could be the researchers' adhoc research requirements as in Borsci et al. (2015), where the main goal of the study is to explore the effect of amount of experience on usability, rather than making an adaptation of the UMUX scale. Similarly, Lah et al. (2020) translated UMUX for their study on the relationship of usability with technology acceptance. Furthermore, the recent revisions made in ISO 9214-11 (2018) might have affected the researchers' perspective of usability, which could lead to deviations in translations, since the original UMUX items were established in 2010. ...
... There is also further evidence on the unidimensionality of UMUX through factor analysis in these studies. Furthermore, Lah et al. (2020) provided evidence of convergent validity of UMUX in comparison with PEU (perceived ease of use) and PU (perceived usefulness) of TAM (technology acceptance model) (Davis, 1989) and a single item LTR (Likelihood to Recommend) measure through regression analysis. Based on their results, they claim that PEU and SUS are two different measures of the same construct of perceived usability and UMUX-LITE can be used as a short-form alternative to SUS since it had a high correspondence with concurrently collected SUS scores. ...
Within the last decade, the Usability Metric for User Experience is translated into many languages. This study aimed to create an adapted version of UMUX into Turkish, based on item translations made from Arabic, Chinese, English, Italian and Slovene versions, which vary for the content of the items as well as their differences in how they are processed by machine translation systems and understood by human translators. Based on different translations, 45 UMUX variants in Turkish language are assessed psychometrically as a formative construct, regarding the criteria based on PLS measurement models for formative constructs and Rasch analysis. The results are benchmarked with the data collected via UMUX, SUS and CSUQ in Arabic and English or Turkish and English concurrently. Results show that all 45 Turkish variants of UMUX reveal strong psychometric qualities as a formative construct, as well as UMUX and Arabic UMUX. While there is evidence that usability can be measured as a formative construct, differences between item sets suggest that UMUX may not be a complete measure that includes all aspects of usability regarding ISO 9241-11. Our results also suggest that UMUX scores are sensitive to the native language of participants since the mean scores were significantly different between the native and English UMUX versions which were responded concurrently to assess the same software.
... The correlation (or consistency) between usability measurement and TAM needs to be further examined. Lah et al. (2020) and Pal and Vanijja (2020) investigated the relationships between the various measures of perceived usability and TAM components. By modifying TAM to fit the usability assessment after actual usage, Lah et al. (2020) found that the PEU of TAM is another measure of the perceived usability structure; TAM and perceived usability, to this effect, are structurally correlated. ...
... Lah et al. (2020) and Pal and Vanijja (2020) investigated the relationships between the various measures of perceived usability and TAM components. By modifying TAM to fit the usability assessment after actual usage, Lah et al. (2020) found that the PEU of TAM is another measure of the perceived usability structure; TAM and perceived usability, to this effect, are structurally correlated. In an evaluation of online learning platforms, Pal and Vanijja (2020) also found that the PEU structure is similar to SUS in perceived usability. ...
... In most cases, the correspondence among questionnaires was not confirmed in the cross-cultural adaptation. Recent studies focusing on the English version of the questionnaire (e.g., Borsci et al., 2015;Lah et al., 2020;Lewis, 2018bLewis, , 2019aLewis et al., 2015) have confirmed that there is a relationship between different usability standardized questionnaires. However, other researchers have observed differences between the psychological measurement of translated questionnaire versions and English versions in terms of cross-cultural adaptation (Borsci et al., 2015;Gao et al., 2020;Gronier & Baudet, 2021). ...
In this study, we explored usability structures in the Chinese environment by (i) developing a Chinese version of the modified Technology Acceptance Model (mTAM), which uses a strict translation and prediction test process with strict psychometric evaluation characteristics and (ii) examining the relationships among Chinese versions of mTAM, SUS, and UMUX; we find that the three questionnaires have a strong correspondence. Further verification revealed that the correlation between SUS and PEU is stronger than between SUS and PU. Though the three questionnaires were developed independently in the Chinese environment, all can be used to effectively measure perceived usability.
... Al-Tahat (2020) successfully cross-culturally adapted the UMUX for Arabic language speakers. Lah et al. (2020) translated the UMUX into Slovenian; their translated version demonstrated similar reliability and relevance to the English version, although individual translations may require further refinement. These studies have promoted the cross-cultural application of perceived usability scales and give us confidence that the Chinese UMUX can be successfully localized. ...
... The latest study by Lewis (2019) showed that UMUX and SUS have relatively high correlation (0.9) and reliability (0.89), and that the factor structure of the UMUX is similar to that of the SUS. Lah et al. (2020) also investigated the correspondence between SUS, UMUX, and TAM. There appears to be relatively high correlation between the UMUX and other questionnaires. ...
... Many researchers have observed a high correlation between the UMUX-LITE and SUS (Borsci et al., 2015b;Lewis, 2019Lewis, , 2018a. Lah et al. (2020) found that the correlation between the UMUX-LITE and UMUX is high enough to use the two methods interchangeably. There is also correspondence with the PU and PEU in dual dimensions of the TAM questionnaire, which is indicative of an effective measure of perceived usability. ...
A new Usability Metric for User Experience (UMUX) is translated and validated for native Chinese speakers in this study. The forward-backward translation method is applied to translate the UMUX. The results are optimized through structure-back interviews to obtain a final, unambiguous UMUX version. The Chinese version of the UMUX questionnaire is proven to have high reliability, sensitivity, and effectiveness. The correspondence between the Chinese UMUX version, UMUX-LITE, and SUS is also investigated. The results of this work may provide Chinese usability practitioners with a standardized scale after rigorous testing, as well as a closer understanding of the relationship between the three questionnaires in assisting users to make sound choices.
... Al-Tahat (2020) successfully cross-culturally adapted the UMUX for Arabic language speakers. Lah et al. (2020) translated the UMUX into Slovenian; their translated version demonstrated similar reliability and relevance to the English version, although individual translations may require further refinement. These studies have promoted the cross-cultural application of perceived usability scales and give us confidence that the Chinese UMUX can be successfully localized. ...
... The latest study by Lewis (2019) showed that UMUX and SUS have relatively high correlation (0.9) and reliability (0.89), and that the factor structure of the UMUX is similar to that of the SUS. Lah et al. (2020) also investigated the correspondence between SUS, UMUX, and TAM. There appears to be relatively high correlation between the UMUX and other questionnaires. ...
... Many researchers have observed a high correlation between the UMUX-LITE and SUS (Borsci et al., 2015b;Lewis, 2019Lewis, , 2018a. Lah et al. (2020) found that the correlation between the UMUX-LITE and UMUX is high enough to use the two methods interchangeably. There is also correspondence with the PU and PEU in dual dimensions of the TAM questionnaire, which is indicative of an effective measure of perceived usability. ...
A new Usability Metric for User Experience (UMUX) is translated and validated for native Chinese speakers in this study. The forward-backward translation method is applied to translate the UMUX. The results are optimized through structure-back interviews to obtain a final, unambiguous UMUX version. The Chinese version of the UMUX questionnaire is proven to have high reliability, sensitivity, and effectiveness. The correspondence between the Chinese UMUX version, UMUX-LITE, and SUS is also investigated. The results of this work may provide Chinese usability practitioners with a standardized scale after rigorous testing, as well as a closer understanding of the relationship between the three questionnaires in assisting users to make sound choices.
... TAM has not only been used in explaining the adoption intention of a variety of new technologies Pal & Triyason, 2018), but also in the context of new educational technologies (Park, 2009;Scherer et al., 2019). The original TAM model as in (Davis, 1989) is designed to elicit likelihood ratings, rather than the actual ratings because it predicts the future use of a technology, and not the experience obtained after using the technology (Lah et al., 2020). Since the purpose of the current research is to evaluate the actual usage scenario (after interacting with the system/video contents), rather than predicting the likelihood of use, therefore, the TAM questionnaire has been slightly modified in this study to suite the current needs. ...
... Similarly, multimedia contents are watched best on big-screen devices (Maniar et al., 2008;Molnar, 2017). Therefore, the usability varies between the smartphones and laptops depending upon the usage context, and such a variation is expected to reflect the perceived usefulness of such devices also (Kortum & Sorber, 2015;Lah et al., 2020). Perceived usability has a close association with the usefulness construct of TAM (Lah et al., 2020). ...
... Therefore, the usability varies between the smartphones and laptops depending upon the usage context, and such a variation is expected to reflect the perceived usefulness of such devices also (Kortum & Sorber, 2015;Lah et al., 2020). Perceived usability has a close association with the usefulness construct of TAM (Lah et al., 2020). Considering a sizable difference in terms of the screen size between the smartphones vs. laptops along with previous research reporting the interdependence between screen size and the viewing experience (Maniar et al., 2008;Molnar, 2017;Pal & Vanijja, 2017a, 2017bZhou et al., 2014), we feel that it is justified to investigate whether different types of devices moderate the relationship between perceived usefulness and actual use of the system. ...
Physical closure of educational institutes worldwide due to the COVID-19 pandemic has resulted in the emergence of a new era of video-based learning. The current circumstances are unique that have forced students to use digital technologies for their learning purpose. However, the successful usage of such a system relies on the understanding of the adoption factors, for which an integrated model is proposed based on the Technology Acceptance Model and the Task Technology Fit Model. Additionally, this work considers the moderating effects of gender and digital inequality. Data are collected from 232 students all of whom have taken part in a full-semester video-based online learning course in times of the pandemic. A Partial Least Squares method is used for analyzing the data. Results show that video-based learning positively fits into the student's perception and their actual use of the system. The effect of individual characteristics is more than the technology characteristics on the task-technology-fit chain. The integrated model can predict 64.6% of the variance in the final dependent variable: actual usage of video-based learning. Further, while the moderating effect of gender is found to be significant, that for digital inequality is non-significant. Finally, the theoretical and practical implications are discussed.
... As its name implies, the purpose of the model was to predict potential acceptance or rejection of the technology. Much later, the TAM-based questionnaire was adopted for the evaluation of applications, providing an alternative to usability measures such as the System Usability Scale (SUS) and the Usability Metric for User Experience (UMUX); for instance, the recently introduced mTAM (modified Technology Acceptance Model)score, essentially the average of PU and PEU scaled up to the 0-100 range, has been reported to be highly correlated with both SUS (0.70-0.90, depending on the case), and UMUX (0.62-0.90) [62]. ...
... Each item was assessed with a 7-point Likert-type scale from 1 to 7; note that compared to the original form [61], we reversed the scale to make the interpretation simpler; according to prior research, such a change does not affect the magnitude or structure of the measurements [65]. Following the practice of using TAM-based questionnaire for evaluation [62], we also replaced the original scale labels, from "unlikely/likely" (exacting ex ante perspective) to "disagree/agree" (having no such constraint). ...
With increasing awareness of the advantages of game-based learning, there is a growing number of studies showing its application to both computer science education and sustainable development education. In this paper, we describe, with the example of the Eco JSity application, how both of these areas can be combined into a single tool. The presented evaluation results showed that despite incorporating the sustainable development topics into both the theme and mechanics of the educational game, both the usefulness and ease-of-use of the application were still assessed positively by students. We also identified three clusters of students depending on their attitude towards combining education on computer science and sustainable development. We concluded that game-based learning provided a capable means of incorporating sustainable development topics into teaching of unrelated technical skills.
... As a theoretical research context, and expanded technology adoption model has been used in other studies [30]. By integrating three new constructs: personal innovation, environmental awareness, and perceived danger, the findings show that personal innovation, environmental awareness, and perceived usefulness are positively related to user intentions to use ride-sharing services, whereas purpose and perceived usefulness are negatively related to the perceived danger [31][32][33][34]. There are results from another study that shows that the intention to use the services of an application is a function of perceived ease of use, technology anxiety, social influence, perceived ease of use, trustworthiness, facilitation conditions, perceived risk, and resistance to technology [28][29][30][31][32][33][34]. ...
... By integrating three new constructs: personal innovation, environmental awareness, and perceived danger, the findings show that personal innovation, environmental awareness, and perceived usefulness are positively related to user intentions to use ride-sharing services, whereas purpose and perceived usefulness are negatively related to the perceived danger [31][32][33][34]. There are results from another study that shows that the intention to use the services of an application is a function of perceived ease of use, technology anxiety, social influence, perceived ease of use, trustworthiness, facilitation conditions, perceived risk, and resistance to technology [28][29][30][31][32][33][34]. His research study confirms the application of TAM by the inclusion of additional variables to model service adoption [34]. ...
Acceptance of information systems is very obvious in the continuity of its use, especially an organization when implementing information systems requires no short time and no small cost. This research aims to find out what factors influence the acceptance of the use of ubiquitous registration information systems in rural areas. The research model used is a technology acceptance model. The research method used three stages consisting of a literature review, operationalization, and reporting. The results that have been obtained from this study are the attitude towards use has a significant effect on the intention to be used, the perception of ease of use has a significant effect on attitude towards use, and the perception of ease of use has a significant effect on the perception of usefulness. The results of this study are expected to provide input to related parties, especially decision-makers in the development and application of an information system.
... The currently recommended scoring method for the UMUX-LITE is to use linear interpolation, similar to the SUS and UMUX, to convert scores to a 0-to 100-point scale. In the initial research that led to the UMUX-LITE, Lewis et al. (2013) published a regression formula that brought UMUX-LITE scores into closer correspondence with the SUS, but later research has indicated that it is better to compute and use the unadjusted score (Lah, Lewis, & Šumak, 2019;Lewis, 2019). Be aware that the early UMUX-LITE literature does not have conventions for discriminating between unadjusted and regression-adjusted versions, but because regression is a linear transformation, it does not affect measures of reliability or correlationjust correspondence of magnitude. ...
... 1.6] based on 12 independent estimates of the difference), and a similar analysis of corresponding grade point averages had a mean of −0.1 (95% confidence interval = [−0.4, 0.2]) (Lah et al., 2019). Thus, the current best estimate is that the difference between SUS and UMUX-LITE mean scores is less than a half of a point on a scale ranging from 0 to 100, and the absolute difference in grade point averages is about a tenth of a point. ...
Objective
To assess versions of the shorter form variant of Usability Metric for User Experience (UMUX-LITE) questionnaire differing in the number of response options for the items (3, 5, 7, and 11).
Background
The UMUX-LITE is an efficient (two-item) standardized questionnaire that measures perceived usability. A growing body of evidence shows it closely corresponds to one of the most widely used standardized usability questionnaires, the System Usability Scale (SUS), with regard to both correlation and magnitude of concurrently collected means. Although the “standard” version of the UMUX-LITE uses items with seven response options, there is some variance in practice.
Method
Members of a corporate user experience panel ( n = 242) completed surveys rating a recent Web site experience with the SUS and UMUX-LITE, also providing ratings of overall experience and likelihood-to-recommend.
Results
Scale reliabilities were acceptable (coefficient α >.70) with the exception of UMUX-LITE with three response options. All UMUX-LITE correlations with SUS, overall experience, and likelihood-to-recommend were highly significant. For likelihood-to-recommend, there was a significant difference in the magnitude of correlations, with 11 response options higher than three. Although some statistically significant differences were observed in correspondence between SUS and UMUX-LITE scores, these did not seem to translate to practically significant differences.
Conclusion
The number of UMUX-LITE response options does not matter much, especially in practice. Because the version with three response options showed some weakness with regard to reliability and correlation with likelihood-to-recommend, practitioners should avoid it.
Application
Unless there is a strong reason to do otherwise, use the “standard” version with seven response options.
... More importantly, they enable the data to be used for its fundamental purpose-enabling decision making. It has been shown that reduced cognitive load is vital for information processing, and ease-of-use is a crucial factor influencing human interaction with technology (Lah et al., 2020;Castro-Alonso et al., 2021). ...
The potential of big data capabilities to transform and understand global agricultural and biological systems often relies on data from different sources that must be considered together or aggregated to provide insights. The value of data is however not only in its collection and storage, but largely in its re-use. Big data storage repositories are not enough when we consider a world brimming with escalating volumes of data, here we need to consider innovative systems and tools which address data harmonization and standardization and importantly, ones that can bridge the gap between science and end users. In this paper, we will demonstrate how CGIAR (including the Alliance of Bioversity International and CIAT) develops a culture of co-operation and collaboration among custodians of agrobiodiversity data, as well as new directions for big data. CGIAR first launched the Platform for Big Data in Agriculture to enhance the development and maintenance of its data. This helped establish workflows of cross-platform synthesis, annotate and apply the lessons learnt. The Platform then built GARDIAN (Global Agricultural Research Data Innovation and Acceleration Network)—a digital tool that harvests from ∼40 separate open data and publication repositories that 15 CGIAR centres have used for data synthesis. While there have been significant advances in big data management and storage, we also identify the gaps to improve use, and the re-use of data in order to reveal its added value in decision making.
... Perceived Usefulness [102] The degree to which a person believes that use of a particular system would enhance his or her job performance ...
Among the plethora of instruments present in healthcare environments, the hospital bed is undoubtedly one of the most important for patients and caregivers. However, their design usually follows a top-down approach without considering end-users opinions and desires. Exploiting Human-centered design (HCD) permits these users to have a substantial role in the final product outcome. This study aims to empower caregivers to express their opinion about the hospital bed using a qualitative approach. For a holistic vision, we conducted six focus groups and six semi-structured interviews with nurses, nursing students, social-health operators and physiotherapists belonging to many healthcare situations. We then used thematic analysis to extract the themes that participants faced during the procedures, providing a comprehensive guide to designing the future electrical medical bed. These work results could also help overcome many issues that caregivers face during their everyday working life. Moreover, we identified the User Experience features that could represent the essential elements to consider.
... The above concepts are quite close to the concept of usability [30], which in turn is familiar from user interface design and the development of digital services. Usability of digital services refers specifically to the usability of software with dimensions like efficiency, engagement, error tolerance and being easy to learn [31]. ...
Along with the digitalisation of societies and services, the accessibility of digital content has become the focus of attention. However, emphasis has been on technical accessibility, ignoring the large number of people suffering from cognitive challenges that are expected to increase as the population ages. The purpose of this study is to demonstrate that the problem affects not only the elderly but also the young. Utilising multivariate methods and a data set of 14,892 young Finns, the study examines the impact of educational disparities on young people's digital usage and skills. It is observed that the level of education, the form of education and the regularity of the education path are related to differences in digital abilities of young people. Based on the results, the risks of being excluded from digital inclusion accumulate among adolescents for the youngest, but especially for those with a wide range of language, learning or motivational difficulties that manifest as delays in education path. As is known from previous research, such differences are expected to be reinforced in later life, threatening to become rather determinative.
... In accordance with information theory (Cole et al., 2002), visualizations can be helpful for schematizing relationships and aiding pattern recognition, thus optimizing the cognitive load of the user (Sweller et al., 2019). It was shown that reduced cognitive load (Hu et al., 2017;Sweller et al., 2019;Castro-Alonso et al., 2021) is vital for information processing, and ease-of-use is a crucial factor influencing human interaction with technology (Hassenzahl et al., 2010;Venkatesh et al., 2016;Lah et al., 2020). ...
Scholarly knowledge graphs provide researchers with a novel modality of information retrieval, and their wider use in academia is beneficial for the digitalization of published works and the development of scholarly communication. To increase the acceptance of scholarly knowledge graphs, we present a dashboard, which visualizes the research contributions on an educational science topic in the frame of the Open Research Knowledge Graph (ORKG). As dashboards are created at the intersection of computer science, graphic design, and human-technology interaction, we used these three perspectives to develop a multi-relational visualization tool aimed at improving the user experience. According to preliminary results of the user evaluation survey, the dashboard was perceived as more appealing than the baseline ORKG-powered interface. Our findings can be used for the development of scholarly knowledge graph-powered dashboards in different domains, thus facilitating acceptance of these novel instruments by research communities and increasing versatility in scholarly communication.
... Researchers identified the need to measure behavioral users' intentions, perceived usefulness, and perceived ease of use when implementing technology (Ajzen & Fishbein, 1980;Sarver, 1983). Numerous researchers further reinforced the viability of the TAM in accurately identifying users' acceptance and use of technology (Lah et al., 2020). Leaders can integrate technology in a manner that is in line with their business model using the TAM and extending the TAM to further enhance business capabilities. ...
Improperly planned capital investments in new technologies could result in lost productivity for e-procurement and supply chain managers in the e-commerce industries. Grounded in the technology acceptance model, the qualitative multiple case study design was used to explore strategies business managers use to adopt technology innovations to improve product marketing and profitability. Participants were five supply chain business managers and owners from five separate organizations in the southwestern region of the United States. Data were collected via responses submitted through Google Forms and emails, written logs, telephone calls, and recorded audio. Thematic analysis resulted in two key themes: perceived ease of use and perceived usefulness. A key recommendation is implementing strategies that involve agile and lean thinking processes, artificial intelligence, and automation. Implications for positive social change include the potential to improve the customer experience while contributing to a sustainable environment.
... This definition relates directly to user and business requirements as effectiveness refers to success in achieving goals; efficiency refers to not wasting time, and satisfaction referes to a willingness to use the system. This new construct has been proposed because various studies outside the UTAUT literature have reported that factors of this kind have a significant influence on users' acceptance of educational technologies (Holden & Rada, 2011;Juarez Collazo et al., 2014;Lah et al., 2020;Lin, 2013;Tsakonas & Papatheodorou, 2008). ...
The United Theory of Acceptance and Use of Technology (UTAUT) model has been widely used to study new technological systems and has proven to be a robust theoretical framework for predicting system intentional use. Many of the UTAUT studies have focused on educational technologies like learning management systems, mobile learning, instructional devices, online collaboration tools and educational services. This paper reviews previous work done on the UTAUT model and proposes an extended model to study educational technology acceptance by introducing additional constructs like usability, learnability and attitude.
... The motivation that led to the completion of this study is twofold. Firstly, testing how the system was perceived by caregivers, focusing on aspects of perceived security, usability, ease of learning, and privacy protection, delegating job responsibilities to home automation, as essential factors for the acceptance of technologies (Gücin and Berk, 2015;Lah et al., 2020). Secondly, identifying a series of guidelines based on caregivers' opinions and suggestions to inform designers and developers to build cuttingedge systems to improve the quality of operators' work and of the life of people they care for. ...
Millions of people with motor and cognitive disabilities face hardships in daily life due to the limited accessibility and inclusiveness of living spaces which limit their autonomy and independence. The DOMHO project deals with these fundamental issues by leveraging an innovative solution: a smart co-housing apartment. Besides, the project aims at exploiting the well know effects of co-housing on individuals’ health and well-being in combination with ambient assisted living technologies. The present study focused on the interaction of caregivers with the control application of an integrated smart system. Participants performed different tasks, fill out a questionnaire, and were interviewed. Performance and usability of the user interface, trust in technology, privacy, and attitudes towards home automation were explored. A series of guidelines for domotic technology control interfaces design was identified, and a high level of trust in these advanced tools was shown. Caregivers considered smart technologies as a work aid and a means for enhancing autonomy and life quality for users with disabilities.
... So, we use the technology acceptance model theory to analyze the attitudes of using information technology in senior high school students in the social studies learning process. The Technology Acceptance Model (T.A.M.) has become a basic model in understanding individual decisions to use technology [39]. According to Folkinshteyn and Lennon [40], T.A.M. asserts that perceived usefulness (P.U.) and perceived ease of use (P.E.O.U.) are the two main things that ultimately lead to the actual use of information technology. ...
Pedagogic competence is a skill that teachers must have in mastering learning methods in delivering subject matter. This study aims to improve the pedagogical competence of social studies teachers through android-based digital books. We encourage teachers and students to use digital books as easy-to-use learning materials. The digital book provides audio, images, and videos to increase students interest in social studies learning. The feasibility test results show that the digital book social studies are declared fit for use following the technology acceptance model. The evaluation results showed an increase in student social studies learning outcomes for the good category from 12.5% pretest to 47.5% at posttest. The assessment of media experts and learning materials shows that digital books are very suitable for the social studies learning process in senior high school.
... To complement the above ToC model, we drew on the modified technology acceptance model (mTAM) to explain stakeholders' acceptance and use of digital health interventions in the three participating states (24). The classic technology acceptance model (25) proposes that two primary factors influence an individual's intention to use a technology: (i) perceived usefulness or the extent to which the technology will enhance job performance, and (ii) perceived ease of use or the extent to which using the technology will be effortless. A person's motivation to use an emerging technology is believed to be higher if that technology is easy to use. ...
Background: Strengthening health systems to improve access to maternity services remains challenging for Nigeria due partly to weak and irregular in-service training and deficient data management. This paper reports the implementation of digital health tools for video training (VTR) of health workers and digitization of health data at scale, supported by satellite communications (SatCom) technology and existing 3G mobile networks.
Objective: To understand whether, and under what circumstances using digital interventions to extend maternal, newborn and child health (MNCH) services to remote areas of Nigeria improved standards of healthcare delivery.
Methods: From March 2017 to March 2019, VTR and data digitization interventions were delivered in 126 facilities across three states of Nigeria. Data collection combined documents review with 294 semi-structured interviews of stakeholders across four phases (baseline, midline, endline, and 12-months post-project closedown) to assess acceptability and impacts of digital interventions. Data was analyzed using a framework approach, drawing on a modified Technology Acceptance Model to identify factors that shaped technology adoption and use.
Results: Analysis of documents and interview transcripts revealed that a supportive policy environment, and track record of private-public partnerships facilitated adoption of technology. The determinants of technology acceptance among health workers included ease of use, perceived usefulness, and prior familiarity with technology. Perceptions of impact suggested that at the micro (individual) level, repeated engagement with clinical videos increased staff knowledge, motivation and confidence to perform healthcare roles. At meso (organizational) level, better-trained staff felt supported and empowered to provide respectful healthcare and improved management of obstetric complications, triggering increased use of MNCH services. The macro level saw greater use of reliable and accurate data for policymaking.
Conclusions: Simultaneous and sustained implementation of VTR and data digitization at scale enabled through SatCom and 3G mobile networks are feasible approaches for supporting improvements in staff confidence and motivation and reported MNCH practices. By identifying mechanisms of impact of digital interventions on micro, meso, and macro levels of the health system, the study extends the evidence base for effectiveness of digital health and theoretical underpinnings to guide further technology use for improving MNCH services in low resource settings.
Trial Registration: ISRCTN32105372.
The digitalization of education has made learning management systems (LMSs) an indispensable tool for facilitating teaching and learning in higher education institutions (HEIs). Research studies on the factors that influence LMS usage in Sub-Saharan Africa is lacking specifically in considering both technology adoption and usability as theoretical lenses. This study investigates the factors that influence the use of LMSs in HEIs in Ghana. Specifically, what factors influence perceived usefulness (PU) and perceived usability (PUsab) and if PU and PUsab can be used together in modelling the usage of the LMS. This quantitative study used a validated questionnaire to conduct a survey with 300 respondents from three Ghanaian HEIs. The data was analyzed using the Partial Least Squares Structural Equation Modeling regression technique. The findings confirm that human factors (self-efficacy and learning style), social influence and access to technical support influence PUsab and the same factors also influence perceived PU. Both PUsab and PU influences LMS use among students in Ghana. The constructs PU and PUsab were found to be highly related when evaluating the factors influencing LMS usage and could thus not be adequately distinguished to be modelled together. Therefore, the construct PUsab could not replace perceived ease of use (PEU), as previous studies have suggested. While the constructs of PUsab and PEU seem semantically similar, PUsab includes the effectiveness construct, which could explain the relatedness between PUsab and PU. This finding has important implications for understanding how the technology adoption literature intersects with the usability literature.KeywordsLearning Management System Perceived usabilityPerceived ease of usePerceived usefulnessLMSTAMGhana
Quality and time are uncompromisable performance indicators of the physical exhibition production process. Design changes during the installation phase of the process, which arise from poor design representations, cause expensive rework and therefore cost increase. Since non-AEC (architecture/engineering) construction) decision-makers can only partially imagine the outcome through traditional 2D and 3D design representations; we proposed a model incorporating virtual reality technologies to solve design miscommunications and improve stakeholder participation. We adopted a qualitative method and tested the model through case studies with five expert museum professionals. We confirm that VR enhances the design communication among the contributing professionals and, unlike other traditional methods, makes it possible to present multimedia, lighting, sound effects, and other digital elements throughout the exhibition production process. Findings indicate that VR has the potential for significant cost reductions and quality improvement in the construction phase of the exhibitions.
Visualization influences the collaborative design process and design thinking (DT) education outcomes. Accordingly, digital visual collaborative learning (DVCL) technology has great potential to facilitate students’ remote collaborative design experiences. This study investigates the fitness of DVCL for DT learning activities and the factors influencing students’ continued use of the technology with a model integrating technology acceptance model (TAM) and design task-technology fit (DTTF). The research findings demonstrate that the perceived usefulness of the technology is the primary direct factor influencing the continuance intention toward the technology. In addition, the DTTF mediated by perceived usefulness and ease of use significantly affected the intention to continue using the technology. The visual characteristics of the technology influenced the DTTF in the collaborative design environment. This study highlights visualization as a critical feature of collaborative learning in DT education and provides implications from both theoretical and practical perspectives.
The journey of a prosthetic user is characterized by the opportunities and the limitations of a device that should enable activities of daily living (ADL). In particular, experiencing a bionic hand as a functional (and, advantageously, embodied) limb constitutes the premise for promoting the practice in using the device, mitigating the risk of its abandonment. In order to achieve such a result, different aspects need to be considered for making the artificial limb an effective solution to accomplish activities of daily living. According to such a perspective, this review aims at presenting the current issues and at envisioning the upcoming breakthroughs in upper limb prosthetic devices. We first define the sources of input and feedback involved in the system control (at user-level and device-level), alongside the related algorithms used in signal analysis. Moreover, the paper focuses on the user-centered design challenges and strategies that guide the implementation of novel solutions in this area in terms of technology acceptance, embodiment, and, in general, human-machine integration based on co-adaptive processes. We here provide the readers (belonging to the target communities of researchers, designers, developers, clinicians, industrial stakeholders, and end-users) with an overview of the state-of-the-art and the potential innovations in bionic hands features, hopefully promoting interdisciplinary efforts for solving current issues of ULPs. The integration of different perspectives should be the premise to a transdisciplinary intertwining leading to a truly holistic comprehension and improvement of the bionic hands design. Overall, this paper aims to move the boundaries in prosthetic innovation beyond the development of a tool and towards the engineering of human-centered artificial limbs.
It was found that the implementation of quality education is a reaction to a number of changes in circumstances related to changes in context related to the distribution of student profiles, the internalization of higher education and the job market, the emergence of the workforce and students, dissatisfaction from workers and students, pressure due to limited funds, and demands for accountability to institutions. Sebelas April University has implemented Information Systems Internal Quality Assurance Systems to avoid these problems. The purpose of this study was to analyze and determine the Usability factor in Information Systems Internal Quality Assurance Systems at Sebelas April University, which was recently formed. The measurement model used is the Usability Model from Nielsen's. The results obtained from the evaluation of the Internal Quality Assurance System for Information Systems that refer to the Nielsen Model variables (Learnability, Efficiency, Memorability, Errors, and User Satisfaction) are considered adequate by the user, with a significance level used of 0.1 (10%), total 743 respondents and 88 samples, resulting in a continuum line value for each variable, namely the Learnability variable 1730 (enough), the Efficiency variable 1620 (good), the memory variable 1107 (enough), the Error variable 1380 (enough) and the Satisfaction variable 1441 (enough). The results of this evaluation can be considered to be used as a guide in the framework of the Information System development process.
The significant benefits of using uncrewed aircraft systems for industrial inspections are now recognized, the implementation of these commercial drones is increasing exponentially. However, there is a lack of tools to explain and understand the remote pilot's risk perception. The established VMUTES model measures a remote pilot's behavioral intention to use drones for commercial purposes, thus, filling a critical gap in the literature and providing a tool to measure behavioral intent. However, in the established VMUTES model, perceived risk was not explored as the outcome variable. A remote pilot's perception of risk is critical for safe operations. This study modifies the VMUTES model using the original VMUTES dataset to explain factors contributing to a remote pilot's risk perception. The theory of planned behavior, the technology acceptance model, and the original VMUTES model support the foundational theory for this study. Good model fit was achieved with the confirmatory factor analysis rather than the full structural equation model; however, the efficacy of the VMUTES was confirmed. Surprisingly, the results indicated a significant negative relationship between Perceived Usefulness and Perceived Risk suggesting remote pilots' perceptions of usefulness impact risk perception.
Telecare has the potential to tackle emerging problems in public healthcare resulting from the graying of population. This study aims to investigate the personal, social and regulatory factors associated with the telecare acceptance for older adults in Hong Kong. A structured questionnaire survey was administered to 110 seniors aged 60–80 years. The results revealed that older people prefer to use the products that they are familiar with. Seniors of very poor economic status are likely to use telecare service. Family and friends support, self-satisfaction, and social relationship are critical factors which positively influence telecare usage of older adults via influencing their perceived ease of use and perceived usefulness. Regulatory factor of perceived incentives positively associates with the attitude of older people toward telecare service, and in turn facilitates telecare usage. This is the first study which stressed that public regulations, such as incentive interventions, should be emphasized in promoting telecare acceptance.
Information technologies exist to enable us to either do things we have not done before or do familiar things more efficiently. Metaverse (i.e. extended reality: XR) enables novel forms of engrossing telepresence, but it also may make mundate tasks more effortless. Such technologies increasingly facilitate our work, education, healthcare, consumption and entertainment; however, at the same time, metaverse bring a host of challenges. Therefore, we pose the question whether XR technologies, specifically Augmented Reality (AR) and Virtual Reality (VR), either increase or decrease the difficulties of carrying out everyday tasks. In the current study we conducted a 2 (AR: with vs. without) × 2 (VR: with vs. without) between-subject experiment where participants faced a shopping-related task (including navigating, movement, hand-interaction, information processing, information searching, storing, decision making, and simple calculation) to examine a proposed series of hypotheses. The NASA Task Load Index (NASA-TLX) was used to measure subjective workload when using an XR-mediated information system including six sub-dimensions of frustration, performance, effort, physical, mental, and temporal demand . The findings indicate that AR was significantly associated with overall workload, especially mental demand and effort, while VR had no significant effect on any workload sub-dimensions. There was a significant interaction effect between AR and VR on physical demand, effort, and overall workload. The results imply that the resources and cost of operating XR-mediated realities are different and higher than physical reality.
The intricateness and dynamism of any software development project are viewed as prevailing attributes that affect its outcome. However, explicit utilization of recognized software development principles and standards ensures a higher possibility of achieving greater success. This paper presents the results of the efforts in the design and development of Comprehensive Pili (Canarium ovatum Engl.) Information System and Marketplace particularly the user experiences. This project was funded by the Department of Science and Technology (DOST) with the intention of establishing a dynamic web of knowledge for the Pili industry, thus addressing the dearth of accessible Pili information. The development process follows strict adherence to PiliIS and Marketplace Software Development Paradigm - researchers made software development procedures adopting the Agile software development concepts. This paradigm ensures explicit compliance to ISO/IEC 25010 standards. An internal evaluation team was employed to examine the user interface judging its compliance with the recognized usability principles. A diverse group of stakeholders evaluated the system following Jakob Nielsen’s principles for interaction design. Complying to these highly recognized software quality standards and adhering to the researchers’ paradigm, the developed software was marked by stakeholders as “highly usable”, “highly acceptable” and “highly reliable” with average ratings of 94.65%, 93.71%, and 93.53%, respectively. Additionally, Philippine Pili Industry League, Inc. (PhilPili) rated the system as “excellent”. Given the limited study on the development and evaluation of information systems designed to support the marginalized sectors in the country, this study provides significant insights on the software quality principles and competencies as universal standards that boost favorable customer satisfaction and optimum user experience.
Background: Nigeria has one of the worst health and development profile globally. A weak health system, poor infrastructure and varied socio-cultural factors are cited as inhibitors to optimal health system performance and improved maternal and child health status. eHealth has become a major solution to closing these gaps in health care delivery in low- and middle-income countries (LMICs). This paper reports the use of satellite communication (SatCom) technology and existing 3G mobile network for providing video training (VTR) for health workers and improving digitization of healthcare data.
Objective: To evaluate whether the expected project outcomes that were achieved at end-line evaluation of 2019, were sustained 12 months after the project ended.
Methods: From March 2017 to March 2019, digital innovations including VTR and data digitization interventions were delivered in 62 healthcare facilities in Ondo State, southwest Nigeria, most of which lacked access to 3G mobile network. Data collection for the evaluation combined documents review with quantitative data extracted from health facility registers, and 24 most significant change stories to assess the longevity of outcomes and impacts of digital innovation in four domains of healthcare: use of eHealth technology for data management, utilization of health facilities by patients, standard of care, and staff attitude. Stories of most significant changes were audio-recorded, transcribed for analysis and categorised by the above domains to identify the most significant changes 12 months after project closedown.
Results: Findings showed that four project outcomes which were achieved at end-line evaluation were sustained 12 months after project closedown namely: staff motivation and satisfaction; increased staff confidence to perform healthcare roles; improved standard of healthcare delivery; and increased adoption of eHealth innovations beyond health sector. Conversely, an outcome that was reversed following discontinuation of SatCom from health facilities is availability of accurate and reliable data for decision-making.
Conclusion: Digital technology can have lasting impacts on health workers, patients and the health system, through improving data management for decision-making, standard of maternity service delivery, boosting attendance at health facilities and utilization of services. Locally driven investment is essential for ensuring long-term survival of eHealth projects to achieve SDGs in LMICs.
The present study has aimed to understand the customers’ behavioural intention to use electronic customer relationship management (E-CRM) in the banking industry, using the technology acceptance model (TAM). Purposive sampling technique is used to collect data from 300 banks that included customers from Delhi (India), and partial least square-structural equation modelling (PLS-SEM) was employed. The reliability and the validity of the overall model were evaluated using the measurement model, and the hypotheses were tested using the structural model. The results reveal that E-CRM had a positive impact on both perceived usefulness and perceived ease of use, which, in turn, had a positive impact on the attitude towards using E-CRM, which ultimately led to a positive behavioural intention of customers towards using E-CRM. This study reveals that there is a large potential for policymakers to raise awareness and acceptance of E-CRM services among the customers.
The available continuous measurement designs for longitudinal research on perceived usability include immediate and retrospective measurement methods. The present study was conducted to examine the gap between different continuous measurement designs. Thirty users completed a longitudinal study on typesetting software. We used two measurement methods to design three stages of testing. Each stage consists of one immediate measurement and two retrospective measurements. A long-term retrospective test was also conducted and SUS questionnaire measurements were gathered. We observed significant differences in perceived usability between participants with varying quantities of event experiences (i.e. the number of times they used the system). Within short periods of time (3 days), there may be significant differences between retrospective and immediate assessments—especially when users make negative evaluations. In the 3-day period, the user’s perceived usability retrospective evaluation is also based on the most recent experience as opposed to the most unpleasant. The recency effect is lost, however, over a long-term retrospective assessment (1 month later). This study is significant for two main reasons: (i) its unique methodology and (ii) its unprecedented level of consistency between measured results and actual user experiences. We hope that the results presented here provide a workable basis for continuous measurement design in future longitudinal studies. We found that in order to secure accurate measurements, it is necessary to set limits on the measurement time and the number of uses (events).
This study aims to examine the factors affecting the behavioural intentions toward online hotel booking. The study integrates visual presentations and online reviews with the technology acceptance model (TAM). Partial least squares structural equation modelling (PLS-SEM) was used to test the proposed hypotheses in this research. The results report that booking intentions are mainly determined by visual presentations and perceived usefulness. The statistical analysis supports the notion that visual presentations and online reviews have a positive impact on perceived usefulness and perceived ease of use. However, online reviews do not have any significant influence on booking intention directly. The study concludes that online consumers are more likely to book a hotel online if visual presentations and online reviews appear to be useful and easy to use. The findings contribute several implications for researchers and practitioners in the hospitality field.
This study attempts to understand the determinants and consequences of service experience toward retailers’ digital transformation by comparing empirical results from entrepreneurs and consumers. For entrepreneurs, the critical antecedents of service experience include entertainment, tangibility, reliability, and empathy. For customers, the critical antecedents include entertainment, interactivity, tangibility, and empathy. Meanwhile, regarding the consequences of service experience, service experience directly affects both continuous use intention and satisfaction. However, satisfaction mediates the relationship between service experience and continuous use intention for entrepreneurs. In addition, no direct effect is found. Overall, service experience has a meditating effect between stimulus and response variables. Meanwhile, entrepreneurs and customers have different perceptions of the proposed topic.
Social context is a vital component of Human-Computer Interaction (HCI) design for security; however, there is little discussion about how to provide security functions based on the social context. This study examines the fit between security design and social contexts in mobile payment applications by investigating preferences and perceptions of security design based on the task-technology fit model and the technology acceptance model. To approach fit of security design in different social contexts, this study followed the approach proposed by design science research and employed a full factorial design experiment. We developed two interfaces—a customizable interface without feedback information and a customizable interface with feedback information—and asked participants to modify security settings in the interfaces according to social contexts, conduct payment transactions, and report their perceptions of security and usability in four payment scenarios. The observed behaviors in relation to security settings and perceptions revealed the fit for security settings and feedback design in different social contexts. Implications were provided to allow insights for security design in mobile payment transactions according to social contexts.
p>In general, students' math skills in Indonesia are low. An effort must be made to get students to enjoy learning mathematics so that their mathematical abilities improve. The rapid development of smartphone technology provides an opportunity for use in mathematics learning. This study aims to develop an Android-based mathematics learning game for junior high school students on the topic of congruence and similarity. The research method used in this study is the Plomp development method with three stages: preliminary research, the prototyping stage, and the assessment phase. The subjects of this study were 56 seventh-grade students from the State Junior High School (SMPN 1) Wagir Malang. Six students participated in small group trials and 50 students participated in field trials. The results showed that the Android-based learning game on the topic of congruence and similarity had been successfully developed. The game media have been validated by experts with a score of 3.36, which shows that the media created are valid. The practicality tests in the small group trial resulted in a score of 3.50, while the field trial resulted in a score of 3.51. The effectiveness tests carried out in the small group trial resulted in a score of 3.36, while the field trial results had a score of 3.38. These results indicate that the learning games developed are practical and effective for use in mathematics learning.</p
This study aims to provide a synthetic examination of the determinants of the behavioral intention (BI) of older adults to use technology and to investigate the moderating variables that contribute to heterogeneity in prior literature. This study is developed through meta-analysis involving 35 primary studies over 15 years. Results indicate that perceived usefulness, perceived ease of use, and social influence are substantially correlated with the BI of older adults to adopt technology. These determinants perform differently when regional disparity is considered, thereby suggesting a cross-cultural study in the future. Moreover, older adults’ willingness to use familiar technologies, such as smartphones, is strongest, whereas their acceptance of unfamiliar technologies, such as healthcare systems and devices, is likely affected by the three determinants. The synthesized results may benefit researchers in determining future research directions and decision makers in gerontechnology research and development and marketing.
In 2018, Noam Tractinsky published a provocative paper entitled, “The usability construct: A dead end?” He argued the following:
• Usability is an umbrella concept.
• There is a mismatch between the construct of usability and its empirical measurements.
• Scientific progress requires unbundling the usability construct and replacing it with well-defined constructs.
Tractinsky (2018) offered the Hirsch and Levin (1999) definition of an umbrella construct as a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (p. 200), noting that this diversity of phenomena makes it impossible to achieve a goal of unidimensional measurement.
Tractinsky’s (2018) paper is undoubtedly a valuable contribution to the literature of usability science. Although I do not agree with its premises or conclusions, I admire its construction and I learned a lot from reading it. I hope that it will lead to additional research that will improve the understanding of the construct of usability, as it has inspired the writing of this essay. Following are the reasons why I disagree with his arguments.
First, I question whether usability is truly an umbrella construct—a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (Hirsch & Levin, 1999, p. 200), at least in the context of industrial usability testing. Structural analysis of objective and subjective data from industrial usability studies (Sauro & Lewis, 2009) has provided evidence consistent with an underlying construct of usability that can manifest itself through objective and subjective measurement. It seems plausible that when a system intended for human use has been properly designed, then the users of that system will complete tasks successfully and quickly, and will be sufficiently aware of this to experience, at a minimum, satisfaction as a consequence of perceived usability.
With regard to perceived usability, it now appears that reports of meaningful factor structure in the SUS may have been premature (Lewis & Sauro, 2009), with more recent analysis indicating a nuisance structure due to the mixed positive and negative tone of its items (Lewis & Sauro, 2017). Furthermore, the development of subscales developed using factor analysis does not preclude the calculation of an overall measure of perceived usability. Investigation of correlation and correspondence of three independently developed usability questionnaires (CSUQ, SUS, and UMUX) has provided compelling evidence that they are measuring the same underlying construct (Lewis 2018a, 2018c, 2018d). There is also evidence that this is the same underlying construct assessed by the PEOU component of the TAM (Lewis, 2018b).
So, rather than being a dead end, I believe the construct of usability has a bright future both in usability science (theory) and usability engineering (practice), either alone or as a fundamental part of the larger assessment of user experience. Any report of its death is an exaggeration.
In 2018, Noam Tractinsky published a provocative paper entitled, “The usability construct: A dead end?” He argued the following:
• Usability is an umbrella concept.
• There is a mismatch between the construct of usability and its empirical measurements.
• Scientific progress requires unbundling the usability construct and replacing it with well-defined constructs.
Tractinsky (2018) offered the Hirsch and Levin (1999) definition of an umbrella construct as a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (p. 200), noting that this diversity of phenomena makes it impossible to achieve a goal of unidimensional measurement.
Tractinsky’s (2018) paper is undoubtedly a valuable contribution to the literature of usability science. Although I do not agree with its premises or conclusions, I admire its construction and I learned a lot from reading it. I hope that it will lead to additional research that will improve the understanding of the construct of usability, as it has inspired the writing of this essay. Following are the reasons why I disagree with his arguments.
First, I question whether usability is truly an umbrella construct—a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (Hirsch & Levin, 1999, p. 200), at least in the context of industrial usability testing. Structural analysis of objective and subjective data from industrial usability studies (Sauro & Lewis, 2009) has provided evidence consistent with an underlying construct of usability that can manifest itself through objective and subjective measurement. It seems plausible that when a system intended for human use has been properly designed, then the users of that system will complete tasks successfully and quickly, and will be sufficiently aware of this to experience, at a minimum, satisfaction as a consequence of perceived usability.
With regard to perceived usability, it now appears that reports of meaningful factor structure in the SUS may have been premature (Lewis & Sauro, 2009), with more recent analysis indicating a nuisance structure due to the mixed positive and negative tone of its items (Lewis & Sauro, 2017). Furthermore, the development of subscales developed using factor analysis does not preclude the calculation of an overall measure of perceived usability. Investigation of correlation and correspondence of three independently developed usability questionnaires (CSUQ, SUS, and UMUX) has provided compelling evidence that they are measuring the same underlying construct (Lewis 2018a, 2018c, 2018d). There is also evidence that this is the same underlying construct assessed by the PEOU component of the TAM (Lewis, 2018b).
So, rather than being a dead end, I believe the construct of usability has a bright future both in usability science (theory) and usability engineering (practice), either alone or as a fundamental part of the larger assessment of user experience. Any report of its death is an exaggeration.
Usability Metric for User Experience (UMUX) and its shorter form variant UMUX-LITE are recent additions to standardized usability questionnaires. UMUX aims to measure perceived usability by employing fewer items that are in closer conformance with the ISO 9241 definition of usability, while UMUX-LITE conforms to the technology acceptance model (TAM). UMUX has been criticized regarding its reliability, validity, and sensitivity, but these criticisms are mostly based on reported findings associated with the data collected by the developer of the questionnaire.
Our study re-evaluates the UMUX and UMUX-LITE scales using psychometric methods with data sets acquired through two usability evaluation studies: an online word processor evaluation survey (n = 405) and a web-based mind map software evaluation survey for three applications (n = 151).
Data sets yielded similar results for indicators of reliability. Both UMUX and UMUX-LITE items were sensitive to the software when the scores for the evaluated software were not very close, but we could not detect a significant difference between the software when the scores were closer.
UMUX and UMUX-LITE items were also sensitive to users’ level of experience with the software evaluated in this study. Neither of the scales was found to be sensitive to the participants’ age, gender, or whether they were native English speakers. The scales significantly correlated with the System Usability Scale (SUS) and the Computer System Usability Questionnaire (CSUQ), indicating their concurrent validity. The parallel analysis of principal components of UMUX pointed out a single latent variable, which was confirmed through a factor analysis, that showed the data fits better to a single-dimension factor structure.
Nowadays, practitioners extensively apply quick and reliable scales of user satisfaction as part of their user experience (UX) analyses to obtain well-founded measures of user satisfaction within time and budget constraints. However, in the human-computer interaction (HCI) literature the relationship between the outcomes of standardized satisfaction scales and the amount of product usage has been only marginally explored. The few studies that have investigated this relationship have typically shown that users who have interacted more with a product have higher satisfaction. The purpose of this paper was to systematically analyze the variation in outcomes of three standardized user satisfaction scales (SUS, UMUX and UMUX-LITE) when completed by users who had spent different amounts of time with a website. In two studies, the amount of interaction was manipulated to assess its effect on user satisfaction. Measurements of the three scales were strongly correlated and their outcomes were significantly affected by the amount of interaction time. Notably, the SUS acted as a unidimensional scale when administered to people who had less product experience, but was bidimensional when administered to users with more experience. We replicated previous findings of similar magnitudes for the SUS and UMUX-LITE (after adjustment), but did not observe the previously reported similarities of magnitude for the SUS and the UMUX. Our results strongly encourage further research to analyze the relationships of the three scales with levels of product exposure. We also provide recommendations for practitioners and researchers in the use of the questionnaires.
Over the recent years, the notion of a non-instrumental, hedonic quality of interactive products received growing interest. Based on a review of 151 publications, we summarize more than ten years research on the hedonic to provide an overview of definitions, assessment tools, antecedents, consequences, and correlates. We highlight a number of contributions, such as introducing experiential value to the practice of technology design and a better prediction of overall quality judgments and product acceptance. In addition, we suggest a number of areas for future research, such as providing richer, more nuanced models and tools for quantitative and qualitative analysis, more research on the consequences of using hedonic products and a better understanding of when the hedonic plays a role and when not.
Over the recent years, the notion of a non-instrumental, hedonic quality of interactive products received growing interest. Based on a review of 151 publications, we summarize more than ten years research on the hedonic to provide an overview of definitions, assessment tools, antecedents, consequences, and correlates. We highlight a number of contributions, such as introducing experiential value to the practice of technology design and a better prediction of overall quality judgments and product acceptance. In addition, we suggest a number of areas for future research, such as providing richer, more nuanced models and tools for quantitative and qualitative analysis, more research on the consequences of using hedonic products and a better understanding of when the hedonic plays a role and when not.
In this paper we present the UMUX-LITE, a two-item questionnaire based on the Usability Metric for User Experience (UMUX) [6]. The UMUX-LITE items are This system's capabilities meet my requirements and This system is easy to use." Data from two independent surveys demonstrated adequate psychometric quality of the questionnaire. Estimates of reliability were .82 and .83 -- excellent for a two-item instrument. Concurrent validity was also high, with significant correlation with the SUS (.81, .81) and with likelihood-to-recommend (LTR) scores (.74, .73). The scores were sensitive to respondents' frequency-of-use. UMUX-LITE score means were slightly lower than those for the SUS, but easily adjusted using linear regression to match the SUS scores. Due to its parsimony (two items), reliability, validity, structural basis (usefulness and usability) and, after applying the corrective regression formula, its correspondence to SUS scores, the UMUX-LITE appears to be a promising alternative to the SUS when it is not desirable to use a 10-item instrument.
This study is a part of a research effort to develop the Questionnaire for User Interface Satisfaction (QUIS). Participants, 150 PC user group members, rated familiar software products. Two pairs of software categories were compared: 1) software that was liked and disliked, and 2) a standard command line system (CLS) and a menu driven application (MDA). The reliability of the questionnaire was high, Cronbach's alpha=.94. The overall reaction ratings yielded significantly higher ratings for liked software and MDA over disliked software and a CLS, respectively. Frequent and sophisticated PC users rated MDA more satisfying, powerful and flexible than CLS. Future applications of the QUIS on computers are discussed.
Usability evaluators used an 18-item, post-study questionnaire in three related usability tests. I conducted an exploratory factor analysis to investigate statistical justification to combine items into subscales. The factor analysis indicated that three factors accounted for 87 percent of the total variance. Coefficient alpha analyses showed that the reliability of the overall summative scale was .97, and ranged from .91 to .96 for the three subscales. In the sensitivity analyses, the overall scale and all three subscales detected significant differences among the user groups; and one subscale indicated a significant system effect. Correlation analyses support the validity of the scales. The overall scale correlated highly with the sum of the After-Scenario Questionnaire ratings that participants gave after each scenario. The overall scale also correlated moderately with the percentage of successful scenario completion. These results are consistent with the hypothesis that these alternative measurements tap into a common underlying construct. This construct is probably usability, based on the content of the questionnaire items and the measurement context.
Computer systems cannot improve organizational performance if they aren't used. Unfortunately, resistance to end-user systems by managers and professionals is a widespread problem. To better predict, explain, and increase user acceptance, we need to better understand why people accept or reject computers. This research addresses the ability to predict peoples' computer acceptance from a measure of their intentions, and the ability to explain their intentions in terms of their attitudes, subjective norms, perceived usefulness, perceived ease of use, and related variables. In a longitudinal study of 107 users, intentions to use a specific system, measured after a one-hour introduction to the system, were correlated 0.35 with system use 14 weeks later. The intention-usage correlation was 0.63 at the end of this time period. Perceived usefulness strongly influenced peoples' intentions, explaining more than half of the variance in intentions at the end of 14 weeks. Perceived ease of use had a small but significant effect on intentions as well, although this effect subsided over time. Attitudes only partially mediated the effects of these beliefs on intentions. Subjective norms had no effect on intentions. These results suggest the possibility of simple but powerful models of the determinants of user acceptance, with practical value for evaluating systems and guiding managerial interventions aimed at reducing the problem of underutilized computer technology.
Correlations between prototypical usability metrics from 90 distinct usability tests were strong when measured at the task-level (r between .44 and .60). Using test-level satisfaction ratings instead of task-level ratings attenuated the correlations (r between .16 and .24). The method of aggregating data from a usability test had a significant effect on the magnitude of the resulting correlations. The results of principal components and factor analyses on the prototypical usability metrics provided evidence for an underlying construct of general usability with objective and subjective factors. Author Keywords
When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding).
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.
The Usability Metric for User Experience (UMUX) is a four-item Likert scale used for the subjective assessment of an application’s
perceived usability. It is designed to provide results similar to those obtained with the 10-item System Usability Scale,
and is organized around the ISO 9241–11 definition of usability. A pilot version was assembled from candidate items, which
was then tested alongside the System Usability Scale during usability testing. It was shown that the two scales correlate
well, are reliable, and both align on one underlying usability factor. In addition, the Usability Metric for User Experience
is compact enough to serve as a usability module in a broader user experience metric.
Valid measurement scales for predicting user acceptance of computers are in short supply. Most subjective measures used in practice are unvalidated, and their relationship to system usage is unknown. The present research develops and validates new scales for two specific variables, perceived usefulness and perceived ease of use, which are hypothesized to be fundamental determinants of user acceptance. Definitions for these two variables were used to develop scale items that were pretested for content validity and then tested for reliability and construct validity in two studies involving a total of 152 users and four application programs. The measures were refined and streamlined, resulting in two six-item scales with reliabilities of .98 for usefulness and .94 for ease of use. The scales exhibited high convergent, discriminant, and factorial validity. Perceived usefulness was significantly correlated with both self-reported current usage (r=.63, Study 1) and self-predicted future usage (r =.85, Study 2). Perceived ease of use was also significantly correlated with current usage (r=.45, Study 1) and future usage (r=.59, Study 2). In both studies, usefulness had a significantly greater correlation with usage behavior than did ease of use. Regression analyses suggest that perceived ease of use may actually be a causal antecedent to perceived usefulness, as opposed to a parallel, direct determinant of system usage. Implications are drawn for future research on user acceptance.
The purpose of this research was to investigate the effect of manipulating item formats for a revised version of the Technology Acceptance Model (TAM) questionnaire. The TAM has 12 items, six assessing perceived usefulness (PU) and six assessing perceived ease of use (PEU). Its original purpose was to quantify likelihood of technology acceptance, using an item format with end anchors of "Likely" on the left and "Unlikely" on the right, and seven response options (from left to right) of "Extremely, Quite, Slightly, Neither, Slightly, Quite, Extremely." To revise the TAM to measure user experience (UX), it is necessary to change the items from assessments of likelihood to agreement. In this study, 546 participants rated their experience using IBM Notes with one of four versions of the modified TAM created by crossing two independent variables: Response format (labels or numbers) and Response order (increasing from right-to-left or from left-to-right), with participants about evenly divided among the four formats. A check on ratings of overall experience showed no significant difference as a function of format group, with similar nonsignificant results for the overall TAM scores. An analysis of variance examining the main effects and interaction of the two independent variables (response format and order) on TAM scores was also nonsignificant at similar levels. Factor analyses for each version produced the same alignment of items with the PU and PEU factors, consistent with the item alignment in the original TAM research. Beta weights for regression models predicting likelihood-of-use and overall experience from PU and PEU ratings were very similar for all four versions. The results indicate that the item format differences did not lead to any important differences in the magnitude or structure of TAM measurement, but there were significantly more response errors when the magnitude of agreement increased from right to left.
A recent contribution to the ongoing debate concerning the concept of usability and its measures proposed that usability reached a dead end – i.e. a construct unable to provide stable results and to unify scientific knowledge. Extensive commentaries rejected the conclusion that researchers need to look for alternative constructs to measure the quality of interaction. Nevertheless, several practitioners involved in this international debate asked for a constructive way to move forward the usability practice. In fact, two key issues of the usability field were identified in this debate: (i) knowledge fragmentation in the scientific community, and (ii) the unstable relationship among the usability metrics. We recognise both the importance and impact of these key issues, although, in line with others, we may not agree with the conclusion that the usability is a dead end. Under the light of the international debate, this work discusses the strengths and weaknesses of usability construct and its application. Our discussion focuses on identifying alternative explanations to the issues and to suggest mitigation strategies, which may be considered the starting point to move forward the usability field. However, scientific community actions will be needed to implement these mitigation strategies and to harmonise the usability practice.
This research continued previous investigation of the relationships among measures of perceived usability: the System Usability Scale (SUS), three metrics derived from the Usability Metric for User Experience (UMUX), and the Computer System Usability Questionnaire (CSUQ), this time with ratings of four everyday products (Excel, Word, Amazon, and Gmail). SUS ratings of these products were generally consistent with previous reports. Significant differences in SUS means across studies could be due to differences in frequency of use, with implications for using these data as usability benchmarks. Correspondence among the various measures of perceived usability was also consistent with previous research. Considering frequency of use, mean differences ranged from -2.0 to 1.8 (average shift in Sauro-Lewis grade range from -0.6 to 0.8). When SUS scores were above average, the range restriction of the UMUX-LITEr led to relatively large discrepancies with SUS, suggesting it might not always be better than unadjusted UMUXLITE.
The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. This review of the SUS covers its early history from inception in the 1980s through recent research and its future prospects. From relatively inauspicious beginnings, when its originator described it as a “quick and dirty usability scale,” it has proven to be quick but not “dirty.” It is likely that the SUS will continue to be a popular measurement of perceived usability for the foreseeable future. When researchers and practitioners need a measure of perceived usability, they should strongly consider using the SUS.
The primary purpose of this research was to investigate the relationship between two widely used questionnaires designed to measure perceived usability: the Computer System Usability Questionnaire (CSUQ) and the System Usability Scale (SUS). The correlation between concurrently collected CSUQ and SUS scores was 0.76 (over 50% shared variance). After converting CSUQ scores to a 0–100-point scale (to match the range of the SUS scores), there was a small but statistically significant difference between CSUQ and SUS means. Although this difference (just under 2 scale points out of a possible 100) was statistically significant, it did not appear to be practically significant. Although usability practitioners should be cautious pending additional independent replication, it appears that CSUQ scores, after conversion to a 0–100-point scale, can be interpreted with the Sauro–Lewis curved grading scale. As a secondary research goal, investigation of variations of the Usability Metric for User Experience (UMUX) replicated previous findings that the regression-adjusted version of the UMUX-LITE (UMUX-LITEr) had the closest correspondence with concurrently collected SUS scores. Thus, even though these three standardized questionnaires were independently developed and have different item content and formats, they largely appear to be measuring the same thing, presumably, perceived usability.
“Usability” is a construct conceived by the human–computer interaction (HCI) community to denote a desired quality of interactive systems and products. Despite its prominence and intensive use in HCI research, the usefulness of the usability construct to HCI theories and to our understanding of HCI has been meager. In this article I propose and discuss two reasons for this state of affairs. The first is that usability is an umbrella construct. Umbrella constructs are prevalent in scientific fields that are broad, diverse, and lack a unifying research paradigm. Accordingly, umbrella constructs, such as usability, tend to be vague and loose, characteristics that challenge our ability to accumulate and communicate knowledge and to capture real-world phenomena. The second reason involves the nature of the relations between the usability construct and its measures, a topic rarely discussed in HCI research. There appears to be a mismatch between how the HCI community has (implicitly) conceptualized these relations and how it has empirically examined them. The relations have been conceptualized according to a formative measurement model but have mostly been tested according to a reflective measurement model. The trouble is that representing the usability construct by the reflective model appears inappropriate, and representing it by the formative model involves considerable difficulties. Possible ways of addressing these issues are discussed, each with its advantages and drawbacks. I conclude that for scientific research on this subject to progress, the usability construct ought to be unbundled and replaced by well-defined constructs. The issues discussed in this article are relevant to other HCI umbrella concepts and constructs such as user experience.
You're being asked to quantify your usability improvements with statistics. But even with a background in statistics, you are hesitant to statistically analyze their data, as they are often unsure which statistical tests to use and have trouble defending the use of small test sample sizes. The book is about providing a practical guide on how to solve common quantitative problems arising in usability testing with statistics. It addresses common questions you face every day such as: Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on the 1st attempt? How long will it take users to purchase products on the website? This book shows you which test to use, and how provide a foundation for both the statistical theory and best practices in applying them. The authors draw on decades of statistical literature from Human Factors, Industrial Engineering and Psychology, as well as their own published research to provide the best solutions. They provide both concrete solutions (excel formula, links to their own web-calculators) along with an engaging discussion about the statistical reasons for why the tests work, and how to effectively communicate the results. *Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices *Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data *Recommends ways for practitioners to communicate results to stakeholders in plain English. © 2012 Jeff Sauro and James R. Lewis Published by Elsevier Inc. All rights reserved.
The purpose of this research was to investigate various measurements of perceived usability, in particular, to assess (a) whether a regression formula developed previously to bring Usability Metric for User Experience LITE (UMUX-LITE) scores into correspondence with System Usability Scale (SUS) scores would continue to do so accurately with an independent set of data; (b) whether additional items covering concepts such as findability, reliability, responsiveness, perceived use by others, effectiveness, and visual appeal would be redundant with the construct of perceived usability or would align with other potential constructs; and (c) the dimensionality of the SUS as a function of self-reported frequency of use and expertise. Given the broad use of and emerging interpretative norms for the SUS, it was encouraging that the regression equation for the UMUX-LITE worked well with this independent set of data, although there is still a need to investigate its efficacy with a broader set of products and methods. Results from a series of principal components analyses indicated that most of the additional concepts, such as findability, familiarity, efficiency, control, and visual appeal covered the same statistical ground as the other more standard metrics for perceived usability. Two of the other items (Reliable and Responsive) made up a reliable construct named System Quality. None of the structural analyses of the SUS as a function of frequency of use or self-reported expertise produced the expected components, indicating the need for additional research in this area and a need to be cautious when using the Usable and Learnable components described in previous research.
The System Usability Scale (SUS) is a widely adopted and studied questionnaire for usability evaluation. It is technology independent and has been used to evaluate the perceived usability of a broad range of products, including hardware, software, and websites. In this article we present a Slovene translation of the SUS (the SUS-SI) along with the procedure used in its translation and psychometric evaluation. The results indicated that the SUS-SI has properties similar to the English version. Slovene usability practitioners should be able to use the SUS-SI with confidence when conducting user research.
The Usability Metric for User Experience (UMUX) is a four-item Likert scale aimed at replicating the psychometric properties of the System Usability Scale (SUS) in a more compact form. As part of a special issue of the journal Interacting with Computers, the UMUX is being examined in terms of purpose, reliability, validity and structure. This response to commentaries addresses concerns with these issues through updated archival research, deeper analysis on the original data and some updated results with an average-scoring system. The new results show the UMUX performs as expected for a wide range of systems and consists of one underlying usability factor.
The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including Web sites, cell phones, interactive voice response systems, TV applications, and more. It provides an easy-to-understand score from 0 (negative) to 100 (positive). While a 100-point scale is intuitive in many respects and allows for relative judgments, information describing how the numeric score translates into an absolute judgment of usability is not known. To help answer that question, a seven-point adjective-anchored Likert scale was added as an eleventh question to nearly 1,000 SUS surveys. Results show that the Likert scale scores correlate extremely well with the SUS scores (r=0.822). The addition of the adjective rating scale to the SUS may help practitioners interpret individual SUS scores and aid in explaining the results to non-human factors professionals.
This paper proposed a revised technology acceptance model for measuring end user computing (EUC) acceptance. An empirical study was conducted to collect data. This data was empirically used to test the proposed research model. The structural equation modeling technique was used to evaluate the causal model and confirmatory factor analysis was performed to examine the reliability and validity of the measurement model. The results demonstrate that the model explains 56% of the variance. This finding contributes to an expanded understanding of the factors that promote EUC acceptance. The implication of this work to both researchers and practitioners is discussed.
This article presents nearly 10 year's worth of System Usability Scale (SUS) data collected on numerous products in all phases of the development lifecycle. The SUS, developed by Brooke (1996)2.
Brooke , J. 1996. “SUS: A “quick and dirty” usability scale”. In Usability evaluation in industry, Edited by:
Jordan , P. W. ,
Thomas , B. A. Weerdmeester and
McClelland , I. L. 189–194. London: Taylor & Francis. View all references, reflected a strong need in the usability community for a tool that could quickly and easily collect a user's subjective rating of a product's usability. The data in this study indicate that the SUS fulfills that need. Results from the analysis of this large number of SUS scores show that the SUS is a highly robust and versatile tool for usability professionals. The article presents these results and discusses their implications, describes nontraditional uses of the SUS, explains a proposed modification to the SUS to provide an adjective rating that correlates with a given score, and provides details of what constitutes an acceptable SUS score.
Thesis (Ph. D.)--Arizona State University, 1977. Includes vita. Includes bibliographical references (leaves [193]-210). Photocopy.
A protective glycolipid antigen (PAg) was extracted from Leptospira interrogans serovar canicola with chloroform/methanol/water (1:2:0.8, by vol.) and partially purified by silica gel column chromatography. The PAg elicited a protective response in hamsters and in cyclophosphamide-treated mice subsequently challenged with homologous Leptospira. The PAg band was detected as a single smear-like band, corresponding to a protein of 23-30 kDa, by silver-staining in SDS-PAGE. In immunoblots, this band reacted with a monoclonal antibody, A5, which agglutinated serovar canicola and recognized a serovar-specific antigen. Furthermore, the PAg did not migrate on silica gel TLC, but was detected at the origin as a ninhydrin- and naphthol-positive spot. This suggests that PAg is a hydrophilic molecule with a carbohydrate chain that contains amino groups, possibly as amino sugars.
Companies spend lots of time and money on complex tools to assess customer satisfaction. But they're measuring the wrong thing. The best predictor of top-line growth can usually be captured in a single survey question: Would you recommend this company to a friend? This finding is based on two years of research in which a variety of survey questions were tested by linking the responses with actual customer behavior--purchasing patterns and referrals--and ultimately with company growth. Surprisingly, the most effective question wasn't about customer satisfaction or even loyalty per se. In most of the industries studied, the percentage of customers enthusiastic enough about a company to refer it to a friend or colleague directly correlated with growth rates among competitors. Willingness to talk up a company or product to friends, family, and colleagues is one of the best indicators of loyalty because of the customer's sacrifice in making the recommendation. When customers act as references, they do more than indicate they've received good economic value from a company; they put their own reputations on the line. And they will risk their reputations only if they feel intense loyalty. The findings point to a new, simpler approach to customer research, one directly linked to a company's results. By substituting a single question--blunt tool though it may appear to be--for the complex black box of the customer satisfaction survey, companies can actually put consumer survey results to use and focus employees on the task of stimulating growth.
Ergonomic requirements for office work with visual display terminals (VDTs), Part 11
ISO, 1998. Ergonomic requirements for office work with visual display terminals
(VDTs), Part 11, Guidance on usability (ISO 9241-11:1998E). Geneva,
Switzerland: Author.
The computer user satisfaction inventory (CUSI): Manual and scoring key
- J Kirakowski
- A Dillon
Kirakowski, J., & Dillon, A. (1988). The computer user satisfaction inventory (CUSI): Manual and scoring key. Cork, Ireland: Human Factors
Research Group, University College of Cork.
The usability construct: A dead end? Human-Computer Interaction
- N Tractinsky
Tractinsky, N. (2018). The usability construct: A dead end? Human-Computer Interaction, 33(2), 131-177. doi:10.1080/07370024.2017.12
98038
Lewis is a distinguished user experience researcher at MeasuringU, an IBM Master Inventor emeritus (over 90 patents), and the author of Practical Speech User Interface Design and
- R James
James R. Lewis is a distinguished user experience researcher at
MeasuringU, an IBM Master Inventor emeritus (over 90 patents), and
the author of Practical Speech User Interface Design and (with Jeff
Sauro), Quantifying the User Experience.