No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Perceived Usability and the Modified Technology Acceptance Model
International Journal of Human-Computer Interaction
Abstract
In response to recent criticism of the usefulness of the construct of usability, we investigated the relationships between measures of perceived usability and the components of a modified version of the Technology Acceptance Model (mTAM) – Perceived Usefulness (PU) and Perceived Ease-of-Use (PEU). In three surveys, respondents used SUS, UMUX-LITE and mTAM to rate their actual (as opposed to expected) experience with three software products. As expected, the correlations between PEU and other measures of perceived usability tended to be significantly stronger than those with PU. Additional findings support the use of the UMUX-LITE as a compact measure of perceived usability that has a strong relationship to the mTAM and strong correspondence with concurrently collected SUS scores. The main theoretical result of this research were regression results providing evidence that the PEU component of the mTAM appears to be another measure of the construct of perceived usability, connecting the TAM to the construct of perceived usability through the mTAM and providing evidence against the claim that the construct of usability is a theoretical dead end.
... Additionally, commonly used usability scales are very short, and adding a scale to the test does not significantly increase participant effort. In studies that use score difference as a checking method (Lah et al., 2020;Lewis, 2018a;Lewis, 2019), the current screening criteria is very rough, and only a limited number of studies have preliminarily reported the results of a 50-point difference screening. They did not explain the reason for using 50 points as a threshold, which may difficult to achieve and the purpose of an accurate cleaning. ...
... The 50-score difference cleaning standard has not been fully validated. Currently, only a few studies have used a 50score difference for cleaning (Lah et al., 2020;Lewis, 2018b;Lewis, 2019), but 50 points may not be accurate threshold. The effectiveness of data inspection varies widely and the percentage of cleaning is sometimes very small (Lah et al., 2020), which is not consistent with the results reported in crowdsourced testing or remote testing (Wang et al., 2020a). ...
... Currently, only a few studies have used a 50score difference for cleaning (Lah et al., 2020;Lewis, 2018b;Lewis, 2019), but 50 points may not be accurate threshold. The effectiveness of data inspection varies widely and the percentage of cleaning is sometimes very small (Lah et al., 2020), which is not consistent with the results reported in crowdsourced testing or remote testing (Wang et al., 2020a). Therefore, more precise and rigorous cleaning standards need to be explored. ...
To improve the accuracy of crowdsourced data, efficient elimination of low-quality data, commonly referred to as data cleaning, is a straightforward but powerful method. In this research, the effectiveness of various data cleaning methods was tested in two measuring environments. We used the score differences of three perceived usability questionnaires (SUS, UMUX, and mATM) to provide a new basis for accurately cleaning low-quality data. We accomplished this by observing the data cleaning effect of various score difference intervals. Our study ultimately showed that (1) significant differences in scores are found in different measurement settings, (2) completion time is a useful indicator for detecting low-quality data, (3) the correlation of the questionnaire after cleaning proves that a method combined with an inspection item pairs more strictly than a method using only completion time, and (4) a score difference of 30 points on the highly correlated perceived usability scale is a suitable cleaning threshold that is feasible for shorter questionnaires. Therefore, in the case of using a single questionnaire, a cleaning method combining the completion time and inspection item pairs can be used; we also recommend simultaneously using two standardized perceived usability questionnaires, a threshold score difference of 30 points between the two questionnaires can be used as a cleaning criterion.
... In practice, mTAM item ratings are manipulated to produce a measure that can range from 0 to 100. Lah et al. (2020) used data from three surveys to investigate the relationships between mTAM and alternate metrics of perceived usability and usefulness as drivers of overall experience and likelihood to recommend (LTR; Reichheld, 2003). The drivers (independent variables) they evaluated were the System Usability Scale (SUS; Brooke, 1996;Lewis, 2018b) and various metrics derived from the Usability Metric for User Experience (UMUX; Finstad, 2010;Sauro & Lewis, 2016). ...
... In three surveys conducted by Lah et al. (2020), respondents used SUS, UMUX-LITE and mTAM to rate their actual (as opposed to expected) experience with three different software products. As expected, the correlations between the mTAM measure of PEoU and the other measures of perceived usability (SUS and the UMUX-LITE Ease item) tended to be significantly stronger than correlations with measures of perceived usefulness (the mTAM measure of PU and the UMUX-LITE Usefulness item), evidence of construct validity for the distinction between PEoU and PU. ...
... Continuing the investigation into the relationships among various measures of perceived usability (Lah et al., 2020;Lewis, 2018aLewis, , 2019b, the major goals of the current paper were to replicate and extend the Lah et al. models with a new dataset that had some variation in the independent (drivers) and dependent (outcome) variables. ...
We replicated and extended previous research to investigate the extent to which perceived ease of use and perceived usefulness account for variation in overall experience, likelihood to recommend, intention to use, and reported usage in a three-month follow-up. Consistent with previous research, we found little effect on structural equation models from varying three measures of perceived ease and two measures of perceived usefulness. All models had statistically significant standardized estimates and squared multiple correlations and had acceptable fit statistics. Despite these manipulations, the models supported a consistent narrative. Both perceived ease and perceived usefulness are important antecedents that either directly or indirectly affect the experiential and intentional outcomes (perceived usefulness somewhat more than perceived ease), with intention to use accounting for 19% of variation in follow-up ratings of usage. These models support UX practitioners by demonstrating the importance of work that improves perceptions of product ease and usefulness and showing that the two-item UX-Lite questionnaire is an effective and efficient measure of perceived ease and usefulness.
... Effectiveness and efficiency are primarily seen as the goals of socio-technical system usability, which is the fundament for user experience (UX) and a prerequisite for the acceptance of socio-technical information and assistance systems by the users in the specific field of application (Benyon, 2014(Benyon, , 2020J. Brooke, 2013;Lah et al., 2020;Nielsen, 1993). In summary, system usability is linked to effectiveness, efficiency, and the overall acceptance of technology (International Organization for Standardization ISO, 2018). ...
... Recent discourse criticised the use of usability, and Lah et al. (2020) investigated the relationship between perceived usability and perceived usefulness/ perceived ease of use of the TAM (Lewis, 2018a;Tractinsky, 2018). Lewis (2018a) provided evidence that perceived ease of use is another measure of perceived usability, connecting the TAM to the usability construct and providing an outlook into the future: "...I believe the construct of usability has a bright future both in usability science (theory) and usability engineering (practice), either alone or as a fundamental part of the larger assessment of user experience" (Lewis, 2018a, p. 6). ...
... • The high SUS supports the ease of use measure of technology acceptance (Lah et al., 2020). ...
The requirements for the design of information and assistance systems in labour-intensive processes are interdisciplinary and have not yet been sufficiently addressed in research. This dissertation analyses, evaluates and describes possibilities for increasing the effectiveness and efficiency of labour-intensive processes through design-optimised socio-technical systems. The work thus contributes to further developing information and assistance systems for industrial applications and use in healthcare. The central dimensions of people, activity, context and technology are the focus of the scientific investigations following the Design Science Research paradigm. Design principles derived from this, a corresponding taxonomy, and a conceptual reference model for the design of socio-technical systems are the results of this dissertation.
... We also utilized the Technology Acceptance Model (mTAM) questionnaire (Lah et al. 2020) to quantitatively measure effects of the actions we took during this study. The mTAM questionnaire consists of six perceived usefulness (PU) and six perceived ease-of-use (PEU) questions. ...
... Participants who were already part of the development team and had experience on code before refactoring and after DDD refactoring answered to questions can be seen in Table 7. Using the questionnaire results, we computed mTAM scores for PU and PEU, applying the approach in Lah et al. (2020), which is as follows: ...
... As a result of the computation, DDD refactoring scored 85 in PU and 83 in PEU, leading to an overall mTAM score of 84 out of 100. Lah et al. (2020) do not specify a way to interpret the mTAM values. However, mTAM is designed to be similar to the System Usability Scale (SUS), which is a robust and reliable tool for measuring the usability (Brooke 2020). ...
Context
Software developers need to constantly work on evolving the structure and the stability of the code due to changing business needs of the product. There are various refactoring approaches in industry which promise improvements over source code composition and maintainability.
Objective
In our research, we want to improve the maintainability of an existing system through refactoring using Domain-Driven Design (DDD) as a software design approach. We also aim for providing empirical evidence on its effect on maintainability and the challenges as perceived by developers.
Method
In this study, we applied the action research methodology, which facilitates close academia-industry collaboration and regular presence in the studied product. We utilized focus groups to discover problems of the existing system with a qualitative approach. We reviewed the subject codebase to construct our own expert opinion as well and identified problems in the codebase and matched them with the ones raised by engineers in the team. We refactored the existing software system according to DDD principles. To measure the effects of our actions, we utilized Technology Acceptance Model (mTAM) questionnaire, and also semi-structured interviews with the development team for data collection, and card sorting methodology for qualitative analysis. For minimizing bias that might affect our results with the existing software engineers in the team, we extended our measurement with three new joiner software engineers in the team through the think aloud protocol.
Results
We have identified that engineers mostly gave positive answers to our interview questions, which are mapped to software maintainability metrics defined by ISO/IEC 25010. Our DDD refactoring scored 85 in PU and 83 in PEU, leading to an overall mTAM score of 84. This means acceptable on the acceptability scale, B on the grade scale, and good on the adjective rating scale.
Conclusion
Our research led us to conclude that a powerful design approach, like DDD, is an effective tool for restructuring and resolving software issues in this situation. It offers standardization to the software and the refactoring efforts. We realized that DDD entails a certain degree of complexity and cognitive load, which is a barrier for software engineers, but they are aware of its benefits.
... There have been several attempts to estimate SUS scores from UMUX-LITE scores. A first approach (Lewis et al., 2013(Lewis et al., , 2015, which uses a regression formula, was later found to be biased for products that showed higher SUS scores (Lah et al., 2020). Other options are to compute the SUS score directly from the ease-of-use question of the UMUX-LITE or using a regression formula based on this ease-of-use question (Lewis & Sauro, 2021c, 2021d. ...
... There are several estimation methods to predict a SUS score from available UMUX-LITE data. Of course, the pure UMUX-LITE score can be used, as suggested by Lah et al. (2020). Other methods are these: (The abbreviation EoU refers to the ease-of-use question, "This system is easy to use.") 1. Method 1: SUS score = 0.65 * UMUX-LITE score + 22.9 as proposed in Lewis et al. (2013) 2. Method 2: SUS score = (EoU -1) * (100/6) as proposed in Lewis & Sauro (2021c) 3. Method 3: SUS score = -2.279 ...
... much better estimation of the SUS score than the estimation based on the regression formula from Lewis et al. (2013), which is in line with the results in Lah et al. (2020). Overall, the best predictions are obtained by Methods 2 and 3, which only consider the ease-ofuse question and not both questions of the UMUX-LITE. ...
A loyal customer base depends upon a good user experience over the product’s complete lifetime. Successful products are continuously developed over a long period. Their functionality and complexity typically grow over years, so it is important to measure their user experience continuously. A carefully selected, effective questionnaire can collect quantitative results. But with so many established UX questionnaires available, it is often difficult to choose a suitable one for a specific project. The task becomes more complex if different UX questionnaires are used and results must be compared. It is essential to understand the relationship between user experience data collected with different questionnaires.
We investigated three common user experience questionnaires, SUS, UMUX-LITE, and UEQ-S, used to evaluate four common products in an online study of 435 participants: Netflix®, PowerPoint® (PPT), Zoom®, and BigBlueButton™ (BBB). In this way, the measured scale scores of the questionnaires could be compared for these products. Results showed SUS and UMUX-LITE scores as nearly identical for all four products. For usability or UX quality, we found that the selection of the survey has only a limited impact, but for overall UX quality there were clear differences between SUS, UMUX-LITE, and UEQ-S.
... Overall experience is the global responses or evaluation outcomes of users of a product after using it, which have positive correlations with usability, pragmatic quality, and hedonic quality [22,23]. Similarly, some studies [24,25] suggest that affect and usability can significantly predict overall experience. ...
... Overall experience [14,23,25,26,40] OE1 I feel motivated to continue to use the wearable robot. OE2 ...
... In addition, the survey on the UX of wearable robots showed clear effects of usability, hedonic quality, and attitude on the overall user experience when using WRs. In contrast to the findings of [22,23,25], the largest direct effect was found for hedonic quality rather than usability. Nevertheless, users' perceived usability plays an important role in predicting their overall experience of using WRs. ...
Wearable robots (WRs) might interact with humans in a similar manner to teammates to accomplish specific tasks together. However, the available data on WR user experience (UX) studies are limited, especially during the prototyping phase. Therefore, this study aims to examine the overall experience of WRs during the prototyping phase based on an exploratory research model. This theoretical model considered usability, hedonic quality, and attitude toward using WRs as key factors in explaining and predicting overall experience. To test the hypotheses inherent in the research model, quantitative empirical research was conducted and the data were analyzed by partial least squares structural equation modeling (PLS-SEM). The results from the PLS-SEM analysis revealed the significance level of correlations between the latent variables in the research model. The exploratory research model was able to explain up to 53.2% of the variance in the overall experience of using WRs, indicating medium predictive power. This research develops a new quantitative empirical research model that can be used to explain and predict the overall experience of interactive products such as WRs. Meanwhile, the model is needed during WR testing in the prototype phase.
... In this study, we assessed 45 different UMUX variants based on the combinations of 12 candidate items, which are obtained by translations from versions in English (Finstad, 2010), Italian (Borsci et al., 2015), Slovene (Lah et al., 2020), Arabic (Al-Tahat, 2020) and Chinese (Wang et al., 2022). ...
... Another reason could be the researchers' adhoc research requirements as in Borsci et al. (2015), where the main goal of the study is to explore the effect of amount of experience on usability, rather than making an adaptation of the UMUX scale. Similarly, Lah et al. (2020) translated UMUX for their study on the relationship of usability with technology acceptance. Furthermore, the recent revisions made in ISO 9214-11 (2018) might have affected the researchers' perspective of usability, which could lead to deviations in translations, since the original UMUX items were established in 2010. ...
... There is also further evidence on the unidimensionality of UMUX through factor analysis in these studies. Furthermore, Lah et al. (2020) provided evidence of convergent validity of UMUX in comparison with PEU (perceived ease of use) and PU (perceived usefulness) of TAM (technology acceptance model) (Davis, 1989) and a single item LTR (Likelihood to Recommend) measure through regression analysis. Based on their results, they claim that PEU and SUS are two different measures of the same construct of perceived usability and UMUX-LITE can be used as a short-form alternative to SUS since it had a high correspondence with concurrently collected SUS scores. ...
Within the last decade, the Usability Metric for User Experience is translated into many languages. This study aimed to create an adapted version of UMUX into Turkish, based on item translations made from Arabic, Chinese, English, Italian and Slovene versions, which vary for the content of the items as well as their differences in how they are processed by machine translation systems and understood by human translators. Based on different translations, 45 UMUX variants in Turkish language are assessed psychometrically as a formative construct, regarding the criteria based on PLS measurement models for formative constructs and Rasch analysis. The results are benchmarked with the data collected via UMUX, SUS and CSUQ in Arabic and English or Turkish and English concurrently. Results show that all 45 Turkish variants of UMUX reveal strong psychometric qualities as a formative construct, as well as UMUX and Arabic UMUX. While there is evidence that usability can be measured as a formative construct, differences between item sets suggest that UMUX may not be a complete measure that includes all aspects of usability regarding ISO 9241-11. Our results also suggest that UMUX scores are sensitive to the native language of participants since the mean scores were significantly different between the native and English UMUX versions which were responded concurrently to assess the same software.
... In order to provide a strong theoretical base for choosing the most important factors, this article examined two significant streams of literature, that is, (a) TAM and (b) E-CRM. This article selected the TAM model to address the E-CRM acceptance because of its strong theoretical base, and many studies in the advanced economy have supported it (Lah et al., 2020;Yoon & Lim, 2020). TAM has become the leading research stream in the discipline of the Information System (IS) (Hirschheim, 2007), which was developed by Davis in 1989. ...
... When the customers feel that with the use of such technology, they are able to save their valuable time and at the same time are very efficient in using such technology, it arouses the positive attitude among them to use the same in the near future also (Taherdoost, 2018). Additionally, perceived usefulness also influences behavioural intention of the customers in using the technology, which was well supported in the previous literature (Diop et al., 2019;Lah et al., 2020;Yoon & Lim, 2020). Perceived usefulness is one of the important prime factor that influences users' intention to use a technology. ...
... Perceived usefulness is one of the important prime factor that influences users' intention to use a technology. The more the customers possess the convictions of perceived usefulness before use, the more they will be able to pursue and use such technology (Lah et al., 2020). Therefore, the proposed hypotheses were: ...
The present study has aimed to understand the customers’ behavioural intention to use electronic customer relationship management (E-CRM) in the banking industry, using the technology acceptance model (TAM). Purposive sampling technique is used to collect data from 300 banks that included customers from Delhi (India), and partial least square-structural equation modelling (PLS-SEM) was employed. The reliability and the validity of the overall model were evaluated using the measurement model, and the hypotheses were tested using the structural model. The results reveal that E-CRM had a positive impact on both perceived usefulness and perceived ease of use, which, in turn, had a positive impact on the attitude towards using E-CRM, which ultimately led to a positive behavioural intention of customers towards using E-CRM. This study reveals that there is a large potential for policymakers to raise awareness and acceptance of E-CRM services among the customers.
... One of these assumptions is perceived usefulness. TAM assumes that individuals have a high chance of accepting and using a technology if they view it as useful in boosting their performance or making their work easier (Lah, Lewis & Šumak, 2020). Another assumption of this theory is based on perceived ease of use. ...
... The users' perceptions of the ease of using a technology significantly influence their acceptance and adoption of it. The model posits that individuals are likely to embrace technology if they have a perception of technology to be uncomplicated, user-friendly, and convenient to use (Lah, Lewis & Šumak, 2020). This perception of ease of use contributes to the willingness to adopt and utilize the technology. ...
Purpose: The growth of Kenyan commercial banks has been sluggish despite the widespread adoption of the latest trends. This delayed progress can be attributed to the uncertainty surrounding the outcomes of the devised e-banking strategies by management. The growth of commercial banks in Nairobi City County, Kenya has been significantly affected by the rate of adoption of technology, which is one of the latest trends in banking. A majority of the clients of commercial banks in Kenya have been hesitant to embrace the new trends in form of online banking in preference for traditional banking as a result of a lack of trust. In addition, there have been also some challenges in Point of Sale (POS) systems, e-banking transfers, virtual assistants, and chatbots. This research aimed to investigate the impact of e-banking strategies on the growth of selected commercial banks in Nairobi City County, Kenya. Methodology: To achieve the study's objectives, a descriptive research design was employed. The target population comprised managers of commercial banks responsible for strategy implementation. The sample included Digital Banking Managers, Operational Managers, IT Managers, Customer Relations Managers, Finance Managers, and Business Development Managers, totaling 54 respondents. Before data collection, a pilot study was carried out to assess the feasibility of the research. Additionally, validity and reliability tests were carried out to ensure the research instrument accurately measures its intended variables. Data was collected through a combination of closed and open-ended questionnaires, administered via drop-and-pick methods. Subsequently, the collected data was analyzed through descriptive analysis. Results: The study found that e-banking strategies, including SMS banking, POS systems, Electronic Bank Transfers, and virtual assistants/chatbots, significantly impact the growth of commercial banks in Kenya, despite challenges such as unread messages and interoperability issues. Conclusion: It was concluded that banks should address usability concerns, strengthen security measures, and prioritize user-centric design to maximize benefits. Recommendations include enhancing SMS banking usability of commercial banks, improving transaction security, and investing in technological innovations for operational efficiency and competitiveness.
... The study proposed that future studies could consider replacing the PEU construct of TAM with PUsab. In a study conducted by Lah, Lewis, and Šumak [35], PUsab was linked to the PEU construct of TAM. ...
... We could not find any literature to support or explain this relationship and therefore suggest further investigation. The findings of this study also revealed that PU and PUsab are highly related, which is in contrast with the findings of [33][34][35]. PU and PUsab being related imply that these cannot be modelled together and therefore PUsab cannot replace PEU in modelling LMS adoption. To explain this finding, we call attention to the components of the PUsab construct, i.e., effectiveness, efficiency and user satisfaction [42] whereby the effectiveness component could be per definition related to the PU construct of TAM, i.e., if technology is effective in allowing a user to complete a task it would be considered useful. ...
The digitalization of education has made learning management systems (LMSs) an indispensable tool for facilitating teaching and learning in higher education institutions (HEIs). Research studies on the factors that influence LMS usage in Sub-Saharan Africa is lacking specifically in considering both technology adoption and usability as theoretical lenses. This study investigates the factors that influence the use of LMSs in HEIs in Ghana. Specifically, what factors influence perceived usefulness (PU) and perceived usability (PUsab) and if PU and PUsab can be used together in modelling the usage of the LMS. This quantitative study used a validated questionnaire to conduct a survey with 300 respondents from three Ghanaian HEIs. The data was analyzed using the Partial Least Squares Structural Equation Modeling regression technique. The findings confirm that human factors (self-efficacy and learning style), social influence and access to technical support influence PUsab and the same factors also influence perceived PU. Both PUsab and PU influences LMS use among students in Ghana. The constructs PU and PUsab were found to be highly related when evaluating the factors influencing LMS usage and could thus not be adequately distinguished to be modelled together. Therefore, the construct PUsab could not replace perceived ease of use (PEU), as previous studies have suggested. While the constructs of PUsab and PEU seem semantically similar, PUsab includes the effectiveness construct, which could explain the relatedness between PUsab and PU. This finding has important implications for understanding how the technology adoption literature intersects with the usability literature.KeywordsLearning Management System Perceived usabilityPerceived ease of usePerceived usefulnessLMSTAMGhana
... TAM menganggap bahwa dua keyakinan individual, yakni persepsi manfaat (perceived usefulness) dan persepsi kemudahan peng-gunaan (perceived ease of use), sebagai faktor utama perilaku dalam menggunakan teknologi. Pada penelitian ini menggunakan TAM sebagai model dasar, namun variabel perilaku niat untuk menggu-nakan (behavioral intention) dan sikap (attitude toward using) dan ukuran-ukuran penggunaan sesung-guhnya (Actual Use) yang tersedia (Davis, 1989;Davis, 1993;Davis, Bagozzi, Warshaw, 1989;Lah et al., 2020 ...
The purpose of this research is to explain and analyze E-Catalog Adoption in Banjar Regency UMKM using the Technology Acceptance Model (TAM) which consists of exogenous variables namely Perceived Usefulness, Perceived Ease of Use and endogenous variables namely Attitude Toward Using, Behavioral Intention and Actual Use. The respondents in this study were 341 UMKM registered in the Banjar Regency E- Catalog This research uses the SEM (Structural Equation Model) analysis method using Partial Least Square (PLS). The results of this research show that 4 hypotheses are accepted, namely the Perceived usefulness variable has a significant effect on Attitude Toward Using, Perceived Ease of Use has a positive and significant effect on Attitude Toward Using, Perceived Ease of Use has a positive and significant effect on Behavioral Intention, and Behavioral Intention has a positive and significant effect. significant to Actual Usage. Meanwhile, there were 2 hypotheses that were rejected, namely Perceived Usefulness had no significant effect on Behavioral Intention and Attitude Toward Using had no significant effect on Behavioral Intention of the Banjar Regency UMKM E-Catalog.
... The International Organization for Standardization [10] defines usability as the degree to which a product can be used by specific users to achieve particular goals with effectiveness, efficiency, and satisfaction in a given context. Usability directly affects users' intention to adopt and continue using technology [11], and thus, ensuring good usability is crucial for engagement with mental health mHealth solutions [12,13]. Moreover, individuals with mental illnesses, particularly in Africa, often face challenges such as not owning mobile devices or sharing them with family members or caregivers, which compromises privacy and confidentiality. ...
Background
Harnessing mobile health (mHealth) solutions could improve the delivery of mental health services and mitigate their impact in Uganda and similar low-resource settings. However, successful adoption requires that mHealth solutions have good usability. We have previously implemented a telephone service to provide mental health information and advice in English and Luganda, utilizing an automated interactive voice response (IVR) system linked to live agents, including mental health care workers and peer support workers.
Objective
This study aims to assess the usage and usability of this mental health telephone service.
Methods
We obtained usage data from the system’s call logs over 18 months to study call volumes and trends. We then surveyed callers to gather their characteristics and assess usability using the Telehealth Usability Questionnaire. Additionally, call recordings were evaluated for conversation quality by 3 independent health care professionals, using the Telephone Nursing Dialogue Process, and correlations between quality and usability aspects were investigated.
Results
Over 18 months, the system received 2863 meaningful calls (ie, calls that went past the welcome message) from 1125 unique telephone numbers. Of these, 1153 calls (40.27%) stopped at the prerecorded IVR information, while 1710 calls (59.73%) opted to speak to an agent. Among those who chose to speak with an agent, 1292 calls (75.56%) were answered, 393 calls (22.98%) went to voicemail and were returned in the following working days, and 25 calls (1.46%) were not answered. Usage was generally sustained over time, with spikes in call volume corresponding to marketing events. The survey (n=240) revealed that most callers were caregivers of patients with mental health issues (n=144, 60.0%) or members of the general public (n=46, 19.2%), while a few were patients with mental health issues (n=44, 18.3%). Additionally, the majority were male (n=143, 59.6%), spoke English (n=180, 75.0%), had postsecondary education (n=164, 68.3%), lived within 1 hour or less from Butabika Hospital (n=187, 77.9%), and were aged 25-44 years (n=160, 66.7%). The overall usability score for the system was 4.12 on a 5-point scale, significantly higher than the recommended target usability score of 4 (P=.006). The mean scores for usability components ranged from 3.66 for reliability to 4.41 for ease of use, with all components, except reliability, scoring higher than 4 or falling within its CI. Usability scores were higher for Luganda speakers compared with English speakers, but there was no association with other participant characteristics such as sex, distance from the hospital, age, marital status, duration of symptoms, or treatment status. The quality of call conversations (n=50) was rated at 4.35 out of 5 and showed a significant correlation with usability (Pearson r=0.34, P=.02).
Conclusions
We found sustained usage of the mental health telephone service, along with a positive user experience and high satisfaction across various user characteristics. mHealth solutions like this should be embraced and replicated to enhance the delivery of health services in Uganda and similar low-resource settings.
... This score is then transferred to a range between 0 and 100 by multiplying it with a factor of 100/8 = 12.5. By averaging over all participants we finally get a UX-Lite score for the product, which can be compared to the SUS score (also ranging between 0 and 100) and that allows us to use the well-established SUS benchmark (Lewis and Sauro 2018;Lah et al. 2020). Hence, observed UX-LITE scores can be classified into eleven categories shown in Table 1. ...
Converting customer survey feedback data into usable insights has always been a great challenge for large software enterprises. Despite the improvements on this field, a major obstacle often remains when drawing the right conclusions out of the data and channeling them into the software development process. In this paper we present a practical end-to-end approach of how to extract useful information out of a data set and leverage the information to drive change. We describe how to choose the right metrics to measure, gather appropriate feedback from customer end-users, analyze the data by leveraging methods from inferential statistics, make the data transparent, analyze large volumes of user comments efficiently with Large Language Models, and finally drive change with the results. Furthermore, we present an example of a UX dashboard that can be used to communicate the analyses to stakeholders within the company.
... This was also evident in the scale correlations, as UMUX-LITE and SUS showed high correlations with each other. This confirmed the suggestion [39], [40] that the results of the UMUX-LITE can predict those of the SUS in scenarios with a low number of questions. On the other hand, the dimension of pragmatic quality of the UEQ-S also showed high to very high correlations with the SUS and UMUX-LITE. ...
Measuring the user experience (UX) of products, systems and services is individual depending on the research question. On the one hand, the user's goals and environment play a role in the subjective evaluation. On the other hand, different UX factors are relevant depending on the product. In this case, it is practical to have a questionnaire family as an aid, whose questionnaires are geared towards these different use cases. The User Experience Questionnaire (UEQ) family allows researchers and practitioners to choose the right tool for efficient UX measurement from three questionnaire versions. This article summarizes the UEQ, its short version (UEQ-S) and a modular framework (UEQ+) with overall 27 UX factors and purposes in over 30 different languages. In addition, specific instructions and assistance are provided for the statistical evaluation and interpretation of the questionnaire results. With the help of a key performance indicator (KPI), benchmarks and an importance-performance analysis (IPA), the realization of UX measurements is made easier for researchers and practitioners. To make it even more convenient to choose the right questionnaire from the UEQ family, influencing factors on the UX measurement and recommendations for action are given.
... La majorité des études sur l'adoption se penchent sur les technologies de l'information dans les systèmes d'information (Benbasat & Barki, (2007) ; Biot-Paquerot et al., (2018 (Lah et al., 2020). Parmi les modèles utilisés, cette étude présente ceux liés aux caractéristiques de l'innovation, basés sur la théorie de la diffusion de Rogers (1995) & (Kühne, 2019). ...
Abstract:
This thesis aims to add to our knowledge of the adoption of financial technologies by SMEs. It analyses the relationship between SMEs and banks after the adoption of digital financial services (post-adoption phase), using SMEs in the cities of Bukavu and Goma in DR Congo as a case study. To test the hypotheses formulated and achieve the objectives, we used principal component factor analysis (PCA) in an exploratory approach, as well as structural equations under the partial least squares (PLS) approach. The results showed that the adoption of digital financial services has a positive influence on the relationship developed between SMEs in Bukavu and Goma and the banks that offer these services. The variables that directly explain the adoption of digital financial services by SMEs in these cities are perceived usefulness (0.158***), perceived risk (0.856***) and convenience (-0.178***). Similarly, in the adoption phase, the results showed that the variables retained in the model explained 93% of the variance in the intention to use NFS and 2.2% of the variance in the current use of NFS by SMEs in Bukavu and Goma. In the post-adoption phase, the PLS algorithm technique indicated that, overall, the variables introduced into the model explained the model correctly. The m-relationship variables produced satisfactory R2 values: 67% for satisfaction, 52% for trust and 65% for commitment. As for the m-fidelity variables, the results showed an R2 of 47% for the long-term relationship, an R2 of 48% for the in-depth relationship and an R2 of 59% for the extended relationship.
Keywords: Adoption, digital financial services, SME-Bank relationship, City of Bukavu and Goma.
... La majorité des études sur l'adoption se penchent sur les technologies de l'information dans les systèmes d'information (Benbasat & Barki, (2007) ; Biot-Paquerot et al., (2018 (Lah et al., 2020). Parmi les modèles utilisés, cette étude présente ceux liés aux caractéristiques de l'innovation, basés sur la théorie de la diffusion de Rogers (1995) & (Kühne, 2019). ...
Résumé :
Cette thèse vise à enrichir les connaissances sur l’adoption des technologies financières par les PMEs. Elle analyse la relation entre les PMEs et les banques après l’adoption des services financiers numériques (phase post-adoption), en prenant comme cas d’étude les PMEs des villes de Bukavu et Goma en RD Congo. Pour tester les hypothèses formulées et atteindre les objectifs, nous avons utilisé les analyses factorielles en composantes principales (ACP) dans une approche exploratoire, ainsi que les équations structurelles sous l’approche des moindres carrés partiels (PLS). Les résultats ont montré que l’adoption des services financiers numériques a une influence positive sur la relation développée entre les PME de Bukavu et Goma et les banques qui offrent ces services. Les variables qui expliquent directement l’adoption des services financiers numériques par les PMEs de ces villes sont l’utilité perçue (0,158***), le risque perçu (0,856***) et la commodité (-0,178***). De même, dans la phase d’adoption, les résultats ont prouvé que les variables retenues dans le modèle expliquent à 93% la variance de l’intention d’utilisation des SFN et à 2,2% la variance de l’utilisation actuelle des SFN par les PME de Bukavu et Goma. Dans la phase post-adoption, la technique de l’algorithme PLS a indiqué que, dans l’ensemble, les variables introduites dans le modèle expliquent globalement et correctement le modèle. Les variables de la m-relation ont dégagé des R2 satisfaisants : 67% pour la satisfaction, 52% pour la confiance et 65% pour l’engagement. En ce qui concerne les variables de la m-fidélité, les résultats ont dégagé un R2 de 47% pour la relation dans la durée, un R2 de 48% pour la relation approfondie et un R2 de 59% pour la relation étendue.
Mots clés : Adoption, Services financiers numériques, relation PME-Banque, Ville de Bukavu et Goma.
Abstract:
This thesis aims to add to our knowledge of the adoption of financial technologies by SMEs. It analyses the relationship between SMEs and banks after the adoption of digital financial services (post-adoption phase), using SMEs in the cities of Bukavu and Goma in DR Congo as a case study. To test the hypotheses formulated and achieve the objectives, we used principal component factor analysis (PCA) in an exploratory approach, as well as structural equations under the partial least squares (PLS) approach. The results showed that the adoption of digital financial services has a positive influence on the relationship developed between SMEs in Bukavu and Goma and the banks that offer these services. The variables that directly explain the adoption of digital financial services by SMEs in these cities are perceived usefulness (0.158***), perceived risk (0.856***) and convenience (-0.178***). Similarly, in the adoption phase, the results showed that the variables retained in the model explained 93% of the variance in the intention to use NFS and 2.2% of the variance in the current use of NFS by SMEs in Bukavu and Goma. In the post-adoption phase, the PLS algorithm technique indicated that, overall, the variables introduced into the model explained the model correctly. The m-relationship variables produced satisfactory R2 values: 67% for satisfaction, 52% for trust and 65% for commitment. As for the m-fidelity variables, the results showed an R2 of 47% for the long-term relationship, an R2 of 48% for the in-depth relationship and an R2 of 59% for the extended relationship.
Keywords: Adoption, digital financial services, SME-Bank relationship, City of Bukavu and Goma.
... After data cleaning according to the method described by Sauro and Lewis (2011), we retained responses from 255 participants. After further applying the cleaning methods used for two questionnaire score differences (Lah et al., 2020;Lewis, 2019;Wang et al., 2023), we excluded seven more participants whose score differences were greater than 40 points, and ultimately retained 248 valid responses. Each participant received a payment of 3 RMB. ...
This paper describes the development of a cartoon animation system usability scale (A-SUS) based on the established text-based SUS questionnaire. We propose a methodology and design a short graphic interchange format (.GIF) animation for each SUS item. Experimental evaluations confirm that the scale has satisfactory psychometric properties (e.g., structural validity, reliability factor structure, concurrent validity, and sensitivity). A second experiment is used to evaluate and compare the questionnaire experiences associated with the SUS, a pictorial SUS (P-SUS), and the developed A-SUS. The results indicate that A-SUS performs well in terms of recommendations, aesthetics, motivations, and completion time. Compared with the SUS and P-SUS, the animated version is more interesting, and the overall questionnaire experience is better.
... The TAM's validity and its effective explanation of end-user system usage are well established [30]. According to the TAM, people's beliefs about a technology's usefulness and usability are key predictors of their attitude toward it [31]. This attitude then determines their intention to use the technology, which in turn influences actual usage. ...
In contrast to the extensive research on the impact of green buildings on character, property values, energy efficiency, and environmental quality, less emphasis has been placed on understanding the factors driving users’ selection of green real estate projects, an essential element for the future business sustainability of such buildings. This study investigates the connection between various user motivations, such as perceived usefulness, perceived usability, a sense of responsibility, awareness of consequences, and perceived behavioral control, and their impact on Asian users’ willingness to pay for green real estate projects. These users include those who have already invested in and are considering long-term investment in such projects. An empirical methodology, integrating these motivations with a technology acceptance model, is proposed to assess their willingness to pay either through renting or buying. The study applies structural equation modeling to analyze sample data against a hypothetical model, thereby testing the hypotheses. It concludes that, in influencing green real estate projects, the following apply: (1) perceived usefulness and perceived behavioral control significantly shape Asian users’ attitudes towards financial commitment; (2) perceived usability directly influences their perceived usefulness; (3) awareness of consequences acts as a mediating factor in the relationship between perceived usability and behavioral control among these users. Moreover, the study offers strategic design recommendations for future green real estate projects, informed by the diverse motivational factors of Asian users, to guide their purchasing and selection decisions.
... The emphasis on constructivist pedagogy and authentic assessment is consistent with past research advocating student-centered, real-world application of knowledge (Darling-Hammond et al., 2020;Judd, 2018). Moreover, the Technology Acceptance Model (TAM) has been validated by the willingness of educators to explore various digital tools, demonstrating their perceived usefulness and ease of use in the teaching process (Bower et al., 2015;Lah et al., 2020). The implications of the study's findings offer valuable insights for educators, administrators, and policymakers in marketing science education. ...
This research emerged in response to the challenge of improving students' academic success in marketing science. With the development of digital technology, digital-based teaching methods are considered as potential innovations to enrich the learning experience. The main objective of this study is to evaluate the effectiveness of digital-based teaching innovations. It includes understanding whether the use of digital tools and interactive methods enhances students' understanding of marketing science concepts and academic performance. This study is a case study that focuses on the implementation of digital-based teaching methods in a marketing science education environment. The research subjects involved students participating in digital-based marketing science classes at the Sultan Muhammad Syafiuddin Islamic Institute in Sambas. This research applies a combined approach between qualitative and quantitative through digital-based teaching using various digital tools such as multimedia presentations, interactive online platforms, and virtual simulations. The results of this case study show that digital-based marketing science teaching innovation has a positive impact. Students show high interest, active participation, increased understanding of concepts, and improved academic performance. Overall, this research supports the effectiveness of digital-based teaching in marketing science. The integration of digital tools and interactive methods can improve the quality of education and provide a more engaging learning environment for students.
... In [37], it was discovered that perceived usability holds crucial importance in the TAM and that its presence explains a greater amount of variance in the model compared to its absence. In [38], it was found that perceived ease-of-use (PEU) is strongly related to perceived usability, which is a component of the modified TAM (mTAM). However, Ref. [39] proposed a theoretical model that suggests perceived ease of use is determined by control, intrinsic motivation, and emotion, and that it adjusts over time to reflect objective usability and perceptions of external control. ...
Due to the irruption of new technologies in cities such as mobile applications, geographic information systems, internet of things (IoT), Big Data, or artificial intelligence (AI), new approaches to citizen management are being developed. The primary goal is to adapt citizen services to this evolving technological environment, thereby enhancing the overall urban experience. These new services can enable city governments and businesses to offer their citizens a truly immersive experience that facilitates their day-to-day lives and ultimately improves their standard of living. In this arena, it is important to emphasize that all investments in infrastructure and technological developments in Smart Cities will be wasted if the citizens for whom they have been created eventually do not use them for whatever reason. To avoid these kinds of problems, the citizens’ level of adaptation to the technologies should be evaluated. However, although much has been studied about new technological developments, studies to validate the actual impact and user acceptance of these technological models are much more limited. This work endeavors to address this deficiency by presenting a new model of personalized recommendations based in the technology acceptance model (TAM). To achieve the goal, this research introduces an assessment system for tourists’ digital maturity level (DMT) that combines a fuzzy 2-tuple linguistic model and the analytic hierarchy process (AHP). This approach aims to prioritize and personalize the connection and communication between tourists and Smart Cities based on the digital maturity level of the tourist. The results have shown a significant correlation between technology usage and the potential for personalized experiences in the context of tourism and Smart Cities.
... O TAM foi proposto inicialmente por [Davis et al. 1989] e vem sendo empregado na criação de modelos de aceitação em tecnologia da informação [Lah et al. 2020]. De acordo com o modelo, quando os usuários são apresentados a uma nova tecnologia, a utilidade e a facilidade de uso percebida são fatores que influenciam na decisão de como e quando seus usuários utilizarão a nova tecnologia. ...
O presente artigo apresenta a concepção do framework iHorus enquanto parte do Projeto SANEP-I2MF, direcionado a capturar informações sensoriadas em uma base operacional amplamente distribuída com a finalidade de atender às demandas do Serviço Autônomo de Saneamento de Pelotas (SANEP). A solução proposta emprega o middleware EXEHDA e explora ciência de contexto para inferir o estado dos diferentes artefatos eletromecânicos presentes na infraestrutura de saneamento básico. A Internet das Coisas (IoT, do inglês Internet of Things) é adotada como um recurso de ubiquidade para atender as diferentes interoperações necessárias. Para avaliação do iHorus, foi utilizado o Modelo TAM (Technology Acceptance Model), mensurando a utilidade percebida e a sua facilidade de uso, junto a diferentes segmentos da comunidade usuária. Os resultados foram promissores e apontam para continuidade da pesquisa.
... In [36] discovered that perceived usability holds crucial importance in the TAM and that its presence explains a greater amount of variance in the model compared to its absence. In [37] found that perceived ease-of-use (PEU) is strongly related to perceived usability, which is a component of the modified TAM (mTAM). However, in [38] proposed a theoretical model that suggests perceived ease of use is determined by control, intrinsic motivation, and emotion, and that it adjusts over time to reflect objective usability and perceptions of external control. ...
Due to the irruption of new technologies in cities such as mobile applications, geographic infor-mation systems, Big Data, Internet of Things (IoT) or Artificial Intelligence (AI), new approaches to citizen management are being developed with the aim of adapting citizen services to this new en-vironment. These new services can enable city governments and businesses to offer their citizens a truly immersive experience that facilitates their day-to-day lives and ultimately improves their quality of life. In this sense, it is important to emphasize that all investments in infrastructure and technological developments in smart cities will be wasted if the citizens for whom they have been created eventually do not use them for whatever reason. To avoid these kinds of problems, the citizen's level of adaptation to the technologies should be evaluated. However, although much has been studied about new technological developments, studies to validate the actual impact and user acceptance of these technological models are much more limited. This paper tries to fill this gap presenting a new model of recommender system based in the most cited and used model in the scientific and academic literature: the Technology Acceptance Model (TAM) and using the most cited and agreed upon criteria in the existing literature. To accomplish the objective, this study in-troduces a novel recommender system that utilizes a fuzzy 2-tuple linguistic model in conjunction with the analytic hierarchy process (AHP) method to prioritize and personalize the relationship between tourists and smart cities. The methodology proposed in this paper was tested and validated in a case of study through different clusters to derive global recommendations tailored to each specific cluster. The main findings reveal that the use of technology is closely linked to the ability to enjoy personalized experiences in the realm of Smart Cities and Smart Tourism. As future works, authors recommend extending the recommender system model to any cluster of tourists using the proposed methodology and evaluate also other kind of disruptive technologies such as artificial intelligence (AI) in supporting the citizens.
... Analisa dan Evaluasi kegunaan aplikasi Payo Kepasar dengan SUS dan UMUX-Lite. Setelah jawaban responden kuesioner diperoleh selanjutnya melakukan rekapitulasi dan analisis tingkat kegunaan aplikasi Payo Kepasar dengan metode System Usability Scale (SUS) yakni[13]: a. Untuk pertanyaan bernomor Ganjil : skor yang diberikan responden kurangi 1 ('X' -1)[14] b. Untuk pertanyaan bernomor Genap : Angka 5 dikurangi dengan skor yang diberikan responden. ...
Perdagangan elektronik (e-commerce) melibatkan penggunaan Internet, World Wide Web (Web), dan aplikasi seluler serta peramban yang berjalan di perangkat seluler untuk melakukan transaksi bisnis dengan kemajuan teknologi Internet memungkinkan perusahaan untuk langsung menjual produk ke konsumen melalui platform e-commerce, guna meningkatkan penjualan adapun prinsip utama desain E-commerce adalah kegunaan. Payo Kepasar merupakan salah satu bentuk e-commerce guna memudahkan masyarakat dalam memperoleh sayur mayur di pasar tradisional saat Pemberlakuan Pembatasan Kegiatan Masyarakat (PPKM) di tengah pandemi Covid-19 di kota Jambi yang tersedia dalam bentuk aplikasi berbasis android maupun berbasis website sejak Agustus 2019 dan dirancang oleh Muhammad Khatami. Untuk menjamin bahwa aplikasi Payo Kepasar telah sesuai dengan kebutuhan masyarakat kota Jambi, maka perlu dilakukan evaluasi tingkat kegunaan (usability) agar konten yang disajikan sesuai dengan harapan pengguna. Untuk mengetahui tingkat kegunaan (usability) aplikasi serta situs website Payo Kepasar dihitung dengan menggunakan metode System Usability Scale (SUS) dan Usability Metric for User Experience-Lite (UMUX-Lite), sebab SUS dan UMUX-Lite telah menjadi standar kuesioner yang paling banyak digunakan peneliti dan praktisi untuk penilaian tingkat kegunaan (usability).
... Meanwhile, the correlation between AU and PEOU is much stronger than that of AU and PU. Lah et al. (2020) also found the correlations between PEOU and other measures of perceived usability tended to be significantly stronger than those with PU. In fact, many studies show both PEOU and PU directly affect teachers' actual use of technology. ...
This paper is designed to explore the current status of Chinese EFL teachers’ use of digital resources in doing research and its influential factors. It classifies digital resources into six types aligned to the research process and uses the revised TAM to find out and explain its influential factors. A total of 180 teachers were investigated via the online questionnaire. The results show the Chinese EFL teachers’ use of six-type resources is at an intermediate low level. Among them, the type of resources for getting data analysis software is most frequently used; The type of resources for managing literature is least frequently used. However, teachers’ intention to use the resources is relatively high. Frequency of doing research, facilitating condition (FC), and perceived ease of use of resources (PEOU) are the direct predictive factors for EFL teachers’ actual use of digital resources (AU). Self-efficacy in using resources, ICT skill, and FC are significant direct predictors of PEOU, and they are indirect predictors of AU.
... [18][19][20]). Furthermore, TAM has been widely utilized in various application areas to enhance its explanatory power and validity [21,22]. ...
... More importantly, they enable the data to be used for its fundamental purpose-enabling decision making. It has been shown that reduced cognitive load is vital for information processing, and ease-of-use is a crucial factor influencing human interaction with technology (Lah et al., 2020;Castro-Alonso et al., 2021). ...
The potential of big data capabilities to transform and understand global agricultural and biological systems often relies on data from different sources that must be considered together or aggregated to provide insights. The value of data is however not only in its collection and storage, but largely in its re-use. Big data storage repositories are not enough when we consider a world brimming with escalating volumes of data, here we need to consider innovative systems and tools which address data harmonization and standardization and importantly, ones that can bridge the gap between science and end users. In this paper, we will demonstrate how CGIAR (including the Alliance of Bioversity International and CIAT) develops a culture of co-operation and collaboration among custodians of agrobiodiversity data, as well as new directions for big data. CGIAR first launched the Platform for Big Data in Agriculture to enhance the development and maintenance of its data. This helped establish workflows of cross-platform synthesis, annotate and apply the lessons learnt. The Platform then built GARDIAN (Global Agricultural Research Data Innovation and Acceleration Network)—a digital tool that harvests from ∼40 separate open data and publication repositories that 15 CGIAR centres have used for data synthesis. While there have been significant advances in big data management and storage, we also identify the gaps to improve use, and the re-use of data in order to reveal its added value in decision making.
... Perceived Usefulness [102] The degree to which a person believes that use of a particular system would enhance his or her job performance ...
Among the plethora of instruments present in healthcare environments, the hospital bed is undoubtedly one of the most important for patients and caregivers. However, their design usually follows a top-down approach without considering end-users opinions and desires. Exploiting Human-centered design (HCD) permits these users to have a substantial role in the final product outcome. This study aims to empower caregivers to express their opinion about the hospital bed using a qualitative approach. For a holistic vision, we conducted six focus groups and six semi-structured interviews with nurses, nursing students, social-health operators and physiotherapists belonging to many healthcare situations. We then used thematic analysis to extract the themes that participants faced during the procedures, providing a comprehensive guide to designing the future electrical medical bed. These work results could also help overcome many issues that caregivers face during their everyday working life. Moreover, we identified the User Experience features that could represent the essential elements to consider.
... The above concepts are quite close to the concept of usability [30], which in turn is familiar from user interface design and the development of digital services. Usability of digital services refers specifically to the usability of software with dimensions like efficiency, engagement, error tolerance and being easy to learn [31]. ...
Along with the digitalisation of societies and services, the accessibility of digital content has become the focus of attention. However, emphasis has been on technical accessibility, ignoring the large number of people suffering from cognitive challenges that are expected to increase as the population ages. The purpose of this study is to demonstrate that the problem affects not only the elderly but also the young. Utilising multivariate methods and a data set of 14,892 young Finns, the study examines the impact of educational disparities on young people's digital usage and skills. It is observed that the level of education, the form of education and the regularity of the education path are related to differences in digital abilities of young people. Based on the results, the risks of being excluded from digital inclusion accumulate among adolescents for the youngest, but especially for those with a wide range of language, learning or motivational difficulties that manifest as delays in education path. As is known from previous research, such differences are expected to be reinforced in later life, threatening to become rather determinative.
... In accordance with information theory (Cole et al., 2002), visualizations can be helpful for schematizing relationships and aiding pattern recognition, thus optimizing the cognitive load of the user (Sweller et al., 2019). It was shown that reduced cognitive load (Hu et al., 2017;Sweller et al., 2019;Castro-Alonso et al., 2021) is vital for information processing, and ease-of-use is a crucial factor influencing human interaction with technology (Hassenzahl et al., 2010;Venkatesh et al., 2016;Lah et al., 2020). ...
Scholarly knowledge graphs provide researchers with a novel modality of information retrieval, and their wider use in academia is beneficial for the digitalization of published works and the development of scholarly communication. To increase the acceptance of scholarly knowledge graphs, we present a dashboard, which visualizes the research contributions on an educational science topic in the frame of the Open Research Knowledge Graph (ORKG). As dashboards are created at the intersection of computer science, graphic design, and human-technology interaction, we used these three perspectives to develop a multi-relational visualization tool aimed at improving the user experience. According to preliminary results of the user evaluation survey, the dashboard was perceived as more appealing than the baseline ORKG-powered interface. Our findings can be used for the development of scholarly knowledge graph-powered dashboards in different domains, thus facilitating acceptance of these novel instruments by research communities and increasing versatility in scholarly communication.
... Researchers identified the need to measure behavioral users' intentions, perceived usefulness, and perceived ease of use when implementing technology (Ajzen & Fishbein, 1980;Sarver, 1983). Numerous researchers further reinforced the viability of the TAM in accurately identifying users' acceptance and use of technology (Lah et al., 2020). Leaders can integrate technology in a manner that is in line with their business model using the TAM and extending the TAM to further enhance business capabilities. ...
Improperly planned capital investments in new technologies could result in lost productivity for e-procurement and supply chain managers in the e-commerce industries. Grounded in the technology acceptance model, the qualitative multiple case study design was used to explore strategies business managers use to adopt technology innovations to improve product marketing and profitability. Participants were five supply chain business managers and owners from five separate organizations in the southwestern region of the United States. Data were collected via responses submitted through Google Forms and emails, written logs, telephone calls, and recorded audio. Thematic analysis resulted in two key themes: perceived ease of use and perceived usefulness. A key recommendation is implementing strategies that involve agile and lean thinking processes, artificial intelligence, and automation. Implications for positive social change include the potential to improve the customer experience while contributing to a sustainable environment.
... This definition relates directly to user and business requirements as effectiveness refers to success in achieving goals; efficiency refers to not wasting time, and satisfaction referes to a willingness to use the system. This new construct has been proposed because various studies outside the UTAUT literature have reported that factors of this kind have a significant influence on users' acceptance of educational technologies (Holden & Rada, 2011;Juarez Collazo et al., 2014;Lah et al., 2020;Lin, 2013;Tsakonas & Papatheodorou, 2008). ...
The United Theory of Acceptance and Use of Technology (UTAUT) model has been widely used to study new technological systems and has proven to be a robust theoretical framework for predicting system intentional use. Many of the UTAUT studies have focused on educational technologies like learning management systems, mobile learning, instructional devices, online collaboration tools and educational services. This paper reviews previous work done on the UTAUT model and proposes an extended model to study educational technology acceptance by introducing additional constructs like usability, learnability and attitude.
This study aims to assess the impact of neighborhood effects (NEs) on the adoption of digital payments (DP), focusing specifically on the northern mountainous regions of Vietnam. The innovation of our research lies in integrating the UTAUT2 model with the theory of herd behavior (THB). By analyzing interview data from 427 respondents using structural equation modeling (SEM), we discovered that NEs exert both a direct negative and an indirect positive influence on the adoption of DP. However, the overall impact is significantly positive. This underscores the crucial role of information shared by relatives, friends and colleagues in promoting awareness of DP. The findings highlight the importance of neighbors in enhancing citizens’ awareness and usage of DP in their daily lives. The article also discusses implications and suggests directions for future research.
The use of technology eases and makes teaching learning process effective and efficient. Thus far, researches on technology acceptance in higher education has been widely conducted; however, only few has focused on vocational high school teachers, especially in the field of business. This survey research aims to analyze the influences of technology acceptance variables: Perceived of usefulness (PU) and Perceived of Ease of Use (PEU) towards two other acceptance variables: among vocational high school (SMK) teachers in the Business and Management field in Surakarta. The population of this research was all vocational high school teachers in the Business and Management field in Surakarta. At the end of the research, there were 172 responses obtained from the teachers involved. Partial Least Squares (PLS) on Structural Equation Modelling (SEM) was used to test the research hypotheses. The empirical findings show that there are significant influences; particularly on PEU and PU, as well as PEU and PU on BI. Furthermore, BI is also proven to have a significant influence on AU. The findings are expected to provide deeper insights into the factors influencing technology adoption by teachers in business education. Moreover, the implication is foreseen to contribute on the development of more effective and efficient business education through the use of digital technology.
The chapter explores the broad implications of automation and artificial intelligence (AI) on e-commerce logistics. It discusses how these technologies are altering supply chains through automated warehousing, order processing, and artificial intelligence-driven demand forecasting and personalization. Despite their transformational potential, the chapter highlights issues such as data security, integration, workforce adaptation, and ethical considerations. It also includes real-world case studies demonstrating the successes of businesses that have embraced automation and artificial intelligence, as well as glimpses into the future of e-commerce logistics, stressing emerging technology and projected trends. Finally, the chapter serves as a comprehensive guide for businesses wishing to harness the potential of automation and artificial intelligence in the ever-changing world of e-commerce logistics.
The chapter traces the evolution of the Technology Acceptance Model (TAM) and underscores its consistent efficacy in predicting user acceptance across diverse technologies over more than three decades. Exploring beyond TAM research, alternative approaches aim to enrich our understanding of primary dependent constructs, specifically behavioural intentions and the actual behaviour (i.e. adoption) of technology. The chapter investigates the proliferation of selected TAM-related behavioural intention models and presents several integrated theoretical approaches. Additionally, it provides a chronological account of the era, illustrating interconnected relationships among the most influential theories and models in the field. Building on three dimensions of influence, this work systematically categorizes additional determinants of behavioural intention derived from various TAM extensions (“TAM++”). Notably, these new variables manifest and align with trends in the evolving landscape of emerging technologies, emphasizing TAM as a powerful and extensively validated theory. Its versatility is apparent across a broad spectrum of technological solutions, systems, environments, tools, applications, services and devices, as exemplified by numerous real-world applications explored in the chapter. TAM establishes itself as a simple and practical tool for delineating the determinants of technology adoption, proving effective even when integrated with well-established theories from related disciplines, thus spanning diverse multidisciplinary domains.
The intricateness and dynamism of any software development projects are viewed as prevailing attributes that affect its outcome. However, explicit utilization of recognized software development principles and standards ensures higher possibility of achieving greater success. This paper presents the results of the efforts in the design and development of Comprehensive Pili (Canarium ovatum Engl.) Information System and Marketplace particularly the user experiences (UX). This project was funded by DOST with the intention of stablishing a dynamic web of knowledge for Pili industry, thus addressing the dearth of accessible Pili information. The development process follows strict adherence to PiliIS and Marketplace Software Development Paradigm - researchers made software development procedures adopting Agile software development concepts. These paradigm ensures explicit compliance to ISO/IEC 25010 standards. Internal evaluation team was employed to examine the user interface judging its compliance with the recognized usability principles. Diverse group of stakeholders evaluated the system following Jakob Nielsen’s principles for interaction design. Complying to these highly recognized software quality standards and adhering to the researchers’ paradigm, the developed software marked by stakeholders as “highly usable”, “highly acceptable” and “highly reliable” with average ratings of 94.65%, 93.71% and 93.53%, respectively. Additionally, Philippine Pili Industry League, Inc. (PhilPili) rated the system “excellent”. Given the limited study on the development and evaluation of information system designed to support the marginalized sectors in the country, this study provides significant insights on the software quality principles and competencies as universal standards that boost favorable customer satisfaction and optimum user experience.
Usability and user experience (UX) are important concepts in the design and evaluation of products or systems intended for human use. This chapter introduces the fundamentals of design for usability and UX, focusing on the application of science, art, and craft to their principled design. It reviews the major methods of usability assessment, focusing on usability testing. The concept of UX casts a broad net over all of the experiential aspects of use, primarily subjective experience. User-centered design and design thinking are methods used to produce initial designs, after which they typically use iteration for design improvement. Service design is a relatively new area of design for usability and UX practitioners. The fundamental goal of usability testing is to help developers produce more usable products. The primary activity in diagnostic problem discovery tests is the discovery, prioritization, and resolution of usability problems.
Substantial research has been performed on Building Information Modeling (BIM) in various topics, for instance, the use and benefit of BIM in design, construction, sustainable environment building, and Facility assets over the past several years. Although there are various studies on these topics, Building Information Modeling (BIM) awareness through facilities management is still relatively poor. The researcher's interest is increased in BIM study is based heavily upon the perception that it can facilitate the exchange and reuse of information during various project phases. This property and others can be used in the Iraqi Construction industry to motivate the government to eliminate the change resistance to use innovations in the Iraqi construction industry. Even though many scholars and practitioners agree on the potential application and benefits of BIM in construction/ Facilities Management FM, it is yet unknown why BIM is used and what criteria facilitate BIM achievement in O&M. As long as the actual usage and acceptance of BIM in the Operation and Maintenance phase is still a central issue in practice, therefor depending on the extensive well ground literature review a conceptual acceptance model is proposed in this paper by applying the technology acceptance theories such as Technology Acceptance Model (TAM) and Task Technology Fit (TTF) to explore how the extent of facilities management staff going to accept and adopt the new international technology (BIM).
Quality and time are uncompromisable performance indicators of the physical exhibition production process. Design changes during the installation phase of the process, which arise from poor design representations, cause expensive rework and therefore cost increase. Since non-AEC (architecture/engineering) construction) decision-makers can only partially imagine the outcome through traditional 2D and 3D design representations; we proposed a model incorporating virtual reality technologies to solve design miscommunications and improve stakeholder participation. We adopted a qualitative method and tested the model through case studies with five expert museum professionals. We confirm that VR enhances the design communication among the contributing professionals and, unlike other traditional methods, makes it possible to present multimedia, lighting, sound effects, and other digital elements throughout the exhibition production process. Findings indicate that VR has the potential for significant cost reductions and quality improvement in the construction phase of the exhibitions.
Visualization influences the collaborative design process and design thinking (DT) education outcomes. Accordingly, digital visual collaborative learning (DVCL) technology has great potential to facilitate students’ remote collaborative design experiences. This study investigates the fitness of DVCL for DT learning activities and the factors influencing students’ continued use of the technology with a model integrating technology acceptance model (TAM) and design task-technology fit (DTTF). The research findings demonstrate that the perceived usefulness of the technology is the primary direct factor influencing the continuance intention toward the technology. In addition, the DTTF mediated by perceived usefulness and ease of use significantly affected the intention to continue using the technology. The visual characteristics of the technology influenced the DTTF in the collaborative design environment. This study highlights visualization as a critical feature of collaborative learning in DT education and provides implications from both theoretical and practical perspectives.
The journey of a prosthetic user is characterized by the opportunities and the limitations of a device that should enable activities of daily living (ADL). In particular, experiencing a bionic hand as a functional (and, advantageously, embodied) limb constitutes the premise for promoting the practice in using the device, mitigating the risk of its abandonment. In order to achieve such a result, different aspects need to be considered for making the artificial limb an effective solution to accomplish activities of daily living. According to such a perspective, this review aims at presenting the current issues and at envisioning the upcoming breakthroughs in upper limb prosthetic devices. We first define the sources of input and feedback involved in the system control (at user-level and device-level), alongside the related algorithms used in signal analysis. Moreover, the paper focuses on the user-centered design challenges and strategies that guide the implementation of novel solutions in this area in terms of technology acceptance, embodiment, and, in general, human-machine integration based on co-adaptive processes. We here provide the readers (belonging to the target communities of researchers, designers, developers, clinicians, industrial stakeholders, and end-users) with an overview of the state-of-the-art and the potential innovations in bionic hands features, hopefully promoting interdisciplinary efforts for solving current issues of ULPs. The integration of different perspectives should be the premise to a transdisciplinary intertwining leading to a truly holistic comprehension and improvement of the bionic hands design. Overall, this paper aims to move the boundaries in prosthetic innovation beyond the development of a tool and towards the engineering of human-centered artificial limbs.
It was found that the implementation of quality education is a reaction to a number of changes in circumstances related to changes in context related to the distribution of student profiles, the internalization of higher education and the job market, the emergence of the workforce and students, dissatisfaction from workers and students, pressure due to limited funds, and demands for accountability to institutions. Sebelas April University has implemented Information Systems Internal Quality Assurance Systems to avoid these problems. The purpose of this study was to analyze and determine the Usability factor in Information Systems Internal Quality Assurance Systems at Sebelas April University, which was recently formed. The measurement model used is the Usability Model from Nielsen's. The results obtained from the evaluation of the Internal Quality Assurance System for Information Systems that refer to the Nielsen Model variables (Learnability, Efficiency, Memorability, Errors, and User Satisfaction) are considered adequate by the user, with a significance level used of 0.1 (10%), total 743 respondents and 88 samples, resulting in a continuum line value for each variable, namely the Learnability variable 1730 (enough), the Efficiency variable 1620 (good), the memory variable 1107 (enough), the Error variable 1380 (enough) and the Satisfaction variable 1441 (enough). The results of this evaluation can be considered to be used as a guide in the framework of the Information System development process.
The significant benefits of using uncrewed aircraft systems for industrial inspections are now recognized, the implementation of these commercial drones is increasing exponentially. However, there is a lack of tools to explain and understand the remote pilot's risk perception. The established VMUTES model measures a remote pilot's behavioral intention to use drones for commercial purposes, thus, filling a critical gap in the literature and providing a tool to measure behavioral intent. However, in the established VMUTES model, perceived risk was not explored as the outcome variable. A remote pilot's perception of risk is critical for safe operations. This study modifies the VMUTES model using the original VMUTES dataset to explain factors contributing to a remote pilot's risk perception. The theory of planned behavior, the technology acceptance model, and the original VMUTES model support the foundational theory for this study. Good model fit was achieved with the confirmatory factor analysis rather than the full structural equation model; however, the efficacy of the VMUTES was confirmed. Surprisingly, the results indicated a significant negative relationship between Perceived Usefulness and Perceived Risk suggesting remote pilots' perceptions of usefulness impact risk perception.
Telecare has the potential to tackle emerging problems in public healthcare resulting from the graying of population. This study aims to investigate the personal, social and regulatory factors associated with the telecare acceptance for older adults in Hong Kong. A structured questionnaire survey was administered to 110 seniors aged 60–80 years. The results revealed that older people prefer to use the products that they are familiar with. Seniors of very poor economic status are likely to use telecare service. Family and friends support, self-satisfaction, and social relationship are critical factors which positively influence telecare usage of older adults via influencing their perceived ease of use and perceived usefulness. Regulatory factor of perceived incentives positively associates with the attitude of older people toward telecare service, and in turn facilitates telecare usage. This is the first study which stressed that public regulations, such as incentive interventions, should be emphasized in promoting telecare acceptance.
News aggregator apps have become one of the major channels for acquiring information for the Chinese. The number of users is increasing but literature on the user experience (UX) of these apps from the perspective of human-computer interaction remains dearth. In study 1, a completely randomized between-participant experiment with prior experience of news aggregator apps (with prior experience vs. without prior experience) as the independent variable was conducted to investigate its effects on the UX of news aggregator app’s functions, including browsing news, searching news, watching short videos, interacting with news, personalized page setup and online shopping. UX was operationalized as perceived usefulness, ease of use, and satisfaction. Results indicated that the experienced users perceived lower levels of usefulness when they used the app to browse news, watch short videos, interact with news and set up the page than inexperienced users. Experienced users also perceived lower levels of ease of use when watching short videos. In addition, veteran users reported lower satisfaction levels when browsing the news. To explore the reason behind this phenomenon, a sentiment analysis (study 2) was conducted on the reports submitted by participants in the experienced group during the training sessions. It was found that the experienced group overall reported negative sentiment scores, and the results indicated positive trends of correlation between the sentiment score and the UX. The current research provides a more nuanced understanding of the effect of prior experience on the UX of news aggregator apps.
Information technologies exist to enable us to either do things we have not done before or do familiar things more efficiently. Metaverse (i.e. extended reality: XR) enables novel forms of engrossing telepresence, but it also may make mundate tasks more effortless. Such technologies increasingly facilitate our work, education, healthcare, consumption and entertainment; however, at the same time, metaverse bring a host of challenges. Therefore, we pose the question whether XR technologies, specifically Augmented Reality (AR) and Virtual Reality (VR), either increase or decrease the difficulties of carrying out everyday tasks. In the current study we conducted a 2 (AR: with vs. without) × 2 (VR: with vs. without) between-subject experiment where participants faced a shopping-related task (including navigating, movement, hand-interaction, information processing, information searching, storing, decision making, and simple calculation) to examine a proposed series of hypotheses. The NASA Task Load Index (NASA-TLX) was used to measure subjective workload when using an XR-mediated information system including six sub-dimensions of frustration, performance, effort, physical, mental, and temporal demand . The findings indicate that AR was significantly associated with overall workload, especially mental demand and effort, while VR had no significant effect on any workload sub-dimensions. There was a significant interaction effect between AR and VR on physical demand, effort, and overall workload. The results imply that the resources and cost of operating XR-mediated realities are different and higher than physical reality.
The intricateness and dynamism of any software development project are viewed as prevailing attributes that affect its outcome. However, explicit utilization of recognized software development principles and standards ensures a higher possibility of achieving greater success. This paper presents the results of the efforts in the design and development of Comprehensive Pili (Canarium ovatum Engl.) Information System and Marketplace particularly the user experiences. This project was funded by the Department of Science and Technology (DOST) with the intention of establishing a dynamic web of knowledge for the Pili industry, thus addressing the dearth of accessible Pili information. The development process follows strict adherence to PiliIS and Marketplace Software Development Paradigm - researchers made software development procedures adopting the Agile software development concepts. This paradigm ensures explicit compliance to ISO/IEC 25010 standards. An internal evaluation team was employed to examine the user interface judging its compliance with the recognized usability principles. A diverse group of stakeholders evaluated the system following Jakob Nielsen’s principles for interaction design. Complying to these highly recognized software quality standards and adhering to the researchers’ paradigm, the developed software was marked by stakeholders as “highly usable”, “highly acceptable” and “highly reliable” with average ratings of 94.65%, 93.71%, and 93.53%, respectively. Additionally, Philippine Pili Industry League, Inc. (PhilPili) rated the system as “excellent”. Given the limited study on the development and evaluation of information systems designed to support the marginalized sectors in the country, this study provides significant insights on the software quality principles and competencies as universal standards that boost favorable customer satisfaction and optimum user experience.
Background: Nigeria has one of the worst health and development profile globally. A weak health system, poor infrastructure and varied socio-cultural factors are cited as inhibitors to optimal health system performance and improved maternal and child health status. eHealth has become a major solution to closing these gaps in health care delivery in low- and middle-income countries (LMICs). This paper reports the use of satellite communication (SatCom) technology and existing 3G mobile network for providing video training (VTR) for health workers and improving digitization of healthcare data.
Objective: To evaluate whether the expected project outcomes that were achieved at end-line evaluation of 2019, were sustained 12 months after the project ended.
Methods: From March 2017 to March 2019, digital innovations including VTR and data digitization interventions were delivered in 62 healthcare facilities in Ondo State, southwest Nigeria, most of which lacked access to 3G mobile network. Data collection for the evaluation combined documents review with quantitative data extracted from health facility registers, and 24 most significant change stories to assess the longevity of outcomes and impacts of digital innovation in four domains of healthcare: use of eHealth technology for data management, utilization of health facilities by patients, standard of care, and staff attitude. Stories of most significant changes were audio-recorded, transcribed for analysis and categorised by the above domains to identify the most significant changes 12 months after project closedown.
Results: Findings showed that four project outcomes which were achieved at end-line evaluation were sustained 12 months after project closedown namely: staff motivation and satisfaction; increased staff confidence to perform healthcare roles; improved standard of healthcare delivery; and increased adoption of eHealth innovations beyond health sector. Conversely, an outcome that was reversed following discontinuation of SatCom from health facilities is availability of accurate and reliable data for decision-making.
Conclusion: Digital technology can have lasting impacts on health workers, patients and the health system, through improving data management for decision-making, standard of maternity service delivery, boosting attendance at health facilities and utilization of services. Locally driven investment is essential for ensuring long-term survival of eHealth projects to achieve SDGs in LMICs.
The purpose of this research was to investigate the effect of manipulating item formats for a revised version of the Technology Acceptance Model (TAM) questionnaire. The TAM has 12 items, six assessing perceived usefulness (PU) and six assessing perceived ease of use (PEU). Its original purpose was to quantify likelihood of technology acceptance, using an item format with end anchors of "Likely" on the left and "Unlikely" on the right, and seven response options (from left to right) of "Extremely, Quite, Slightly, Neither, Slightly, Quite, Extremely." To revise the TAM to measure user experience (UX), it is necessary to change the items from assessments of likelihood to agreement. In this study, 546 participants rated their experience using IBM Notes with one of four versions of the modified TAM created by crossing two independent variables: Response format (labels or numbers) and Response order (increasing from right-to-left or from left-to-right), with participants about evenly divided among the four formats. A check on ratings of overall experience showed no significant difference as a function of format group, with similar nonsignificant results for the overall TAM scores. An analysis of variance examining the main effects and interaction of the two independent variables (response format and order) on TAM scores was also nonsignificant at similar levels. Factor analyses for each version produced the same alignment of items with the PU and PEU factors, consistent with the item alignment in the original TAM research. Beta weights for regression models predicting likelihood-of-use and overall experience from PU and PEU ratings were very similar for all four versions. The results indicate that the item format differences did not lead to any important differences in the magnitude or structure of TAM measurement, but there were significantly more response errors when the magnitude of agreement increased from right to left.
In 2018, Noam Tractinsky published a provocative paper entitled, “The usability construct: A dead end?” He argued the following:
• Usability is an umbrella concept.
• There is a mismatch between the construct of usability and its empirical measurements.
• Scientific progress requires unbundling the usability construct and replacing it with well-defined constructs.
Tractinsky (2018) offered the Hirsch and Levin (1999) definition of an umbrella construct as a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (p. 200), noting that this diversity of phenomena makes it impossible to achieve a goal of unidimensional measurement.
Tractinsky’s (2018) paper is undoubtedly a valuable contribution to the literature of usability science. Although I do not agree with its premises or conclusions, I admire its construction and I learned a lot from reading it. I hope that it will lead to additional research that will improve the understanding of the construct of usability, as it has inspired the writing of this essay. Following are the reasons why I disagree with his arguments.
First, I question whether usability is truly an umbrella construct—a “broad concept or idea used loosely to encompass and account for a set of diverse phenomena” (Hirsch & Levin, 1999, p. 200), at least in the context of industrial usability testing. Structural analysis of objective and subjective data from industrial usability studies (Sauro & Lewis, 2009) has provided evidence consistent with an underlying construct of usability that can manifest itself through objective and subjective measurement. It seems plausible that when a system intended for human use has been properly designed, then the users of that system will complete tasks successfully and quickly, and will be sufficiently aware of this to experience, at a minimum, satisfaction as a consequence of perceived usability.
With regard to perceived usability, it now appears that reports of meaningful factor structure in the SUS may have been premature (Lewis & Sauro, 2009), with more recent analysis indicating a nuisance structure due to the mixed positive and negative tone of its items (Lewis & Sauro, 2017). Furthermore, the development of subscales developed using factor analysis does not preclude the calculation of an overall measure of perceived usability. Investigation of correlation and correspondence of three independently developed usability questionnaires (CSUQ, SUS, and UMUX) has provided compelling evidence that they are measuring the same underlying construct (Lewis 2018a, 2018c, 2018d). There is also evidence that this is the same underlying construct assessed by the PEOU component of the TAM (Lewis, 2018b).
So, rather than being a dead end, I believe the construct of usability has a bright future both in usability science (theory) and usability engineering (practice), either alone or as a fundamental part of the larger assessment of user experience. Any report of its death is an exaggeration.
The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. This review of the SUS covers its early history from inception in the 1980s through recent research and its future prospects. From relatively inauspicious beginnings, when its originator described it as a “quick and dirty usability scale,” it has proven to be quick but not “dirty.” It is likely that the SUS will continue to be a popular measurement of perceived usability for the foreseeable future. When researchers and practitioners need a measure of perceived usability, they should strongly consider using the SUS.
The primary purpose of this research was to investigate the relationship between two widely used questionnaires designed to measure perceived usability: the Computer System Usability Questionnaire (CSUQ) and the System Usability Scale (SUS). The correlation between concurrently collected CSUQ and SUS scores was 0.76 (over 50% shared variance). After converting CSUQ scores to a 0–100-point scale (to match the range of the SUS scores), there was a small but statistically significant difference between CSUQ and SUS means. Although this difference (just under 2 scale points out of a possible 100) was statistically significant, it did not appear to be practically significant. Although usability practitioners should be cautious pending additional independent replication, it appears that CSUQ scores, after conversion to a 0–100-point scale, can be interpreted with the Sauro–Lewis curved grading scale. As a secondary research goal, investigation of variations of the Usability Metric for User Experience (UMUX) replicated previous findings that the regression-adjusted version of the UMUX-LITE (UMUX-LITEr) had the closest correspondence with concurrently collected SUS scores. Thus, even though these three standardized questionnaires were independently developed and have different item content and formats, they largely appear to be measuring the same thing, presumably, perceived usability.
Usability Metric for User Experience (UMUX) and its shorter form variant UMUX-LITE are recent additions to standardized usability questionnaires. UMUX aims to measure perceived usability by employing fewer items that are in closer conformance with the ISO 9241 definition of usability, while UMUX-LITE conforms to the technology acceptance model (TAM). UMUX has been criticized regarding its reliability, validity, and sensitivity, but these criticisms are mostly based on reported findings associated with the data collected by the developer of the questionnaire.
Our study re-evaluates the UMUX and UMUX-LITE scales using psychometric methods with data sets acquired through two usability evaluation studies: an online word processor evaluation survey (n = 405) and a web-based mind map software evaluation survey for three applications (n = 151).
Data sets yielded similar results for indicators of reliability. Both UMUX and UMUX-LITE items were sensitive to the software when the scores for the evaluated software were not very close, but we could not detect a significant difference between the software when the scores were closer.
UMUX and UMUX-LITE items were also sensitive to users’ level of experience with the software evaluated in this study. Neither of the scales was found to be sensitive to the participants’ age, gender, or whether they were native English speakers. The scales significantly correlated with the System Usability Scale (SUS) and the Computer System Usability Questionnaire (CSUQ), indicating their concurrent validity. The parallel analysis of principal components of UMUX pointed out a single latent variable, which was confirmed through a factor analysis, that showed the data fits better to a single-dimension factor structure.
The purpose of this research was to investigate various measurements of perceived usability, in particular, to assess (a) whether a regression formula developed previously to bring Usability Metric for User Experience LITE (UMUX-LITE) scores into correspondence with System Usability Scale (SUS) scores would continue to do so accurately with an independent set of data; (b) whether additional items covering concepts such as findability, reliability, responsiveness, perceived use by others, effectiveness, and visual appeal would be redundant with the construct of perceived usability or would align with other potential constructs; and (c) the dimensionality of the SUS as a function of self-reported frequency of use and expertise. Given the broad use of and emerging interpretative norms for the SUS, it was encouraging that the regression equation for the UMUX-LITE worked well with this independent set of data, although there is still a need to investigate its efficacy with a broader set of products and methods. Results from a series of principal components analyses indicated that most of the additional concepts, such as findability, familiarity, efficiency, control, and visual appeal covered the same statistical ground as the other more standard metrics for perceived usability. Two of the other items (Reliable and Responsive) made up a reliable construct named System Quality. None of the structural analyses of the SUS as a function of frequency of use or self-reported expertise produced the expected components, indicating the need for additional research in this area and a need to be cautious when using the Usable and Learnable components described in previous research.
Nowadays, practitioners extensively apply quick and reliable scales of user satisfaction as part of their user experience (UX) analyses to obtain well-founded measures of user satisfaction within time and budget constraints. However, in the human-computer interaction (HCI) literature the relationship between the outcomes of standardized satisfaction scales and the amount of product usage has been only marginally explored. The few studies that have investigated this relationship have typically shown that users who have interacted more with a product have higher satisfaction. The purpose of this paper was to systematically analyze the variation in outcomes of three standardized user satisfaction scales (SUS, UMUX and UMUX-LITE) when completed by users who had spent different amounts of time with a website. In two studies, the amount of interaction was manipulated to assess its effect on user satisfaction. Measurements of the three scales were strongly correlated and their outcomes were significantly affected by the amount of interaction time. Notably, the SUS acted as a unidimensional scale when administered to people who had less product experience, but was bidimensional when administered to users with more experience. We replicated previous findings of similar magnitudes for the SUS and UMUX-LITE (after adjustment), but did not observe the previously reported similarities of magnitude for the SUS and the UMUX. Our results strongly encourage further research to analyze the relationships of the three scales with levels of product exposure. We also provide recommendations for practitioners and researchers in the use of the questionnaires.
The System Usability Scale (SUS) is a widely adopted and studied questionnaire for usability evaluation. It is technology independent and has been used to evaluate the perceived usability of a broad range of products, including hardware, software, and websites. In this article we present a Slovene translation of the SUS (the SUS-SI) along with the procedure used in its translation and psychometric evaluation. The results indicated that the SUS-SI has properties similar to the English version. Slovene usability practitioners should be able to use the SUS-SI with confidence when conducting user research.
Over the recent years, the notion of a non-instrumental, hedonic quality of interactive products received growing interest. Based on a review of 151 publications, we summarize more than ten years research on the hedonic to provide an overview of definitions, assessment tools, antecedents, consequences, and correlates. We highlight a number of contributions, such as introducing experiential value to the practice of technology design and a better prediction of overall quality judgments and product acceptance. In addition, we suggest a number of areas for future research, such as providing richer, more nuanced models and tools for quantitative and qualitative analysis, more research on the consequences of using hedonic products and a better understanding of when the hedonic plays a role and when not.
Over the recent years, the notion of a non-instrumental, hedonic quality of interactive products received growing interest. Based on a review of 151 publications, we summarize more than ten years research on the hedonic to provide an overview of definitions, assessment tools, antecedents, consequences, and correlates. We highlight a number of contributions, such as introducing experiential value to the practice of technology design and a better prediction of overall quality judgments and product acceptance. In addition, we suggest a number of areas for future research, such as providing richer, more nuanced models and tools for quantitative and qualitative analysis, more research on the consequences of using hedonic products and a better understanding of when the hedonic plays a role and when not.
In this paper we present the UMUX-LITE, a two-item questionnaire based on the Usability Metric for User Experience (UMUX) [6]. The UMUX-LITE items are This system's capabilities meet my requirements and This system is easy to use." Data from two independent surveys demonstrated adequate psychometric quality of the questionnaire. Estimates of reliability were .82 and .83 -- excellent for a two-item instrument. Concurrent validity was also high, with significant correlation with the SUS (.81, .81) and with likelihood-to-recommend (LTR) scores (.74, .73). The scores were sensitive to respondents' frequency-of-use. UMUX-LITE score means were slightly lower than those for the SUS, but easily adjusted using linear regression to match the SUS scores. Due to its parsimony (two items), reliability, validity, structural basis (usefulness and usability) and, after applying the corrective regression formula, its correspondence to SUS scores, the UMUX-LITE appears to be a promising alternative to the SUS when it is not desirable to use a 10-item instrument.
This study is a part of a research effort to develop the Questionnaire for User Interface Satisfaction (QUIS). Participants, 150 PC user group members, rated familiar software products. Two pairs of software categories were compared: 1) software that was liked and disliked, and 2) a standard command line system (CLS) and a menu driven application (MDA). The reliability of the questionnaire was high, Cronbach's alpha=.94. The overall reaction ratings yielded significantly higher ratings for liked software and MDA over disliked software and a CLS, respectively. Frequent and sophisticated PC users rated MDA more satisfying, powerful and flexible than CLS. Future applications of the QUIS on computers are discussed.
Usability evaluators used an 18-item, post-study questionnaire in three related usability tests. I conducted an exploratory factor analysis to investigate statistical justification to combine items into subscales. The factor analysis indicated that three factors accounted for 87 percent of the total variance. Coefficient alpha analyses showed that the reliability of the overall summative scale was .97, and ranged from .91 to .96 for the three subscales. In the sensitivity analyses, the overall scale and all three subscales detected significant differences among the user groups; and one subscale indicated a significant system effect. Correlation analyses support the validity of the scales. The overall scale correlated highly with the sum of the After-Scenario Questionnaire ratings that participants gave after each scenario. The overall scale also correlated moderately with the percentage of successful scenario completion. These results are consistent with the hypothesis that these alternative measurements tap into a common underlying construct. This construct is probably usability, based on the content of the questionnaire items and the measurement context.
Computer systems cannot improve organizational performance if they aren't used. Unfortunately, resistance to end-user systems by managers and professionals is a widespread problem. To better predict, explain, and increase user acceptance, we need to better understand why people accept or reject computers. This research addresses the ability to predict peoples' computer acceptance from a measure of their intentions, and the ability to explain their intentions in terms of their attitudes, subjective norms, perceived usefulness, perceived ease of use, and related variables. In a longitudinal study of 107 users, intentions to use a specific system, measured after a one-hour introduction to the system, were correlated 0.35 with system use 14 weeks later. The intention-usage correlation was 0.63 at the end of this time period. Perceived usefulness strongly influenced peoples' intentions, explaining more than half of the variance in intentions at the end of 14 weeks. Perceived ease of use had a small but significant effect on intentions as well, although this effect subsided over time. Attitudes only partially mediated the effects of these beliefs on intentions. Subjective norms had no effect on intentions. These results suggest the possibility of simple but powerful models of the determinants of user acceptance, with practical value for evaluating systems and guiding managerial interventions aimed at reducing the problem of underutilized computer technology.
Correlations between prototypical usability metrics from 90 distinct usability tests were strong when measured at the task-level (r between .44 and .60). Using test-level satisfaction ratings instead of task-level ratings attenuated the correlations (r between .16 and .24). The method of aggregating data from a usability test had a significant effect on the magnitude of the resulting correlations. The results of principal components and factor analyses on the prototypical usability metrics provided evidence for an underlying construct of general usability with objective and subjective factors. Author Keywords
When designing questionnaires there is a tradition of including items with both positive and negative wording to minimize acquiescence and extreme response biases. Two disadvantages of this approach are respondents accidentally agreeing with negative items (mistakes) and researchers forgetting to reverse the scales (miscoding).
The original System Usability Scale (SUS) and an all positively worded version were administered in two experiments (n=161 and n=213) across eleven websites. There was no evidence for differences in the response biases between the different versions. A review of 27 SUS datasets found 3 (11%) were miscoded by researchers and 21 out of 158 questionnaires (13%) contained mistakes from users.
We found no evidence that the purported advantages of including negative and positive items in usability questionnaires outweigh the disadvantages of mistakes and miscoding. It is recommended that researchers using the standard SUS verify the proper coding of scores and include procedural steps to ensure error-free completion of the SUS by users.
Researchers can use the all positive version with confidence because respondents are less likely to make mistakes when responding, researchers are less likely to make errors in coding, and the scores will be similar to the standard SUS.
The Usability Metric for User Experience (UMUX) is a four-item Likert scale used for the subjective assessment of an application’s
perceived usability. It is designed to provide results similar to those obtained with the 10-item System Usability Scale,
and is organized around the ISO 9241–11 definition of usability. A pilot version was assembled from candidate items, which
was then tested alongside the System Usability Scale during usability testing. It was shown that the two scales correlate
well, are reliable, and both align on one underlying usability factor. In addition, the Usability Metric for User Experience
is compact enough to serve as a usability module in a broader user experience metric.
Valid measurement scales for predicting user acceptance of computers are in short supply. Most subjective measures used in practice are unvalidated, and their relationship to system usage is unknown. The present research develops and validates new scales for two specific variables, perceived usefulness and perceived ease of use, which are hypothesized to be fundamental determinants of user acceptance. Definitions for these two variables were used to develop scale items that were pretested for content validity and then tested for reliability and construct validity in two studies involving a total of 152 users and four application programs. The measures were refined and streamlined, resulting in two six-item scales with reliabilities of .98 for usefulness and .94 for ease of use. The scales exhibited high convergent, discriminant, and factorial validity. Perceived usefulness was significantly correlated with both self-reported current usage (r=.63, Study 1) and self-predicted future usage (r =.85, Study 2). Perceived ease of use was also significantly correlated with current usage (r=.45, Study 1) and future usage (r=.59, Study 2). In both studies, usefulness had a significantly greater correlation with usage behavior than did ease of use. Regression analyses suggest that perceived ease of use may actually be a causal antecedent to perceived usefulness, as opposed to a parallel, direct determinant of system usage. Implications are drawn for future research on user acceptance.
A recent contribution to the ongoing debate concerning the concept of usability and its measures proposed that usability reached a dead end – i.e. a construct unable to provide stable results and to unify scientific knowledge. Extensive commentaries rejected the conclusion that researchers need to look for alternative constructs to measure the quality of interaction. Nevertheless, several practitioners involved in this international debate asked for a constructive way to move forward the usability practice. In fact, two key issues of the usability field were identified in this debate: (i) knowledge fragmentation in the scientific community, and (ii) the unstable relationship among the usability metrics. We recognise both the importance and impact of these key issues, although, in line with others, we may not agree with the conclusion that the usability is a dead end. Under the light of the international debate, this work discusses the strengths and weaknesses of usability construct and its application. Our discussion focuses on identifying alternative explanations to the issues and to suggest mitigation strategies, which may be considered the starting point to move forward the usability field. However, scientific community actions will be needed to implement these mitigation strategies and to harmonise the usability practice.
This research continued previous investigation of the relationships among measures of perceived usability: the System Usability Scale (SUS), three metrics derived from the Usability Metric for User Experience (UMUX), and the Computer System Usability Questionnaire (CSUQ), this time with ratings of four everyday products (Excel, Word, Amazon, and Gmail). SUS ratings of these products were generally consistent with previous reports. Significant differences in SUS means across studies could be due to differences in frequency of use, with implications for using these data as usability benchmarks. Correspondence among the various measures of perceived usability was also consistent with previous research. Considering frequency of use, mean differences ranged from -2.0 to 1.8 (average shift in Sauro-Lewis grade range from -0.6 to 0.8). When SUS scores were above average, the range restriction of the UMUX-LITEr led to relatively large discrepancies with SUS, suggesting it might not always be better than unadjusted UMUXLITE.
“Usability” is a construct conceived by the human–computer interaction (HCI) community to denote a desired quality of interactive systems and products. Despite its prominence and intensive use in HCI research, the usefulness of the usability construct to HCI theories and to our understanding of HCI has been meager. In this article I propose and discuss two reasons for this state of affairs. The first is that usability is an umbrella construct. Umbrella constructs are prevalent in scientific fields that are broad, diverse, and lack a unifying research paradigm. Accordingly, umbrella constructs, such as usability, tend to be vague and loose, characteristics that challenge our ability to accumulate and communicate knowledge and to capture real-world phenomena. The second reason involves the nature of the relations between the usability construct and its measures, a topic rarely discussed in HCI research. There appears to be a mismatch between how the HCI community has (implicitly) conceptualized these relations and how it has empirically examined them. The relations have been conceptualized according to a formative measurement model but have mostly been tested according to a reflective measurement model. The trouble is that representing the usability construct by the reflective model appears inappropriate, and representing it by the formative model involves considerable difficulties. Possible ways of addressing these issues are discussed, each with its advantages and drawbacks. I conclude that for scientific research on this subject to progress, the usability construct ought to be unbundled and replaced by well-defined constructs. The issues discussed in this article are relevant to other HCI umbrella concepts and constructs such as user experience.
You're being asked to quantify your usability improvements with statistics. But even with a background in statistics, you are hesitant to statistically analyze their data, as they are often unsure which statistical tests to use and have trouble defending the use of small test sample sizes. The book is about providing a practical guide on how to solve common quantitative problems arising in usability testing with statistics. It addresses common questions you face every day such as: Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on the 1st attempt? How long will it take users to purchase products on the website? This book shows you which test to use, and how provide a foundation for both the statistical theory and best practices in applying them. The authors draw on decades of statistical literature from Human Factors, Industrial Engineering and Psychology, as well as their own published research to provide the best solutions. They provide both concrete solutions (excel formula, links to their own web-calculators) along with an engaging discussion about the statistical reasons for why the tests work, and how to effectively communicate the results. *Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices *Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data *Recommends ways for practitioners to communicate results to stakeholders in plain English. © 2012 Jeff Sauro and James R. Lewis Published by Elsevier Inc. All rights reserved.
The Usability Metric for User Experience (UMUX) is a four-item Likert scale aimed at replicating the psychometric properties of the System Usability Scale (SUS) in a more compact form. As part of a special issue of the journal Interacting with Computers, the UMUX is being examined in terms of purpose, reliability, validity and structure. This response to commentaries addresses concerns with these issues through updated archival research, deeper analysis on the original data and some updated results with an average-scoring system. The new results show the UMUX performs as expected for a wide range of systems and consists of one underlying usability factor.
The System Usability Scale (SUS) is an inexpensive, yet effective tool for assessing the usability of a product, including Web sites, cell phones, interactive voice response systems, TV applications, and more. It provides an easy-to-understand score from 0 (negative) to 100 (positive). While a 100-point scale is intuitive in many respects and allows for relative judgments, information describing how the numeric score translates into an absolute judgment of usability is not known. To help answer that question, a seven-point adjective-anchored Likert scale was added as an eleventh question to nearly 1,000 SUS surveys. Results show that the Likert scale scores correlate extremely well with the SUS scores (r=0.822). The addition of the adjective rating scale to the SUS may help practitioners interpret individual SUS scores and aid in explaining the results to non-human factors professionals.
This paper proposed a revised technology acceptance model for measuring end user computing (EUC) acceptance. An empirical study was conducted to collect data. This data was empirically used to test the proposed research model. The structural equation modeling technique was used to evaluate the causal model and confirmatory factor analysis was performed to examine the reliability and validity of the measurement model. The results demonstrate that the model explains 56% of the variance. This finding contributes to an expanded understanding of the factors that promote EUC acceptance. The implication of this work to both researchers and practitioners is discussed.
This article presents nearly 10 year's worth of System Usability Scale (SUS) data collected on numerous products in all phases of the development lifecycle. The SUS, developed by Brooke (1996)2.
Brooke , J. 1996. “SUS: A “quick and dirty” usability scale”. In Usability evaluation in industry, Edited by:
Jordan , P. W. ,
Thomas , B. A. Weerdmeester and
McClelland , I. L. 189–194. London: Taylor & Francis. View all references, reflected a strong need in the usability community for a tool that could quickly and easily collect a user's subjective rating of a product's usability. The data in this study indicate that the SUS fulfills that need. Results from the analysis of this large number of SUS scores show that the SUS is a highly robust and versatile tool for usability professionals. The article presents these results and discusses their implications, describes nontraditional uses of the SUS, explains a proposed modification to the SUS to provide an adjective rating that correlates with a given score, and provides details of what constitutes an acceptable SUS score.
Thesis (Ph. D.)--Arizona State University, 1977. Includes vita. Includes bibliographical references (leaves [193]-210). Photocopy.
A protective glycolipid antigen (PAg) was extracted from Leptospira interrogans serovar canicola with chloroform/methanol/water (1:2:0.8, by vol.) and partially purified by silica gel column chromatography. The PAg elicited a protective response in hamsters and in cyclophosphamide-treated mice subsequently challenged with homologous Leptospira. The PAg band was detected as a single smear-like band, corresponding to a protein of 23-30 kDa, by silver-staining in SDS-PAGE. In immunoblots, this band reacted with a monoclonal antibody, A5, which agglutinated serovar canicola and recognized a serovar-specific antigen. Furthermore, the PAg did not migrate on silica gel TLC, but was detected at the origin as a ninhydrin- and naphthol-positive spot. This suggests that PAg is a hydrophilic molecule with a carbohydrate chain that contains amino groups, possibly as amino sugars.
Companies spend lots of time and money on complex tools to assess customer satisfaction. But they're measuring the wrong thing. The best predictor of top-line growth can usually be captured in a single survey question: Would you recommend this company to a friend? This finding is based on two years of research in which a variety of survey questions were tested by linking the responses with actual customer behavior--purchasing patterns and referrals--and ultimately with company growth. Surprisingly, the most effective question wasn't about customer satisfaction or even loyalty per se. In most of the industries studied, the percentage of customers enthusiastic enough about a company to refer it to a friend or colleague directly correlated with growth rates among competitors. Willingness to talk up a company or product to friends, family, and colleagues is one of the best indicators of loyalty because of the customer's sacrifice in making the recommendation. When customers act as references, they do more than indicate they've received good economic value from a company; they put their own reputations on the line. And they will risk their reputations only if they feel intense loyalty. The findings point to a new, simpler approach to customer research, one directly linked to a company's results. By substituting a single question--blunt tool though it may appear to be--for the complex black box of the customer satisfaction survey, companies can actually put consumer survey results to use and focus employees on the task of stimulating growth.
Ergonomic requirements for office work with visual display terminals (VDTs), Part 11
ISO, 1998. Ergonomic requirements for office work with visual display terminals
(VDTs), Part 11, Guidance on usability (ISO 9241-11:1998E). Geneva,
Switzerland: Author.
The computer user satisfaction inventory (CUSI): Manual and scoring key
- J Kirakowski
- A Dillon
Kirakowski, J., & Dillon, A. (1988). The computer user satisfaction inventory (CUSI): Manual and scoring key. Cork, Ireland: Human Factors
Research Group, University College of Cork.
The usability construct: A dead end? Human-Computer Interaction
- N Tractinsky
Tractinsky, N. (2018). The usability construct: A dead end? Human-Computer Interaction, 33(2), 131-177. doi:10.1080/07370024.2017.12
98038
Lewis is a distinguished user experience researcher at MeasuringU, an IBM Master Inventor emeritus (over 90 patents), and the author of Practical Speech User Interface Design and
- R James
James R. Lewis is a distinguished user experience researcher at
MeasuringU, an IBM Master Inventor emeritus (over 90 patents), and
the author of Practical Speech User Interface Design and (with Jeff
Sauro), Quantifying the User Experience.