BookPDF Available

Computer Systems Experiences of Users with and without Disabilities: An Evaluation Guide for Professionals

Authors:

Abstract

The book we propose is not only a classic handbook or a practical guide for evaluation practictioners that presents and discusses one or a set of evaluation techniques for assessing diferent aspects of interaction. Our proposal is at first a new theoretical perspective in the human computer interaction evaluation that aims to integrate, in a multisteps evaluation process, more techniques for obtaining a whole assessment of interaction. Our theorical perspective is supported by an historical and experimental argumentation. Secondary our book by merging a user center perspective with the idea of user experience and with the growing need of disabled users partecipation in the evalaution and in the improvment of the HCI, proposes a reconceptualization of the web, social and portable tecnologies in a new category the “psychotecnologies” with specific properties. The integrated methodology of intercation evalaution is proposed as a framework for practictioners in order to evaluate all the aspects of the interaction from the accessibility (i.e. the more obejective point of view) to the staisfaction (i.e. the most subjective poitn of view). The evalaution techniques we analyse and the evaluation tools we propose in the book are supported by experimental exemplifications and are correlated to their application in the integrated methodology. Our goal is not only to presents the correct application of the techniques, but also to promote a standard evaluation process in which disabled and not disabled peoples are involved in the assessment.
A preview of the PDF is not available
... Good system usability is necessary but not a sufficient condition for a good UX; while usability is one dimension of the interaction, UX is a holistic perspective on how a user feels about using a system. There are various definitions of UX (for a review on UX, see also Borsci, Kurosu, Federici, and Mele, 2013;Bussolon, 2016) including the one provided by Norman, "all aspects of the user's interactions with the product: how it is perceived, learned and used. It includes ease of use and, most important of all, the needs that the product fulfils" (1998, p. 47), and the definition provided by Garrett, "how the product behaves and is used in the real world" (2003, p. 17). ...
... It is not sufficient to consider the object (device and system) and the subject (user) as two extreme points of the interaction continuum. The weakness of this dichotomous model is that it does not properly consider the dynamic relationship between the system and user, which is an emergent phenomenon that is not reducible to components (system and user) (Borsci et al., 2013;Federici et al., 2005). A member of the design team who tries to evaluate user-system interactions should evaluate the relationship between the system and user, considering the perspectives of both the object and subject (Figure 14.4 ...
... The accessibility evaluation covers the evaluation of objective access to the interaction, and is a measure of the way the architecture accomplishes the standards (i.e., objectivity), whereas the usability evaluation covers the subjective use of the interaction (i.e., subjectivity). The separation of objective and subjective aspects of interaction is a limitation of UX studies that can be overcome only by an integrated interaction evaluation model (Borsci et al., 2013;. ...
Chapter
This chapter discusses the relationship between the accessibility and usability constructs and how they relate to the user experience (UX) theoretical approach. We present an integrated model of interaction evaluation, a new evaluation perspective based on UX that is intended to be used as a framework for evaluating users’ interactions with assistive technology (AT) and to organize and evaluate the AT assessment process. The evaluator’s mental model is used to evaluate the relationship between the designers’ and the users’ mental models from objective and the subjective points of view. The new perspective endorsed by the chapter is that the UX concept can be used not only to set up an evaluation of users’ interactions with AT, but also to organize and evaluate the AT assessment process and to design (or redesign) technologies to overcome the barriers to use that disabled users typically experience. The redesign of a sonificated web search engine is presented as an example of the growing need to use a UX-based approach to AT design.
... Accordingly, to fully model the perceived experience of a user, practitioners should include a set of repeated objective and subjective measures in their evaluation protocols to enable satisfaction and benefit analysis as a "subjective sum of the interactive experience" [4]. Several standardized tools have been developed to measure satisfaction, realization of benefit and perceived usability of user with and without disabilities [5][6][7][8][9][10][11]. It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process (i.e., according to a usercentered design, UCD) [5,12]. ...
... Several standardized tools have been developed to measure satisfaction, realization of benefit and perceived usability of user with and without disabilities [5][6][7][8][9][10][11]. It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process (i.e., according to a usercentered design, UCD) [5,12]. ...
Chapter
To fully model the perceived experience of a user, practitioners should include a set of repeated objective and subjective measures in their evaluation protocols to enable satisfaction and benefit analysis as a “subjective sum of the interactive experience.” It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process. In this study, we aim to present how these concepts of UX and UCD inform the process of selecting and assigning assistive technologies (ATs) for people with disabilities (PWD) according to the Matching Person and Technology (MPT) model and assessments. To make technology the solution to the PWD’s needs, the MPT was developed as an international measure evidence-based tool to assess the best match between person and technology, where the user remains the main actor in all the selection, adaptation, and assignment process (user-driven model). The MPT model and tools assume that the characteristics of the person, environment, and technology should be considered as interacting when selecting the most appropriate AT for a particular person’s use. It has demonstrated good qualitative and quantitative psychometric properties for measuring UX, realization of benefit and satisfaction and, therefore, it is a useful resource to help prevent the needs and preferences of the users from being met and can reduce early technology abandonment and the consequent waste of money and energy.
... This paper aims at evaluating the UX of UTAssistant under laboratory conditions through two biobehavioral implicit measures, i.e. facial expression recognition and electroencephalography, and two explicit measures, the SUS (Lewis and Sauro, 2017;Borsci et al., 2015) and the Usability Metric for User Experience (UMUX) (Finstad, 2010;Borsci et al., 2015). The methodology adopted in this usability study was the PCTA (Borsci et al., 2013). ...
... In alpha frontal asymmetry, the power between left and right hemisphere is normalized between 0 (perfect symmetry) and 1 (maximal asymmetry). The experiment follows the PCTA method (Borsci et al., 2013). The PCTA requires that user and evaluator do not verbally interact during the whole duration of a task, but whenever users find a difficulty or want to express an opinion about the quality of their navigation, they are instructed to indicate this with a signal (generally the sound of a desk bell). ...
Conference Paper
Full-text available
This work shows the user experience (UX) assessment of a web-based platform for the semi-automatic usability evaluation of websites, UTAssistant, which is primarily addressed to workers in public administration (PA). The study is part (Phase 1) of a multiple assessment methodology which consists of four phases in total: (1) UX in laboratory conditions; (2) Usability evaluation in remote online conditions; (3) Usability evaluation in workplace conditions; and (4) Heuristic evaluation. In Phase 1, a UX study in laboratory conditions was carried out. Participants' UX of a PA website navigation through UTAssistant was evaluated by both traditional self-report usability assessment tools (SUS and UMUX) and bio-behavioral measurement techniques (facial expression recognition and electroencephalography). Results showed that using the UTAssistant usability assessment tool for webpages did not affect users' perceived usability in terms of self-reports and affective states, which were mostly neutral for all the assessment session. However, frontal alpha asymmetry EEG's scores showed a higher sensitivity of UTAssistant users to the duration of the trial, with a decrease in motivation displayed as the trial ensued. However, this result did not seem to affect emotional experience.
... In the PCTA technique, all user interactions are registered. As soon as the test is complete, the user is invited to identify and verbalize any problems experienced during the interaction [29]. ...
... Phase 1. Heuristic Evaluation. Many heuristic lists are proposed in the literature [29]. In this work, we use 10 heuristics for Web interface analysis created by Nielsen and Molich [16]; these take into account many aspects of the user interaction such as safety, flexibility, and efficiency of use. ...
Chapter
Full-text available
Since 2012, usability testing in Italian public administration (PA) has been guided by the eGLU 2.1 technical protocols, which provide a set of principles and procedures to support specialized usability assessments in a controlled and predictable way. This paper describes a new support tool for usability testing that aims to facilitate the application of eGLU 2.1 and the design of its User eXperience (UX) evaluation methodology. The usability evaluation tool described in this paper is called UTAssistant (Usability Tool Assistant). UTAssistant has been entirely developed as a Web platform, supporting evaluators in designing usability tests, analyzing the data gathered during the test and aiding Web users step-by-step to complete the tasks required by an evaluator. It also provides a library of questionnaires to be administered to Web users at the end of the usability test. The UX evaluation methodology adopted to assess the UTAssistant platform uses both standard and new bio-behavioral evaluation methods. From a technological point of view, UTAssistant is an important step forward in the assessment of Web services in PA, fostering a standardized procedure for usability testing without requiring dedicated devices, unlike existing software and platforms for usability testing.
... The procedures in UTAssistant follow the protocols used in eGLU 2.1 and eGLU-M, and the experimental methodology used to evaluate the usability of UTAssistant also follows the GLU principles and recommendations by integrating them with new bio-behavioral methods for assessing user interaction. This methodology combines various methods and techniques borrowed from standard usability evaluation procedures and psychophysiological investigation methods using bio-behavioral measures [6]. ...
... The new methodology aims to assess usability tools by combining the following: (i) eye-tracking, facial recognition, and electroencephalography measurements [13]; (ii) standard usability evaluation processes, which are compliant with international usability guidelines [6]; (iii) heuristic usability investigations by UX experts [14][15][16][17]; and (iv) remote online usability evaluation with highly representative numbers of end-users recruited through Web-based recruitment platforms. The experimental methodology uses biobehavioral variables that are mostly hidden from users, thus overcoming the most common issues that can occur in traditional assessment methodologies, such as the users' tendency to answer questions in a way that is affected by the presumed expectations of the evaluator, i.e. the social desirability bias. ...
Conference Paper
Full-text available
In the framework of the AVI 2018 Conference, the interuniversity center ECONA has organized a thematic workshop on "Digital Interaction: where are we going?". Six contributions from the ECONA members investigate different perspectives around this thematic.
... Segundo et. al. [25], as perspectivas de Usabilidade, Experiência do Usuário (User eXperience -UX) e Acessibilidade são as mais importantes de todas as discussões centradas na interação usuário-sistema. A Usabilidade segundo a ISO 9241-11 [2], é a medida pela qual um produto pode ser usado por usuários específicos para alcançar objetivos específicos com eficácia, eficiência e satisfação em um contexto de uso específico. ...
... User experience (UX) proposes to focus the interaction evaluation on 'person's perceptions and responses resulting from the use and/or anticipated use of a product' (ISO 9241-210, 2010, 3). UX includes the dimensions of usability (Law and van Schaik 2010) and concurrently attempts to enlarge the assessment's factors with a focus on cognitive, aesthetics, and qualitative aspects of interaction measured throughout time (Tractinsky 1997;Hassenzahl 2005;Hassenzahl and Tractinsky 2006;Petrie and Bevan 2009;Borsci et al. 2013;Bussolon 2016). UX practices were built upon usability and interaction methodology, but also dealt with methodological challenges due to lack of consistent measures to reliably evaluate factors such as people expectation (anticipated use), emotional reactions, etc. (Bargas-Avila and Hornbaek 2011). ...
Article
A recent contribution to the ongoing debate concerning the concept of usability and its measures proposed that usability reached a dead end – i.e. a construct unable to provide stable results and to unify scientific knowledge. Extensive commentaries rejected the conclusion that researchers need to look for alternative constructs to measure the quality of interaction. Nevertheless, several practitioners involved in this international debate asked for a constructive way to move forward the usability practice. In fact, two key issues of the usability field were identified in this debate: (i) knowledge fragmentation in the scientific community, and (ii) the unstable relationship among the usability metrics. We recognise both the importance and impact of these key issues, although, in line with others, we may not agree with the conclusion that the usability is a dead end. Under the light of the international debate, this work discusses the strengths and weaknesses of usability construct and its application. Our discussion focuses on identifying alternative explanations to the issues and to suggest mitigation strategies, which may be considered the starting point to move forward the usability field. However, scientific community actions will be needed to implement these mitigation strategies and to harmonise the usability practice.
... The development of predictive models that are able to determine a person's potential to adopt a particular technology continues to be desirable and to foster research interest. Most of the assistive solutions described above were developed and evaluated using a UCD approach and user-experience (UX) evaluation (Borsci, Kurosu, Federici, and Mele, 2013). Over the past decade, the AAL approach strongly promoted the UCD and process in order to meet end users' needs and expectations by adapting products to the characteristics of the physical environment and social milieu in which they are supposed to be used in order to prevent technology nonuse or abandonment (Federici et al., 2014;Scherer, 2014). ...
Chapter
Heterogeneity in the health status of elderly patients requires a particular care approach and geriatric medicine is the answer. In order to cope with frailty, disability, and diseases, the geriatric assessment approach guides the geriatrician into considering the interaction between functional status and cognitive, medical, affective, environmental, social support, economic, and spirituality dimensions. Rehabilitation is the goal of the geriatric assessment and the introduction of assistive solutions in geriatric rehabilitation makes possible a scenario in which the functioning of elderly people with physical or cognitive limitations is improved. This chapter provides an overview of the areas where technological systems may offer support to the everyday life of the elderly and their caregivers. The contribution of a geriatrician in a Centre for Technical Aid is described, linking the comprehensive geriatric assessment with the ICF model. The lack of implementation of the ICF and the requirement of training in assistive solutions for geriatricians and caregivers are discussed.
Article
Full-text available
http://dx.doi.org/10.5007/1807-9288.2015v11n1p65 Propomos que a literatura em meio digital contemporânea possui uma dimensão que chamamos de “trans-racional”, constituída pela codificação cultural de uma imagem corporal do leitor na mesma materialidade com que se produzem objetos em meio digital. Para isso, visitamos teóricos que analisam como se constroem objetos tecnológicos e digitais e como se recebem desde uma perspectiva fenomenológica. Concluimos que a codificação cultural do corpo no meio digital torna-se ponto de partida para que autores exerçam a sua criatividade.
Conference Paper
User experience has a fundamental role in determining the effectiveness of image compression methods. This work presents the subjective evaluation of a new compression plug-in for current compression formats developed by Cogisen. The quality of image compression methods is often evaluated by objective metrics based on subjective quality datasets, rather than by using subjective quality evaluation tests. Cogisen’s compression method follows an adaptive compression process that evaluates the saliency of any image and calculates the level of compression beyond which viewers shall be aware of image quality degradation. The Single Stimulus Continuous Quality Scale method was used to conduct the subjective quality evaluation of image compression. Pictures compressed by the Facebook Mobile lossy JPEG compression and by the Cogisen plug-in integrated in the Facebook Mobile compression settings were used. The results of the user quality evaluation of pictures show about a 45 % compression improvement, with no loss in perceived image quality, for pictures compressed by the Cogisen plug-in compared to jpeg pictures as compressed by Facebook Mobile.
Article
Full-text available
People are more and more using social networking sites (SNSs) like Facebook and MySpace to engage with others. The use of SNSs can have both positive and negative effect on the individual; however, the increasing use of SNSs might reveal that people look for SNSs because they have a positive experience when they use them. Few studies have tried to identify which particular aspects of the social networking experience make SNSs so successful. In this study we focus on the affective experience evoked by SNSs. In particular, we explore whether the use of SNSs elicits a specific psychophysiological pattern. Specifically, we recorded skin conductance, blood volume pulse, electroencephalogram, electromyography, respiratory activity, and pupil dilation in 30 healthy subjects during a 3-minute exposure to (a) a slide show of natural panoramas (relaxation condition), (b) the subject's personal Facebook account, and (c) a Stroop and mathematical task (stress condition). Statistical analysis of the psychophysiological data and pupil dilation indicates that the Facebook experience was significantly different from stress and relaxation on many linear and spectral indices of somatic activity. Moreover, the biological signals revealed that Facebook use can evoke a psychophysiological state characterized by high positive valence and high arousal (Core Flow State). These findings support the hypothesis that the successful spread of SNSs might be associated with a specific positive affective state experienced by users when they use their SNSs account.
Book
You're being asked to quantify your usability improvements with statistics. But even with a background in statistics, you are hesitant to statistically analyze their data, as they are often unsure which statistical tests to use and have trouble defending the use of small test sample sizes. The book is about providing a practical guide on how to solve common quantitative problems arising in usability testing with statistics. It addresses common questions you face every day such as: Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on the 1st attempt? How long will it take users to purchase products on the website? This book shows you which test to use, and how provide a foundation for both the statistical theory and best practices in applying them. The authors draw on decades of statistical literature from Human Factors, Industrial Engineering and Psychology, as well as their own published research to provide the best solutions. They provide both concrete solutions (excel formula, links to their own web-calculators) along with an engaging discussion about the statistical reasons for why the tests work, and how to effectively communicate the results. *Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices *Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data *Recommends ways for practitioners to communicate results to stakeholders in plain English. © 2012 Jeff Sauro and James R. Lewis Published by Elsevier Inc. All rights reserved.
Article
Various improvements for user-centered design (UCD) are discussed. Collaborative usability inspection is a systematic technique developed and refined specifically for identifying usability defects. Through a highly structured procedure with explicit rules and highly refined roles, collaborative usability inspections identify usability problems more quickly than usability testing. For usability, uses are more important than users, and more importance should be paid to effective user performance than to the promotion of good user experience.