Content uploaded by Stefano Federici
Author content
All content in this area was uploaded by Stefano Federici
Content may be subject to copyright.
A preview of the PDF is not available
Computer Systems Experiences of Users with and without Disabilities: An Evaluation Guide for Professionals
Abstract
The book we propose is not only a classic handbook or a practical guide for evaluation practictioners that presents and discusses one or a set of evaluation techniques for assessing diferent aspects of interaction. Our proposal is at first a new theoretical perspective in the human computer interaction evaluation that aims to integrate, in a multisteps evaluation process, more techniques for obtaining a whole assessment of interaction. Our theorical perspective is supported by an historical and experimental argumentation. Secondary our book by merging a user center perspective with the idea of user experience and with the growing need of disabled users partecipation in the evalaution and in the improvment of the HCI, proposes a reconceptualization of the web, social and portable tecnologies in a new category the “psychotecnologies” with specific properties.
The integrated methodology of intercation evalaution is proposed as a framework for practictioners in order to evaluate all the aspects of the interaction from the accessibility (i.e. the more obejective point of view) to the staisfaction (i.e. the most subjective poitn of view).
The evalaution techniques we analyse and the evaluation tools we propose in the book are supported by experimental exemplifications and are correlated to their application in the integrated methodology. Our goal is not only to presents the correct application of the techniques, but also to promote a standard evaluation process in which disabled and not disabled peoples are involved in the assessment.
... Good system usability is necessary but not a sufficient condition for a good UX; while usability is one dimension of the interaction, UX is a holistic perspective on how a user feels about using a system. There are various definitions of UX (for a review on UX, see also Borsci, Kurosu, Federici, and Mele, 2013;Bussolon, 2016) including the one provided by Norman, "all aspects of the user's interactions with the product: how it is perceived, learned and used. It includes ease of use and, most important of all, the needs that the product fulfils" (1998, p. 47), and the definition provided by Garrett, "how the product behaves and is used in the real world" (2003, p. 17). ...
... It is not sufficient to consider the object (device and system) and the subject (user) as two extreme points of the interaction continuum. The weakness of this dichotomous model is that it does not properly consider the dynamic relationship between the system and user, which is an emergent phenomenon that is not reducible to components (system and user) (Borsci et al., 2013;Federici et al., 2005). A member of the design team who tries to evaluate user-system interactions should evaluate the relationship between the system and user, considering the perspectives of both the object and subject (Figure 14.4 ...
... The accessibility evaluation covers the evaluation of objective access to the interaction, and is a measure of the way the architecture accomplishes the standards (i.e., objectivity), whereas the usability evaluation covers the subjective use of the interaction (i.e., subjectivity). The separation of objective and subjective aspects of interaction is a limitation of UX studies that can be overcome only by an integrated interaction evaluation model (Borsci et al., 2013;. ...
This chapter discusses the relationship between the accessibility and usability constructs and how they relate to the user experience (UX) theoretical approach. We present an integrated model of interaction evaluation, a new evaluation perspective based on UX that is intended to be used as a framework for evaluating users’ interactions with assistive technology (AT) and to organize and evaluate the AT assessment process. The evaluator’s mental model is used to evaluate the relationship between the designers’ and the users’ mental models from objective and the subjective points of view. The new perspective endorsed by the chapter is that the UX concept can be used not only to set up an evaluation of users’ interactions with AT, but also to organize and evaluate the AT assessment process and to design (or redesign) technologies to overcome the barriers to use that disabled users typically experience. The redesign of a sonificated web search engine is presented as an example of the growing need to use a UX-based approach to AT design.
... Accordingly, to fully model the perceived experience of a user, practitioners should include a set of repeated objective and subjective measures in their evaluation protocols to enable satisfaction and benefit analysis as a "subjective sum of the interactive experience" [4]. Several standardized tools have been developed to measure satisfaction, realization of benefit and perceived usability of user with and without disabilities [5][6][7][8][9][10][11]. It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process (i.e., according to a usercentered design, UCD) [5,12]. ...
... Several standardized tools have been developed to measure satisfaction, realization of benefit and perceived usability of user with and without disabilities [5][6][7][8][9][10][11]. It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process (i.e., according to a usercentered design, UCD) [5,12]. ...
To fully model the perceived experience of a user, practitioners should include a set of repeated objective and subjective measures in their evaluation protocols to enable satisfaction and benefit analysis as a “subjective sum of the interactive experience.” It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process. In this study, we aim to present how these concepts of UX and UCD inform the process of selecting and assigning assistive technologies (ATs) for people with disabilities (PWD) according to the Matching Person and Technology (MPT) model and assessments. To make technology the solution to the PWD’s needs, the MPT was developed as an international measure evidence-based tool to assess the best match between person and technology, where the user remains the main actor in all the selection, adaptation, and assignment process (user-driven model). The MPT model and tools assume that the characteristics of the person, environment, and technology should be considered as interacting when selecting the most appropriate AT for a particular person’s use. It has demonstrated good qualitative and quantitative psychometric properties for measuring UX, realization of benefit and satisfaction and, therefore, it is a useful resource to help prevent the needs and preferences of the users from being met and can reduce early technology abandonment and the consequent waste of money and energy.
... This paper aims at evaluating the UX of UTAssistant under laboratory conditions through two biobehavioral implicit measures, i.e. facial expression recognition and electroencephalography, and two explicit measures, the SUS (Lewis and Sauro, 2017;Borsci et al., 2015) and the Usability Metric for User Experience (UMUX) (Finstad, 2010;Borsci et al., 2015). The methodology adopted in this usability study was the PCTA (Borsci et al., 2013). ...
... In alpha frontal asymmetry, the power between left and right hemisphere is normalized between 0 (perfect symmetry) and 1 (maximal asymmetry). The experiment follows the PCTA method (Borsci et al., 2013). The PCTA requires that user and evaluator do not verbally interact during the whole duration of a task, but whenever users find a difficulty or want to express an opinion about the quality of their navigation, they are instructed to indicate this with a signal (generally the sound of a desk bell). ...
This work shows the user experience (UX) assessment of a web-based platform for the semi-automatic usability evaluation of websites, UTAssistant, which is primarily addressed to workers in public administration (PA). The study is part (Phase 1) of a multiple assessment methodology which consists of four phases in total: (1) UX in laboratory conditions; (2) Usability evaluation in remote online conditions; (3) Usability evaluation in workplace conditions; and (4) Heuristic evaluation. In Phase 1, a UX study in laboratory conditions was carried out. Participants' UX of a PA website navigation through UTAssistant was evaluated by both traditional self-report usability assessment tools (SUS and UMUX) and bio-behavioral measurement techniques (facial expression recognition and electroencephalography). Results showed that using the UTAssistant usability assessment tool for webpages did not affect users' perceived usability in terms of self-reports and affective states, which were mostly neutral for all the assessment session. However, frontal alpha asymmetry EEG's scores showed a higher sensitivity of UTAssistant users to the duration of the trial, with a decrease in motivation displayed as the trial ensued. However, this result did not seem to affect emotional experience.
... In the PCTA technique, all user interactions are registered. As soon as the test is complete, the user is invited to identify and verbalize any problems experienced during the interaction [29]. ...
... Phase 1. Heuristic Evaluation. Many heuristic lists are proposed in the literature [29]. In this work, we use 10 heuristics for Web interface analysis created by Nielsen and Molich [16]; these take into account many aspects of the user interaction such as safety, flexibility, and efficiency of use. ...
Since 2012, usability testing in Italian public administration (PA) has been guided by the eGLU 2.1 technical protocols, which provide a set of principles and procedures to support specialized usability assessments in a controlled and predictable way. This paper describes a new support tool for usability testing that aims to facilitate the application of eGLU 2.1 and the design of its User eXperience (UX) evaluation methodology. The usability evaluation tool described in this paper is called UTAssistant (Usability Tool Assistant). UTAssistant has been entirely developed as a Web platform, supporting evaluators in designing usability tests, analyzing the data gathered during the test and aiding Web users step-by-step to complete the tasks required by an evaluator. It also provides a library of questionnaires to be administered to Web users at the end of the usability test. The UX evaluation methodology adopted to assess the UTAssistant platform uses both standard and new bio-behavioral evaluation methods. From a technological point of view, UTAssistant is an important step forward in the assessment of Web services in PA, fostering a standardized procedure for usability testing without requiring dedicated devices, unlike existing software and platforms for usability testing.
... The procedures in UTAssistant follow the protocols used in eGLU 2.1 and eGLU-M, and the experimental methodology used to evaluate the usability of UTAssistant also follows the GLU principles and recommendations by integrating them with new bio-behavioral methods for assessing user interaction. This methodology combines various methods and techniques borrowed from standard usability evaluation procedures and psychophysiological investigation methods using bio-behavioral measures [6]. ...
... The new methodology aims to assess usability tools by combining the following: (i) eye-tracking, facial recognition, and electroencephalography measurements [13]; (ii) standard usability evaluation processes, which are compliant with international usability guidelines [6]; (iii) heuristic usability investigations by UX experts [14][15][16][17]; and (iv) remote online usability evaluation with highly representative numbers of end-users recruited through Web-based recruitment platforms. The experimental methodology uses biobehavioral variables that are mostly hidden from users, thus overcoming the most common issues that can occur in traditional assessment methodologies, such as the users' tendency to answer questions in a way that is affected by the presumed expectations of the evaluator, i.e. the social desirability bias. ...
In the framework of the AVI 2018 Conference, the interuniversity center ECONA has organized a thematic workshop on "Digital Interaction: where are we going?". Six contributions from the ECONA members investigate different perspectives around this thematic.
... The experimenter then needed to record what the participant was doing when the bell rang. As soon as the participant finished the test, they were invited to discuss and verbalize the problems encountered during the interaction [13]. Hence, at the end of the experimental procedure, a short interview was conducted in which the participant was asked how the interaction with the interface had gone in general, the reasons that had prompted them to ring the bell, and the observations they made regarding the application. ...
Smartphones and tablets now offer consumers unique advantages such as portability and accessibility. Developers are also working with a mobile-first approach, and are prioritizing mobile applications over desktop versions. This study introduces eGLU-box Mobile, an application for performing a drive usability test directly from a smartphone. An experimental study was conducted in which the participants were divided into two groups: an experimental group, which used the new mobile application from a smartphone, and a control group, which used the desktop application from a computer. The participants’ behavior was assessed using explicit (self-report questionnaires) and implicit measures (eye movement data). The results were encouraging, and showed that both the mobile and desktop versions of eGLU-box enabled participants to test the usability with a similar level of UX, despite some minimal (although significant) differences in terms of satisfaction of use.
... Segundo et. al. [25], as perspectivas de Usabilidade, Experiência do Usuário (User eXperience -UX) e Acessibilidade são as mais importantes de todas as discussões centradas na interação usuário-sistema. A Usabilidade segundo a ISO 9241-11 [2], é a medida pela qual um produto pode ser usado por usuários específicos para alcançar objetivos específicos com eficácia, eficiência e satisfação em um contexto de uso específico. ...
... User experience (UX) proposes to focus the interaction evaluation on 'person's perceptions and responses resulting from the use and/or anticipated use of a product' (ISO 9241-210, 2010, 3). UX includes the dimensions of usability (Law and van Schaik 2010) and concurrently attempts to enlarge the assessment's factors with a focus on cognitive, aesthetics, and qualitative aspects of interaction measured throughout time (Tractinsky 1997;Hassenzahl 2005;Hassenzahl and Tractinsky 2006;Petrie and Bevan 2009;Borsci et al. 2013;Bussolon 2016). UX practices were built upon usability and interaction methodology, but also dealt with methodological challenges due to lack of consistent measures to reliably evaluate factors such as people expectation (anticipated use), emotional reactions, etc. (Bargas-Avila and Hornbaek 2011). ...
A recent contribution to the ongoing debate concerning the concept of usability and its measures proposed that usability reached a dead end – i.e. a construct unable to provide stable results and to unify scientific knowledge. Extensive commentaries rejected the conclusion that researchers need to look for alternative constructs to measure the quality of interaction. Nevertheless, several practitioners involved in this international debate asked for a constructive way to move forward the usability practice. In fact, two key issues of the usability field were identified in this debate: (i) knowledge fragmentation in the scientific community, and (ii) the unstable relationship among the usability metrics. We recognise both the importance and impact of these key issues, although, in line with others, we may not agree with the conclusion that the usability is a dead end. Under the light of the international debate, this work discusses the strengths and weaknesses of usability construct and its application. Our discussion focuses on identifying alternative explanations to the issues and to suggest mitigation strategies, which may be considered the starting point to move forward the usability field. However, scientific community actions will be needed to implement these mitigation strategies and to harmonise the usability practice.
People are more and more using social networking sites (SNSs) like Facebook and MySpace to engage with others. The use of SNSs can have both positive and negative effect on the individual; however, the increasing use of SNSs might reveal that people look for SNSs because they have a positive experience when they use them. Few studies have tried to identify which particular aspects of the social networking experience make SNSs so successful. In this study we focus on the affective experience evoked by SNSs. In particular, we explore whether the use of SNSs elicits a specific psychophysiological pattern. Specifically, we recorded skin conductance, blood volume pulse, electroencephalogram, electromyography, respiratory activity, and pupil dilation in 30 healthy subjects during a 3-minute exposure to (a) a slide show of natural panoramas (relaxation condition), (b) the subject's personal Facebook account, and (c) a Stroop and mathematical task (stress condition). Statistical analysis of the psychophysiological data and pupil dilation indicates that the Facebook experience was significantly different from stress and relaxation on many linear and spectral indices of somatic activity. Moreover, the biological signals revealed that Facebook use can evoke a psychophysiological state characterized by high positive valence and high arousal (Core Flow State). These findings support the hypothesis that the successful spread of SNSs might be associated with a specific positive affective state experienced by users when they use their SNSs account.
In 1994 a computer program called the Mosaic browser transformed the Internet from an academic tool into a telecommunications revolution. Now a household name, the World Wide Web is part of the modern communications landscape with tens of thousands of servers providing information to millions of users. Few people, however, realize that the Web was born at CERN, the European Laboratory for Particle Physics, in Geneva, and that it was invented by an Englishman, Tim Berners-Lee. This new book, published in the Popular Science list in Oxford Paperbacks, tells how the idea for the Web came about at CERN, how it was developed, and how it was eventually handed over for free for the rest of the world to use. This is the first book-length account of the Web's development and it includes interview material with the key players in the story.
Wellman presents evidence that children as young as age three do possess a commonsense theory of mind—that they grasp the distinction between mental constructs and physical entities and that they have an understanding of the relationship between individuals' mental states and their overt actions. He delves in detail into questions about the nature of adults' commonsense theories of mind and about the nature of commonsense theories.
Wellman then examines the content of the three-year-old's theory of mind, the nature of children's notions of mind before age three, the changes in the theory during subsequent development from ages three to six, and the young child's conception of mind in comparison with those of older children and adults.
Bradford Books imprint
You're being asked to quantify your usability improvements with statistics. But even with a background in statistics, you are hesitant to statistically analyze their data, as they are often unsure which statistical tests to use and have trouble defending the use of small test sample sizes. The book is about providing a practical guide on how to solve common quantitative problems arising in usability testing with statistics. It addresses common questions you face every day such as: Is the current product more usable than our competition? Can we be sure at least 70% of users can complete the task on the 1st attempt? How long will it take users to purchase products on the website? This book shows you which test to use, and how provide a foundation for both the statistical theory and best practices in applying them. The authors draw on decades of statistical literature from Human Factors, Industrial Engineering and Psychology, as well as their own published research to provide the best solutions. They provide both concrete solutions (excel formula, links to their own web-calculators) along with an engaging discussion about the statistical reasons for why the tests work, and how to effectively communicate the results. *Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices *Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data *Recommends ways for practitioners to communicate results to stakeholders in plain English. © 2012 Jeff Sauro and James R. Lewis Published by Elsevier Inc. All rights reserved.
Various improvements for user-centered design (UCD) are discussed. Collaborative usability inspection is a systematic technique developed and refined specifically for identifying usability defects. Through a highly structured procedure with explicit rules and highly refined roles, collaborative usability inspections identify usability problems more quickly than usability testing. For usability, uses are more important than users, and more importance should be paid to effective user performance than to the promotion of good user experience.