added a research item
To fully model the perceived experience of a user, practitioners should include a set of repeated objective and subjective measures in their evaluation protocols to enable satisfaction and benefit analysis as a “subjective sum of the interactive experience.” It is also well known that if the UX of a product is assessed at the end of the design process, product changes are much more expensive than if the same evaluation were conducted throughout the development process. In this study, we aim to present how these concepts of UX and UCD inform the process of selecting and assigning assistive technologies (ATs) for people with disabilities (PWD) according to the Matching Person and Technology (MPT) model and assessments. To make technology the solution to the PWD’s needs, the MPT was developed as an international measure evidence-based tool to assess the best match between person and technology, where the user remains the main actor in all the selection, adaptation, and assignment process (user-driven model). The MPT model and tools assume that the characteristics of the person, environment, and technology should be considered as interacting when selecting the most appropriate AT for a particular person’s use. It has demonstrated good qualitative and quantitative psychometric properties for measuring UX, realization of benefit and satisfaction and, therefore, it is a useful resource to help prevent the needs and preferences of the users from being met and can reduce early technology abandonment and the consequent waste of money and energy.
Through the Web communication development, new users’ needs are rising related to the amount of information and its findability, and also in relation to the increasing necessity to ensure quality for user-technology interaction. The current model of information ranking by search engines is based on quantitative Web Popularity (WP) and it binds the user to a cognitive adaptation based on an heterodirected rank-system restrictions which transposes the rich-get-richer effect from technologies to reality. Several works in literature show the need to implement the current Web-ranking models through the introduction of algorithms, which are able to evaluate the quality of pages and the information they contain on a qualitative level easing users’ cognitive adaptation to the technology.
This work aimed at expanding thearchitecture of WhatsOnWeb [Di Giacomo et al., 2005] following accessibility and usability state-of-the-art criteria. WoW is a visual Web search engine that conveys the indexed dataset using graph-drawing methods on semantically clustered data. In previous studies [Federici et al., forthcoming; Federici et al., 2008], we found that top-down representation of the most widespread search engines does not take into account the level of accessibility of information, and this ranking highlights the distance between the quantitative order of Web Popularity (WP) and the qualitative level of accessibility of the retrieved information. Conversely, WoW semantically analyzes the search results, and automatically links them in a network of concepts and sub-concepts. The whole information retrieved is presented to the user simultaneously in a interactive visual map, overcoming the gap between quantitative order and qualitative level of information. The redesigning process of WoW has been performed by following the user-centered design methodology and in compliance with the WCAG 1.0. This way, we implemented a sonification algorithm that converts data relations and their spatial coordinates in non-speech sounds (sonification). With WhatsOnWeb we aim at providing an autonomous assistive technology tool that allows for an independent navigation, based on both an integrated synthesizer for screen reading and a sonification system, by which conveying geometrical-spatial information representation. The result is an innovative web search and resulttargeting approach, that also reduces the gap between quantitative and qualitative result ranking, as seen on major web search platforms.
Our study analyse the validity and the reliability of two well-known questionnaires for the usability assessment, the SUMI and the QUIS, in their Italian version. The sample is composed of 250 university students of psychology (University of Rome “La Sapienza”, Faculty of Psychology 1), that used two questionnaires to assess the usability of the faculty website. The principal components analysis of the two questionnaires did not show the attended factors divisions. The reliability and the convergent validity analysis show some psychometric limits. The Pearson’s coefficient analysis show a low correlation between these two tools’ data, indicating that the SUMI and the QUIS assess different aspects of usability. These results, showing the limits of the questionnaires, underline the increasing necessity to use integrate usability evaluation methodologies in order to catch all multi-dimensional aspects of the human computer interaction. RIASSUNTO Nel presente lavoro è stata indagata l’operatività di due tra gli strumenti più diffusi per la misurazione dell’usabilità di interfacce, il SUMI e il QUIS, nella loro versione italiana. Un campione di 250 studenti universitari ha partecipato allo studio valutando, attraverso i due strumenti indagati, il sito web della Facoltà di Psicologia 1 dell’Università di Roma “La Sapienza”. Le analisi delle componenti principali condotte sui due strumenti non hanno restituito i raggruppamenti in fattori come atteso per ciascuno degli strumenti. Allo stesso modo si sono evidenziati limiti psicometrici riguardo l’attendibilità e la validità convergente dei due strumenti. Infine, la bassa correlazione fra i dati di valutazione, forniti da tali strumenti, sembra indicare che il QUIS e il SUMI misurino aspetti diversi dell’usabilità. Tali risultati, delineando i limiti di questi questionari, sottolineano la necessità crescente di utilizzare metodologie di valutazione dell’usabilità che impieghino un insieme di tecniche integrate fra loro in grado di cogliere la complessità multidimensionale dell’interazione utente/tecnologia.
It is widely accepted that spatial representation is processed by an amodal system. Recent studies show that blind subjects have a better motion ability than sighted people in performing spatial exploration guided only by auditory cues. The sonification method offers an effective tool able to transmit graphic information, overcoming the digital divide risen by a visuocentric modality in which contents are conveyed. We present a usability evaluation aiming at investigate the interaction differences between both blind and sighted users while surfing WhatsOnWeb, a search engine that displays the information by using graph-drawing methods on semantically clustered data. We compare the visual presentation of three different layouts with the sonificated ones, demonstrating both qualitatively and quantitatively that blind and sighted users perform with no significant differences the interaction. These results remark that the digital divide could be decreased by going beyond the visuocentric way of the commonly adopted visual content representation.
Information Visualization conveys abstract information in intuitive ways by representing it with geometric models. An important and valid alternative to visual traditional methods appeared to be the sonification approach, which implements nonspeech audio information to “represent data relations into perceived relations in an acoustic signal for the purposes of facilitating communication and interpretation”. WhatsOnWeb is a clustering Web search engine that makes use of visualization techniques to improve effectiveness and efficiency of Web searching. In a clustering engine, the search results are analyzed to automatically identify the topics they belong to which are then linked in a network of concepts and sub-concepts that are presented to the user by means of different visual maps. As a case study, we focus on Web search engines for blind people. To this aim, the main activities of our project are: (i) Reengineering the architecture of an existing Web search clustering engine, called WhatsOnWeb (WOW, http://whatsonweb.diei.unipg.it:8080/wow3.2/); (ii) Devising effective ways of interacting with the system by translating the visualization into sounds; (iii) Extending the WOW ranking algorithm to accessibility indexes; (iv) Performing a set of validation activities to demonstrate the effectiveness of the approach for visual impaired people.
In the study entitled "Web usability evaluation with screen reader users: Implementation of the Partial Concurrent Thinking Aloud technique" (Federici et al. 2010), we have proposed a modified protocol of usability evaluation technique for blind users, which integrates the features of the concurrent and the retrospective techniques. This new technique, called partial concurrent thinking aloud (PCTA), while respecting the properties of classic verbal protocols, overcomes the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In order to facilitate understanding and acquisition of the PCTA's technique for practitioners and researchers, we have video recorded three different verbal protocols by visualizing five experimental sections. In the first two videos, we have compared a concurrent with a retrospective's verbal protocol of a sighted user, showing the difference of the verbalizations provided by the user in these two conditions. The third video shows the structural interference of the screen reader, during a blind user concurrent thinking aloud. In the last two videos, we show the difference of a blind user behaviour when PCTA or retrospective protocol is adopted. The videos clearly visualize the advantage of the PCTA use in respect of the two other protocols. In conclusion, the visualization of the PCTA technique confirms that this new verbal protocol promotes and guarantees a more user-driven usability assessment with disabled people, by better involving screen reader users, overcoming the structural interference and the limits of the concurrent and retrospective protocols.
A verbal protocol technique, adopted for a web usability evaluation, requires that the users are able to perform a double task: surfing and talking. Nevertheless, when blind users surf by using a screen reader and talk about the way they interact with the computer, the evaluation is influenced by a structural interference: users are forced to think aloud and listen to the screen reader at the same time. The aim of this study is to build up a verbal protocol technique for samples of visual impaired users in order to overcome the limits of concurrent and retrospective protocols. The technique we improved, called partial concurrent thinking aloud (PCTA), integrates a modified set of concurrent verbalization and retrospective analysis. One group of 6 blind users and another group of 6 sighted users evaluated the usability of a website using PCTA. By estimating the number of necessary users by the means of an asymptotic test, it was found out that the two groups had an equivalent ability of identifying usability problems, both over 80%. The result suggests that PCTA, while respecting the properties of classic verbal protocols, also allows to overcome the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In this way, PCTA reduces the efficiency difference of usability evaluation between blind and sighted users.
The aim of this study was to present a usability evaluation conducted under the User Experience (UX) perspective of a sonificated search engine called WhatsOnWeb, an accessible application based on sophisticated graph visualization algorithms which conveys datasets using graph-drawing methods based on semantically clustered data. Starting with evidence from an amodal system processing spatial representation, the differences between blind and sighted users’ interactions whilst surfing WoW was analysed by following the Partial Concurrent Thinking Aloud protocol. Our results demonstrate that the user’s ability to perform spatial exploration tasks guided by either visual or acoustic cues seems to be functionally equivalent.
Recently Federici and Scherer (2012) proposed an ideal model of an Assistive Technology Assessment (ATA) process that provides reference guidelines for professionals of a multidisciplinary team of assistive technology (AT) service delivery centers to compare, evaluate, and improve their own matching models. The ATA process borrows a user-driven working methodology from the Matching Person and Technology Model (Scherer, 1998) and it embraces the biopsychosocial model (WHO, 2001) aiming at the best combination of AT to promote customers’ personal well-being. As Federici and Scherer (2012) suggest, the multidisciplinary team, by applying the ATA process, may provide for users not only a device, but much more an assistive solution, which is the real outcome of a match process. An assistive solution is provided for the user only when the interaction dialogue between user, device, and environments of use improves the users’ performances in participating in their everyday contexts. In this theoretical framework, the evaluation of the users’ interaction with the AT in different kinds of environments is a key factor for the success of the ATA process, because, as Mirza, Gossett Zakrajsek, and Borsci (2012) claim, the environment is antecedent to the AT and crucial for identifying how the AT works in relation to the users’ needs. In the ATA process a specific Environ-mental Assessment (EA) model for testing the interaction of the user with the environments of use, through the AT, has been defined. The aim of this paper is to describe the EA model steps and discuss the dimensions that a practitioner has to consider for this assessment. Accessibility, universal design, and sustainability are used in the EA model as the dimensions for measuring the relationship between the AT and the environment (Mirza, et al., 2012). The EA model steps and the trade-off among these dimensions are presented through a case example in which practitioners analyze the relationship between a communication aid used by a child and her classroom and home environments.
The book we propose is not only a classic handbook or a practical guide for evaluation practictioners that presents and discusses one or a set of evaluation techniques for assessing diferent aspects of interaction. Our proposal is at first a new theoretical perspective in the human computer interaction evaluation that aims to integrate, in a multisteps evaluation process, more techniques for obtaining a whole assessment of interaction. Our theorical perspective is supported by an historical and experimental argumentation. Secondary our book by merging a user center perspective with the idea of user experience and with the growing need of disabled users partecipation in the evalaution and in the improvment of the HCI, proposes a reconceptualization of the web, social and portable tecnologies in a new category the “psychotecnologies” with specific properties. The integrated methodology of intercation evalaution is proposed as a framework for practictioners in order to evaluate all the aspects of the interaction from the accessibility (i.e. the more obejective point of view) to the staisfaction (i.e. the most subjective poitn of view). The evalaution techniques we analyse and the evaluation tools we propose in the book are supported by experimental exemplifications and are correlated to their application in the integrated methodology. Our goal is not only to presents the correct application of the techniques, but also to promote a standard evaluation process in which disabled and not disabled peoples are involved in the assessment.
The abandonment of assistive technology (AT) is strictly related to the subjective quality of the service delivery regarding the whole AT assignation process. Starting from this consideration, the aim of this work is to show the design of a Web-based follow-up model (WFM) aimed at overcoming the hearing aid abandonment in the Italian Umbria Region AT service delivery system. The WFM model described here is developed in two phases: an implementation phase, and an experimental evaluation which is still under development. The model meets the current objective of the Umbria Region’s Units of Local Health Service to digitize their services in order to easily monitor the quality of the delivery service and evaluate the post-provision outcome.
Background. The introduction of assistive technology (AT) into people’s lives is a deliberative and long-term process, which presupposes teamwork as much as professionalism, time, and experience. The aim of the assistive technology assessment (ATA) process is to suggest guidelines to follow in order to reach valid results during the AT selection and assignment process. Purpose. Critically discuss the application of the model of the ATA process developed by Federici and Scherer in the 2012. Method. Cross-cultural comparison of AT service delivery systems and discussion of the ATA process model adopted by Leonarda Vaccari Institute of Rome. Conclusion. Nowadays, the wide variety of AT devices on the market opens new frontiers to the individual’s enhanced functioning, inclusion, and participation. Since the choice of the most appropriate match is often a complex process, a systematic selection process such as the ATA process described in this article can help practitioners to efficiently achieve successful outcomes.
The purpose of the present work is to present some aspects of the Assistive Technology Assessment (ATA) process model  compatible with the Position Paper 2012 by AAATE/EASTIN . Three aspects of the ATA process will be discussed in light of three topics of the Position Paper 2012: (i) The dimensions and the measures of the User eXperience (UX) evaluation modelled in the ATA process as a way to verify the efficient and the evidence-based practices of an AT service delivery centre; (ii) The relevance of the presence of the psychologist in the multidisciplinary team of an AT service delivery centre as necessary for a complete person-centred assistive solution empowering users to make their own choices; (iii) The new profession of the psychotechnologist, who explores user’s needs by seeking a proper assistive solution, leading the multidisciplinary team to observe critical issues and problems. Through the foundation of the Position Paper 2012, the 1995 HEART study, the Matching Person and Technology model, the ICF framework, and the pillars of the ATA process, this paper sets forth a concept and approach that emphasise the personal factors of the individual consumer and UX as key to positively impacting a successful outcome and AT solution.
Purpose: The study brings together three aspects rarely observed at once in assistive technology (AT) surveys: (i) the assessment of user interaction/satisfaction with AT and service delivery, (ii) the motivational analysis of AT abandonment, and (iii) the management/design evaluation of AT delivery services. Methods: 15 health professionals and 4 AT experts were involved in modelling and assessing four AT Local Health Delivery Service (Centres) in Italy through a SWOT analysis and a Cognitive Walkthrough. In addition 558 users of the same Centres were interviewed in a telephone survey to rate their satisfaction and AT use. Results: The overall AT abandonment was equal to 19.09%. Different Centres' management strategies resulted in different percentages of AT disuse, with a range from 12.61% to 24.26%. A significant difference between the declared abandonment and the Centres' management strategies (p = 0.012) was identified. A strong effect on abandonment was also found due to professionals' procedures (p = 0.005) and follow-up systems (p = 0.002). Conclusions: The user experience of an AT is affected not only by the quality of the interaction with the AT, but also by the perceived quality of the Centres in support and follow-up. Implications for Rehabilitation AT abandonment surveys provide useful information for modelling AT assessment and delivery process. SWOT and Cognitive Walkthrough analyses have shown suitable methods for exploring limits and advantages in AT service delivery systems. The study confirms the relevance of person centredness for a successful AT assessment and delivery process.
Background: This study was an extension of research which began in the Umbria region in 2009. Aim: To investigate the extent to which assistive technology (AT) has been abandoned by users of the Italian National Health Service (ULHS) and the reasons for this. Design: Observational study. Setting: Users who received a hearing device (HD) or mobility device (MD) by ULHS between 2010 and 2013. Population: 749 out of 3,791 ULHS users contacted via telephone completed the interview: 330 (44.06%) had a HD and 419 (55.94%) a MD. Methods: Data were collected using a specially developed telephone interview questionnaire including the Italian version of the Quebec User Evaluation of Satisfaction with AT (QUEST 2.0) and Assistive Technology Use Follow-up Survey (ATUFS). Results: 134 users (17.9%) were no longer using their assigned AT device within seven months of issue and 40% of this group reported that they had never used the device. Duration of use (for how long the AT device was used before abandonment) and satisfaction with service delivery did not predict AT abandonment. People who received a HD where more likely to abandon their device (22.4%) than those who received a MD (14.4%). Conclusions: Abandonment may be due to assignment of inappropriate devices or failure to meet user needs and expectations. These findings are consistent with previous data collected by Federici and Borsci in 2009. Utility of AT in use, reasons of abandonment, and importance of device and service satisfaction for the use or non-use of an AT are presented and discussed. Clinical rehabilitation impact: AT abandonment surveys provide useful information for modelling AT assessment and delivery process. The study confirms the relevance of person centredness approach for a successful AT assessment and delivery process.