Article

Measuring Desirability: New methods for evaluating desirability in a usability lab setting

Authors:
Article

Measuring Desirability: New methods for evaluating desirability in a usability lab setting

If you want to read the PDF, try requesting it from the authors.

Abstract

Difficulty can arise when a practitioner wants to get user input on intangibles such as "desire" and "fun" in a usability lab setting. This paper will introduce you to methods we've created to collect feedback on "desirability" and give some background on how we developed them.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Although the GDPR defines several potential legal bases 1 for the lawful personal data processing 2 , for instance for the provision of a contract, in order to fulfill a legal obligation, in the case of vital interest, in the case of public interest, or for reasons of legitimate interest, in many cases data controllers and processors, will need to obtain consent from data subjects for the processing of their personal data 3 , for example in order to deliver personalized recommendations or to improve their services. According to Art. 4 (11) 4 of the GDPR, consent needs to be "freely given, specific, informed and unambiguous indication of the data subject's wishes by which he or she, 1 GDPR Art. 6(1)(b -f) 2 For the lawful personal data processing data subject's consent is not required. ...
... 3 GDPR Art. 6(1)(a) 4 Art. 4 (11) is complemented by Art. 7 that provides information on conditions for consent. by a statement or by a clear affirmative action, signifies agreement to the processing of personal data relating to him or her". ...
... The users were asked to select adjectives that they would use to describe the UI they were testing. We used the list of adjectives from Microsoft Desirability Toolkit [4], which we adapted to our case. The adjectives that were selected support the results described above. ...
Conference Paper
Although the General Data Protection Regulation (GDPR) defines several potential legal bases for personal data processing, in many cases data controllers, even when they are located outside the European Union (EU), will need to obtain consent from EU citizens for the processing of their personal data. Unfortunately, existing approaches for obtaining consent, such as pages of text followed by an agreement/disagreement mechanism, are neither specific nor informed. In order to address this challenge, we introduce our Consent reqUest useR intErface (CURE) prototype, which is based on the GDPR requirements and the interpretation of those requirements by the Article 29 Working Party (i.e., the predecessor of the European Data Protection Board). The CURE prototype provides transparency regarding personal data processing, more control via a customization, and, based on the results of our usability evaluation, improves user comprehension with respect to what data subjects actually consent to. Although the CURE prototype is based on the GDPR requirements, it could potentially be used in other jurisdictions also.
... In the exit questionnaire, we asked users to send their collected data. We also asked about their satisfaction with each linting category using a Likert scale, and their overall satisfaction with the tool using both a System Usability Scale Questionnaire (Brooke 1996) and Microsoft Reaction Cards (Benedek and Miner 2002). Finally, we asked for suggestions to improve Julynter. ...
... On the other hand, the maximum score was 100, which is categorized as the best imaginable score (Bangor et al. 2008). Figure 25 presents a word cloud representing the Microsoft Product Reaction Cards (Benedek and Miner 2002) we presented in the exit questionnaire. The most selected cards are usable and useful. ...
... This may bias the selection of participants to close contacts. In an attempt to mitigate this threat, we distributed the invitation for Phase III to public data science (Benedek and Miner 2002). The colors vary according to the experiment phase in a gradient. ...
Article
Full-text available
Jupyter Notebooks have been widely adopted by many different communities, both in science and industry. They support the creation of literate programming documents that combine code, text, and execution results with visualizations and other rich media. The self-documenting aspects and the ability to reproduce results have been touted as significant benefits of notebooks. At the same time, there has been growing criticism that the way in which notebooks are being used leads to unexpected behavior, encourages poor coding practices, and makes it hard to reproduce its results. To better understand good and bad practices used in the development of real notebooks, in prior work we studied 1.4 million notebooks from GitHub. We presented a detailed analysis of their characteristics that impact reproducibility, proposed best practices that can improve the reproducibility, and discussed open challenges that require further research and development. In this paper, we extended the analysis in four different ways to validate the hypothesis uncovered in our original study. First, we separated a group of popular notebooks to check whether notebooks that get more attention have more quality and reproducibility capabilities. Second, we sampled notebooks from the full dataset for an in-depth qualitative analysis of what constitutes the dataset and which features they have. Third, we conducted a more detailed analysis by isolating library dependencies and testing different execution orders. We report how these factors impact the reproducibility rates. Finally, we mined association rules from the notebooks. We discuss patterns we discovered, which provide additional insights into notebook reproducibility. Based on our findings and best practices we proposed, we designed Julynter, a Jupyter Lab extension that identifies potential issues in notebooks and suggests modifications that improve their reproducibility. We evaluate Julynter with a remote user experiment with the goal of assessing Julynter recommendations and usability.
... " At this point, we engaged into a discussion first asking what their first impression of the product was. Then, we used the Product Reaction Cards [4], which are part of the Desirability toolkit by Benedek and Miner. Microsoft's product reaction cards are a customizable list of 118 words used to evaluate any product or concept. ...
... People are asked to choose words from the list, that would in their opinion describe the product ( Figure 10). This constitutes an ideal base for discussion in an interview setting [4]. We used the reduced list of 64 words by Neil Turner [59]. ...
... After being briefly explained what to expect from Laina, P1 immediately expressed curiosity: "I'm very curious what's going to come out, what's going to happen if I have done a run". When being asked to choose some cards from the Product Reaction Cards [4] to describe their first impression, the words chosen at least twice were powerful, simplistic, innovative, inviting and attractive. Powerful: "It really has a presence, if it was smaller you would be able to look past it or forget about it" -P2. ...
... In the questionnaire the users were prompted to select adjectives that they would use to describe the UI prototype. We used the list of adjectives from Microsoft Desirability Toolkit, developed by Joey Benedeck and Trish Miner [3]. Since the original list consists of large amount of words, it is recommended to shorten and adapt the list, which we did in our usability evaluation. ...
... The online version of the prototype is available at: https://bit.ly/2Z1yrKs.3 The online version of the prototype is available at: https://bit.ly/2U6TkQw. ...
Conference Paper
The General Data Protection Regulation (GDPR) requires, except for some predefined scenarios (e.g., contract performance, legal obligations, vital interests, etc.), obtaining consent from the data subjects for the processing of their personal data. Companies that want to process personal data of the European Union (EU) citizens but are located outside the EU also have to comply with the GDPR. Existing mechanisms for obtaining consent involve presenting the data subject with a document where all possible data processing, done by the entire service, is described in very general terms. Such consent is neither specific nor informed. In order to address this challenge, we introduce a consent request (CoRe) user interface (UI) with maximum control over the data processing and a simplified CoRe UI with reduced control options. Our CoRe UI not only gives users more control over the processing of their personal data but also, according to the usability evaluations reported in the paper, improves their comprehension of consent requests.
... Some of the methodologies focus on a narrow part of the process, aiming to generate innovative solutions and produce a great number of ideas and concepts [3,4]. These include methods for exchanging knowledge in multidisciplinary product development teams [5], solving ergonomic design and usability issues [6], examining the aesthetic aspects of the product [7], or focusing on the design of particular objects and demands such as soft wearable devices, sports equipment, or emergency ventilators [8][9][10]. Other methods take a more holistic approach and observe the process throughout its length and breadth [11]. ...
... Benedek and Miner [7] developed a method to check the emotional response and desirability of a design or product called "Microsoft reaction cards." They created a desirability Toolkit, using 118 cards with different product-reaction words and phrases written on each one representing a broad spectrum of options and dimensions of desirability (for example, accessible, annoying, boring, exciting, familiar, etc.). ...
Article
Full-text available
The overall goal of the novel Functionality, Usability, Look-and-Feel, and Evaluation (FULE) user-centered methodology for product design proposed in this paper is to develop usable and aesthetic products. Comprising several product design methods, this novel methodology we devised focuses on the product designer’s role and responsibility. Following the first three formative assessment phases that define the product’s functionality, usability, and look-and-feel, the summative evaluation phase not only assesses the product, but also provide guidelines to its implementation, marketing, and support. A case study devoted to the design of an autonomous medical device illustrates how the FULE methodology can provide the designer with tools to better select among design alternatives and contribute to reducing bias and subjective decisions.
... The second post-questionnaire corresponds to the 118-word test (Microsoft Product Reaction Cards) [58]. This test was developed as part of a "desirability toolkit" created to evaluate the aspect of desirability resulting from a user's experience with a product, by choosing descriptive words or multi-word expressions from 118 reaction cards [59]. ...
... The 118 words/multi-word expressions used in the 118-word test[58,62]. The words we considered as positive, neutral, and negative, in the context of our home assistant, are indicated using green, black, and red, respectively. ...
Article
Full-text available
Our homes are becoming increasingly sensorized and smarter. However, they are also becoming increasingly complex, making accessing them and their advantages difficult. Assistants have the potential for improving the accessibility of smart homes, by providing everyone with an integrated, natural, and multimodal way of interacting with the home’s ecosystem. To demonstrate this potential and contribute to more environmentally friendly homes, in the scope of the project Smart Green Homes, a home assistant highly integrated with an ICT (Information and communications technology) home infrastructure was developed, deployed in a demonstrator, and evaluated by seventy users. The users’ global impression of our home assistant is in general positive, with 61% of the participants rating it as good or excellent overall and 51% being likely or very likely to recommend it to others. Moreover, most think that the assistant enhances interaction with the smart home’s multiple devices and is easy to use by everyone. These results show that a home assistant providing an integrated view of a smart home, through natural, multimodal, and adaptive interaction, is a suitable solution for enhancing the accessibility of smart homes and thus contributing to a better living ambient for all of their inhabitants.
... Studies have found that even the color alone has a psychological effect on the user by triggering the human arousal system and affecting product perception and trust in the product [26,27]. Designers use the word desirability to describe how attractive the visibility of a product is and whether the product is perceived as "worth having or seeking" and as being useful, advantageous, or pleasing [28]. ...
... Perception of visual appearance can be measured in different ways; ascribing product reaction words to different designs [28], using Semantic Differential scales (SD scales) where respondents are asked to indicate their position on a scale between two bipolar words or using Likert scales where subjects are asked to state their agreement with different statements. In the world of robotics, two common questionnaires are used for the evaluation of users' perceptions: Godspeed [64] and Robotic Social Attributes Scale (RoSAS) [65]. ...
Preprint
Full-text available
Human-SAR (socially assistive robot) relationships vary by context of use and interaction level. We argue that context and interaction considerations must be incorporated into the SAR’s physical design requirements to align the robotic visual qualities (VQs) with users’ expectations. We propose to consider situational-based and dynamics-based human-SAR relationship models in constructing the requirements. Previous studies contributed to the understanding of users` perceptions and preferences regarding existing commercially available SARs. Yet, very few studies regarding SARs’ appearance used designated SAR designs, and even fewer evaluated isolated visual features. In this work, we aim to systematically assess the effect of isolated VQs. To achieve this, we first deconstruct the VQs attributed to SARs. Then, a reconstruction of body structure, outline, and color scheme was done, resulting in the creation of 30 new SAR models that differ in their VQs, allowing us to isolate one character at a time. We used these new designs to evaluate users’ preferences and perceptions in two empirical studies. Our empirical findings link visual qualities with perceptions of SAR characteristics. Together with the relationship models, the outcomes are an exemplar of how to form guidelines for the industrial design processes of new SARs to match user expectations.
... Usability "The collapsible menu configuration can encourage positive versatile encounters on both Android and iPhone gadgets, anyway the breadcrumb plan possibly increasingly reasonable for more extensive portable screens" [37]. Satisfaction "Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" [39]. Navigation Navigation allows the user to move from one page to another through either buttons, text, or an image in a flexible and easy way; sometimes, it contains animation [17]. ...
... The data collection procedure includes the sampling technique and the targeted population. Studies have shown that questionnaires are a valuable technique for cross-platform usability engineering when evaluating the cross-platform user experience [5,10,14,35,39,42]. We designed this study to capture the opinions and impressions of end-users. ...
... After exposure to the three conditions the participants were asked to fill out the same questionnaire used for S1. Before the focus group interviews, we administered an adapted version of the Microsoft Desirability Toolkit (MSDT) [3]. Because we administered these product reaction cards with the aim of exploring the participants' aesthetic experience, we removed all words that were only relevant for functionality or performance, as recommended by Moran [14]. ...
... was assessed using a nine-item, seven-point semantic differential scale (e.g. "not worth having" = 1 to "worth having" = 7) based on Benedek and Miner (2002). We manipulated Instagram content priming condition by showing participants in the Instagram content priming condition a screenshot of the object embedded in an Instagram frame prior to viewing the online shop stimulus. ...
Article
Marketing on social media has become ubiquitous. Consequently, social media platforms are increasing the level of advertising content that users may later encounter when navigating online shopping websites. It is unclear how this amplification of exposure to marketing messages through social media affects consumers’ attitudes to products online. Furthermore, the roles of social media participation and proneness to experience Fear of Missing Out on product attitude remain largely unexplored. In this research ( N = 1002), we employed an online survey of US Instagram users. These data were submitted to three-way moderation regression analyses with attitude toward the product as the dependent variable. Consumers who are more active on social media and had high (vs low) Fear of Missing Out expressed more favorable attitudes toward online products after being exposed to Instagram content (vs not exposed). The theoretical and practical implications for cognitive processing research and advertising strategy and study limitations are discussed.
... The IPQ is divided into subscales, measuring levels of General Presence (GP), Spatial Presence (SP), Involvement (INV), and Experienced Realism (REAL). Additionally, the Product Reaction Card (PRC) evaluation from the Microsoft Desirability Toolkit was used [4]. Users select words, each written on slips of paper, that describes feelings matching the experience they had in the VE. ...
Conference Paper
Full-text available
This paper explores the potential of using a virtual reality application to support pregnant individuals strengthen their parental bonds. The application simulates sensory development from a fetal point-of-view inside a virtual womb. More than 20 healthcare professionals comprising midwives, doctors, and nurses assisted the iterative design process through meetings, exposure, and open discussion. The evaluation used the iGroup Presence Questionnaire and Microsoft Product Reaction Cards to assess user presence and emotional impact. 48 participants (19 females, 29 males) comprising pregnant individuals, healthcare professionals, and miscellaneous participants took part in the evaluation. Results indicate feelings of presence, significant emotional impact, as well as positively inclined responses towards the experience. Healthcare professionals supported the evaluation, saw application potential, and expressed interest in further development and implementation.
... The participants also selected three Product Reaction Cards (PRCs) that they thought mostly described CHECKSUM. The participants were given a subset of the original 118 PRCs [11], which included 10 positive terms and 10 negative ones. The positive terms included the following: appealing, desirable, easy to use, effective, empowering, exciting, friendly, straightforward, useful, and valuable. ...
Article
Full-text available
Models are often used to represent various types of systems. This is especially true for software systems, where cooperating teams create models using a modeling language (e.g., UML). In cooperative modeling scenarios, it is useful to identify contributions and changes performed by individuals and teams. This paper presents a technique called CHECKSUM, which monitors the cooperative work done on models and maintains an immutable changelog. CHECKSUM uses its changelog to measure contributions based on points, time, and quality, and to enable the auditing of a model’s change-history. This paper also presents GEneric Meta-Model (GEMM). The latter unifies the underlying representation of different types of models that follow varying visualization patterns including box and line, container, and interleaving. GEMM enables CHECKSUM to support an extensible variety of model types. We developed a prototype tool that realizes CHECKSUM’s concepts and integrates it into two existing modeling tools. We conducted two studies to evaluate CHECKSUM from two perspectives: technical and user. The studies yielded positive results concerning various qualities including integrability into existing tools, effectiveness, efficiency, usability, and usefulness.
... Questions and topics were related to ease of use, understandability of the user manual, positive/negative reactions, loyalty and overall satisfaction. The reaction cards are a collection of words from which the subjects can choose (e.g., too technical, time-saving, stressful, efficient, high quality, etc.) that have been used successfully to measure subjective satisfaction [25]. The subjects had the opportunity to choose five keywords that they associated with the test approaches. ...
Chapter
TESTAR is a traversal-based and scriptless tool for test automation at the Graphical User Interface (GUI) level. It is different from existing test approaches because no test cases need to be defined before testing. Instead, the tests are generated during the execution, on-the-fly. This paper presents an empirical case study in a realistic industrial context where we compare TESTAR to a manual test approach of a web-based application in the rail sector. Both qualitative and quantitative research methods are used to investigate learnability, effectiveness, efficiency, and satisfaction. The results show that TESTAR was able to detect more faults and higher functional test coverage than the used manual test approach. As far as efficiency is concerned, the preparation time of both test approaches is identical, but TESTAR can realize test execution without the use of human resources. Finally, TESTAR turns out to be a learnable test approach. As a result of the study described in this paper, TESTAR technology was successfully transferred and the company will use both test approaches in a complementary way in the future.
... In particular, we asked participants about: (1) using a similar environment in a real scenario, (2) using services composed by others when creating theirs, (3) share their services with others; (4) reasons for sending following requests; and (5) the usefulness of the recommendation algorithms. Furthermore, we also included some questions to apply the Microsoft Product Reaction Cards [49] in order to evaluate the enduser satisfaction level. This method consists of providing participants with a list of words and asking them to choose the words that they would use to describe a product. ...
Article
Full-text available
Nowadays, end users are surrounded by plenty of services that are somehow supporting their daily routines and activities. Involving end users into the process of service creation can allow end users to benefit from a cheaper, faster, and better service provisioning. Even though we can already find tools that face this challenge, they consider end users as isolate individuals. In this paper, we investigate how social networks can be used to improve the composition of services by end users. To do so, we propose a graph-based definition of a social structure, and analyse how social connections can be exploited to both facilitate end users to discover services through browsing these connections, and recommend services to end users during the composition activity. As proof of concept, we implement and evaluate the proposed social network in the context of EUCalipTool, a mobile end-user environment for composing services.
... Participants were asked to provide positive and negative comments for each slider, as well as a possible use case or context. • Providing feedback on the interaction experiences, using the same 5 attributes as in the expectations, for each slider • For each slider, selecting 5 words that best described the experience from a list of 20 possible terms, (Following the approach of [6], and adopted from the Product Reaction Cards (PRC) method [1]). • Selecting a favorite slider A, B or C After Task 1, the participants continued to Task 2: ...
Conference Paper
Full-text available
We investigate an unconventional user interface (UI) material, water, and how applying it to a finger-touch based input affects the user experience (UX). We compare the use of touchscreen, mechanical and liquid covered sliders, and three different approaches of how water is applied to the surface of the slider. The salient findings of our user study (n=25) show that liquid changes the perception of the slider control making it more hedonic and experiental. When interacting with water, users prefer larger and more free-form gestures, and factors such as depth and temperature of the liquid affect the experience. Our work provides information for interaction designers focusing on user experience, unconventional materials for interactive systems, aesthetics, and calm UIs.
... We collected data from a mixed method approach from playtesting, a method used to evaluate subjective fun, and reaction cards to capture sentiment [1,7,13]. The session structure during the 1.5 hours was as follows; 5-minute introduction, 25-minute benchmark tasks, 25-minute game, 25-minute autonomy, and 10-minute final assessment. ...
Conference Paper
Full-text available
Personality of Artificial Intelligence (A.I) is a vast and complex domain. Traits like trustworthiness and humanness have long been evaluated, but less so for traits like fun. In this study we evaluate fun in four conversational agents with a mix of menial tasks, games, and free play in the US and India on voice and text-based platforms. Our research explores user sentiment of fun in an A.I. personality, implications of fun personality on voice-based and text-based agents, and the perception of a fun A.I. personality in two culturally distinct markets. We used a mixed method approach to evaluate these objectives with 44 participants. Our results end with design principles for creating fun in artificial intelligence: 1) balance fun through conversational humor and goal-oriented actions, 2) create fun experiences through playful interactions, 3) convey fun through platform specific expressions like tone of voice or visual gestures and emojis, and 4) be inclusively fun by being culturally relevant so your conversational agent translates across borders.
... 3. Afterwards, we measured the satisfaction level of participants. To do so, we used the Microsoft Product Reaction Cards (Benedek and Miner 2002). This method consists on providing participants with a list of 118 words, and asking them to choose the words that they would use to describe a product (we limited the number of words to be selected to 5 to keep the exercise short). ...
Article
Full-text available
Nowadays, we live surrounded by heterogeneous and distributed services that are available to people anytime and anywhere. Even though these services can be used individually, it is through their synchronized and combined usage that end-users are provided with added value. However, existing solutions to service composition are not targeted at ordinary end-users. In fact, these solutions require technical knowledge to deal with the technological heterogeneity in which they are offered to the market. To this end, the paper presents a tool-supported platform that is aided by: (1) EUCalipTool, an end-user mobile tool that implements a Domain Specific Visual Language, which has been specifically designed to compose services on mobile devices; (2) a Faceted Service Registry, which plays the role of gateway between service implementations and end-users, hiding technological issues from the latter when including services in a composition; and (3) a Generation Module, which transforms end-user descriptions into BPMN specification that are interpreted by an execution infrastructure developed for that purpose.
... The qualitative evaluation was based on a product reaction cards (PRC) evaluation method from the Microsoft Desirability Toolkit (Benedek and Miner, 2002) followed by a semi-structured interview. For the PRC, from a list of 118 available words, participants were instructed to select five words that described their experience in the VE. ...
Article
Full-text available
Cochlear implants (CI) enable hearing in individuals with sensorineural hearing loss, albeit with difficulties in speech perception and sound localization. In noisy environments, these difficulties are disproportionately greater for CI users than for children with no reported hearing loss. Parents of children with CIs are motivated to experience what CIs sound like, but options to do so are limited. This study proposes using virtual reality to simulate having CIs in a school setting with two contrasting settings: a noisy playground and a quiet classroom. To investigate differences between hearing conditions, an evaluation utilized a between-subjects design with 15 parents (10 female, 5 male; age M = 38.5, SD = 6.6) of children with CIs with no reported hearing loss. In the virtual environment, a word recognition and sound localization test using an open-set speech corpus compared differences between simulated unilateral CI, simulated bilateral CI, and normal hearing conditions in both settings. Results of both tests indicate that noise influences word recognition more than it influences sound localization, but ultimately affects both. Furthermore, bilateral CIs are equally to or significantly beneficial over having a simulated unilateral CI in both tests. A follow-up qualitative evaluation showed that the simulation enabled users to achieve a better understanding of what it means to be an hearing impaired child.
... Toolkit (Benedek & Miner, 2002). In addition, these words can be used as a basis for developing 438 more robust or more precise instruments for measuring trust (Li, Alsaid, Noejovich, Cross, & Lee, 439 2020). ...
Preprint
Full-text available
Trust has emerged as a prevalent construct to describe relationships between people and between people and technology in myriad domains. Across disciplines, researchers have relied on many different questionnaires to measure trust. The degree to which these questionnaires differ has not been systematically explored. In this paper, we use a word-embedding text analysis technique to identify the differences and common themes across the most used trust questionnaires and provide guidelines for questionnaire selection. A mapping review was conducted to identify the existing trust questionnaires. In total, we included 46 trust questionnaires from three main domains (i.e., Automation, Humans, and E-commerce) with a total of 626 items measuring different trust layers (i.e., Dispositional, Learned, and Situational). Next, we encoded the words within each questionnaire using GloVe word embeddings and computed the embedding for each questionnaire item, and for each questionnaire. We reduced the dimensionality of the resulting dataset using UMAP to visualize these embeddings in scatterplots and implemented the visualization in a web app for interactive exploration of the questionnaires (link to app). At the word level, the semantic space serves to produce a lexicon of trust-related words. At the item and questionnaire level, the analysis provided recommendation on questionnaire selection based on the dispersion of questionnaires’ items and at the domain and layer composition of each questionnaire. Along with the web app, the results help explore the semantic space of trust questionnaires and guide the questionnaire selection process. The results provide a novel means to compare and select trust questionnaires and to glean insights about trust from spoken dialog or written comments.
... We conducted an online survey among 127 participants. For each scenario, we asked the participants to choose a selection of five words based on the Product Reaction Cards [1]. Following the characteristics of AoF, we envisioned that an AoF design should be associated with both positive and negative feelings. ...
... Participants were asked to justify their answer. For each infographic they were also asked to choose 3 words from a list of 10 negative and 10 positive words to describe their opinion on the design; based on the Microsoft Desirability Toolkit (Benedek and Miner, 2002). ...
Article
Full-text available
Infographics are becoming a common tool in the communication of public-health information. However, research-based resources in how to create effective infographics are rare. The application of design principles in the creation of infographics has been shown to more effectively communicate information. Here, the research explores the adherence of 84 research-based infographic design principles on 3 designs of varying levels of application. A multi-method approach, including eye-tracking, was used to record information location efficiency, memorability and user perception. Support was found in favour of utilising design principles in the creation of public health infographics; resulting in improved user opinion and information location.
... At the end of the tasks execution, a user experience evaluation activity was planned, in order to gather qualitative feedback and learn more about the user's perception of the experience. The planned activity is inspired by Microsoft's Product P r e p r i n t Reaction Cards method [29]. It is adapted to visually impaired users by offering objects to touch, rather than verbal descriptions, to describe the browsing experience. ...
Chapter
The growing diffusion of fashion e-commerce websites shows the appreciation by users, highlighting the importance of offering this service also to users with different disabilities. To this end, e-commerce should be not only accessible - implementing all the technical requirements for accessibility - but also usable, paying attention to the offered user experience. This study aims to investigate the current e-commerce usability considering visually impaired users’ navigation experience and understand which aspects define a good usability level for this target. An expert analysis of a set of fashion e-commerce websites and user testing were conducted, considering five different market segment categories. The analysis highlighted a gap in the consideration of visually impaired users’ navigation needs and style, as for instance non-uniformity of layout and page structure. All the findings have been structured in usability guidelines to favor the e-commerce usability improvement, with the goal of offering visually impaired users a better shopping experience.
... The reason for selecting keyword descriptive terms was due to literature encapsulating specific characteristics about travellers' information-seeking behaviour into similar terms 12 (e.g. Schmitt et al., 2013, Schmitt et al., 2015 and that other literature uses keywords terms as a consistent approach for measuring the desirability of information systems from the user's perspective (Benedek and Miner, 2002). This meant that each source was reviewed according to a thematic analysis: (1) familiarisation with contents produced from each method 13 described separately; ...
Thesis
Every day, UK towns and cities are full of different traveller types, each with a different level of exposure to the environment around them, which is made up of the local landscape and transport system operations. For example, the travellers that are most comfortable with the information around them and can respond accurately to it are those who travel regularly and are familiar with the environment. However, other travellers unfamiliar with these things will need a little more help in understanding it and the available information relating to it. In some cases, service providers and key stakeholders turn to external information systems to resolve pressure points caused by a lack of local familiarity. These include journey planners used to help travellers gain an understanding of day-to-day operations within that landscape. These external systems focus on distributing the available travel information and not on the user’s primary travel information needs relating to their journey. Little research has addressed how travel information should be presented to travellers to inform effective action. In essence, giving an individual access to relevant information and advice means the production of accurate travel plans that correctly match the local landscape and transport operations in a clear and understandable way. This thesis established a broad view of the different traveller type personas based on their level of familiarity and the stages of information use. That knowledge was captured in a Traveller Planning Types (TPT) framework conceptualised through a triangulation study comprised of a contextual review, focus groups and a literature review collated using thematic analysis. The TPT framework was confirmed as an appropriate framework using a Delphi study of these key stakeholders of external information provision systems. The TPT framework was then used to establish a new method of measuring pre-trip travel information needs for familiar or unfamiliar journey planning situations using probing and process observation techniques using a screen monitoring system. The results show that there is a statistically significant difference in how travellers feel when pre-planning a familiar or unfamiliar journey, especially with regards to translating that information into accurate travel plans and the confidence to conduct the pre-planned journey. The research has identified that pre-planning travel information has yet to meet the standards set by the fourth rule of citizenship. Specific strategic guidelines were developed to guide future development of such external information systems to take into account user’s travel information needs.
... Product reaction words, such as 'fun' and 'calm' describe intangible emotional response towards the interface. 33 Lastly, participants will share their likes and dislikes of the Voxe platform design and comment on the platform's ease of use and elements of functionality during a virtual semi-structured qualitative interview. Semi-structured interviews foster reciprocity between the participant and interviewer, allow the interviewer to ask pertinent follow-up questions to elicit rich data and enable the participant to express themselves using their own words. ...
Article
Full-text available
Introduction Patient-reported outcome measures (PROMs) provide an opportunity for meaningful patient engagement and shared decision-making. The objective of this research programme is to improve health outcomes for paediatric solid organ transplant patients by implementing PROMs into clinical care. The current study aims to create Voxe, a paediatric user-centred electronic PROM platform, by engaging patients and healthcare providers throughout the design and development process. Methods and analysis The creation of Voxe will occur over two phases that build on previous research. The user interface design phase employs a ‘user-centric’ approach to identify end-users’ needs and iteratively refine the look and layout of Voxe to meet these needs. Transplant recipients, aged 10–17, and healthcare providers will participate in three rounds of testing (24 participants total). Participants will: (1) complete task-based activities (outcomes—effectiveness and efficiency), (2) complete questionnaires (outcome—satisfaction) and (3) participate in a semi-structured interview. The following phase involves software development and Voxe usability testing. Transplant recipients, aged 8–17, and healthcare providers will participate in four rounds of iterative testing (24–40 participants total). The think-aloud technique will be employed, and participants will describe their thoughts and feelings while interacting with a Voxe prototype. Participants will: (1) log into Voxe and complete tasks (outcomes—time on task, successful task completion, frequency of critical and non-critical errors and error-free rate), (2) complete questionnaires (outcome—satisfaction) and (3) participate in a semi-structured interview. Findings will result in the creation and launch of a user-centred electronic PROM platform. Ethics and dissemination Research ethics board approval has been provided by The Hospital for Sick Children. This research is critical to answering methodological and operational questions to inform Voxe implementation in paediatric clinical settings and facilitate PROM data collection. Future investigations will include an implementation-effectiveness evaluation.
Chapter
The present study investigates the affordance of vibrotactile signals in a simulated haunted house. Participants experienced a virtual séance using a head-mounted display, sound, and haptic stimuli on the palm and thighs. In one condition, six unique, handcrafted haptic signals (cicadas, frog, thunder, earthquake, heartbeat, knock) were presented alongside appropriate events in the narrative. In another condition, a single multiplexed stimulus was presented for every event; this signal was a composite of the six distinct signals. Adjective ratings were collected for both conditions for each participant. Results showed that the extent to which a haptic signal enhanced the sense of immersion depended on the match between the signal and the natural phenomenon it represented. The unique, handcrafted signals generally were rated as more immersive than the multiplexed signal. However, the signal cicadas had a distinct spectral signature that stood out in the multiplexed signal. Participants rated the distinct cicadas signal and the multiplexed signal as similarly immersive. Our results demonstrate that carefully handcrafted vibrotactile signals can enhance the sense of immersion in virtual reality. Furthermore, participants may rate a haptic signal as more immersive if it contains features congruent with the natural event it represents, regardless of extraneous, incongruent features.
Article
Full-text available
Long-distance romantic relationships have become prevalent in today's globalized world. Although mainstream communication technologies have provided instant, cheap, and convenient channels for people to communicate at a distance, the emphasis of these technologies is placed on functionality as they are designed for a large variety of end users, rather than providing emotional communication which the author argues is meaningful and necessary for the individuals to maintain ties to their romantic partners who are forced to live physically apart for some reason. The author envisions that there is a gap between understanding the users' needs in research and designing technologies for them in practice. The author's PhD research has been dedicated to bridging this gap by mediating emotional communication for serious long-distance romantic relationships through unconventional artefacts. This manuscript presents an overview of the work thus far.
Article
This work contributes a research protocol for evaluating human-AI interaction in the context of specific AI products. The research protocol enables UX and HCI researchers to assess different human-AI interaction solutions and validate design decisions before investing in engineering. We present a detailed account of the research protocol and demonstrate its use by employing it to study an existing set of human-AI interaction guidelines. We used factorial surveys with a 2x2 mixed design to compare user perceptions when a guideline is applied versus violated, under conditions of optimal versus sub-optimal AI performance. The results provided both qualitative and quantitative insights into the UX impact of each guideline. These insights can support creators of user-facing AI systems in their nuanced prioritization and application of the guidelines.
Article
This study looks at the effects on users’ choices when presented with Product Reaction Cards (PRC) in parallel and sequential order. Participants were given brief tasks on two websites and presented with PRC in parallel or sequential order to describe their sentiments. We found that participants selected 25% more words in the sequential condition, including the selection of more positive words (23%). However, the sequential condition took on average 5 minutes longer to complete word selection. Therefore, it is important to understand that the PRC presentation modality can affect the quantity and the choice of vocabulary used by participants.
Conference Paper
Full-text available
RESUMO Este artigo tem como objetivo descrever o processo de design interativo centrado no usuário de componentes de aprendizagem para professores na área de educação matemática, especificamente das estruturas aditivas. Foram abordados os problemas do relacionamento com especialistas no conteúdo de problemas envolvendo combinação, assim como a composição multidisciplinar das equipes: roteiristas, designers, desenvolvedores, especialistas em áreas de estudo e em usabilidade. Este processo mescla práticas consolidadas de prototipagem rápida em papel e recursos inovadores, como a introdução da figura do narrador em testes de usabilidade e técnicas para avaliação da desirability. Neste estudo estão descritos o modelo de processo criado, uma breve descrição das técnicas utilizadas e algumas experiências adquiridas durante o estudo. O objetivo é servir de base para outros projetos de inovação industrial na área de multimídias e novas mídias para formação de professores, em especial, de matemática. Palavras-chave Prototipagem de software educativo, Design instrucional, Estruturas aditivas.
Preprint
BACKGROUND There is an increasing need to organise the care around the patient and not the disease, as well as taking into account the complex realities of multiple physical, psycho-social conditions and polypharmacy. Integrated patient-centred care delivery platforms have been developed for both patients and clinicians. These platforms could provide a promising way to achieve a collaborative environment that improves the provision of integrated care for patients via enhanced ICT solutions. OBJECTIVE The C3-Cloud project has developed two collaborative computer platforms for patients and members of the Multi-Disciplinary Team and deployed these in three different European settings. The objective of this study is to pilot test the platforms and evaluate their impact on patients, informal caregivers, healthcare professionals and, in extend, healthcare systems. METHODS This paper describes the protocol for conducting an evaluation of the user-centred design, user experience, acceptability, and usefulness of the platforms. For this, four ‘testing and evaluation’ phases have been defined, involving multiple qualitative methods, and advanced impact modelling. RESULTS The technology trial in this 4-year funded project (2016-2020) is currently in its execution phase. The testing and evaluation phase 1 and 2 have been completed with satisfying results on system component tests, and promising results on application and usability tests. The pilot technology trial for evaluation phase 3 and 4 was launched in August 2019. Data collection for these phases is underway and results are forthcoming, approximately in April 2020. We believe that the phased, iterative approach taken is useful as it involves relevant stakeholders at crucial stages in the platform development and allows for a sound user acceptance assessment of the final product. CONCLUSIONS Patients with multiple chronic conditions often experience shortcomings in the care they receive. It is hoped that personalised care plan platforms for patients and collaboration platforms for members of Multi-Disciplinary Teams can help to tackle the specific challenges of clinical guideline reconciliation for multimorbid patients and improved the management of poly-pharmacy. The initial evaluative phases have indicated promising results of platform usability. The phased methodology has shown useful results in the first two phases, while results of phase 3 and 4 are pending. CLINICALTRIAL https://www.clinicaltrials.gov/ct2/show/NCT03834207
Chapter
Keyboard shortcuts have been proven to be the most efficient method of issuing commands in computer software. Using the mouse in graphical user interfaces provides an intuitive but slow method for executing functions in a software. Many fail to make the transition to faster modalities, such as keyboard shortcuts. This is not just the case for novices, but also users with years of experience. This study examines the research on this behavior, as well as how the concepts of nudging and ambient suggestion can be used to actively encourage and support the usage of keyboard shortcuts. Based on this research, a design is proposed and implemented in a simple word processor application. A user experience evaluation was done, by having participants perform writing and formatting tasks inside the application. Using the Microsoft Reaction Card Method followed by a semi-structured interview, the users elaborated on their experience. In the interviews topics and questions of motivation, distraction, and annoyance were raised. The results showed that most participants found the system convenient and helpful in learning shortcuts without being too obtrusive. There are promising first indications of it having potential in promoting the usage of keyboard shortcuts, however further research is required in order to make any generalizations.
Chapter
In this paper, we present a non-light-emitting electrochromic ambient display, Our Little Secret, for supporting the communication in long-distance relationships (LDRs). The unobtrusive display technique was prototyped with an interactive picture frame, which was used as a probe to chart the possibilities of the technique in an interview-based user study (n = 12) and an in-the-wild deployment (one couple) of people living remotely from their partner. The salient findings showed positive response especially on the non-light-emitting nature of the display, and indicates that this type of solution has the potential to support LDR communication and relationship through a pair of private, meaningful, and always-on yet calm displays.
Chapter
Design teams should not consider a person’s computer interface as just a set of hardware and software that make upcompose the computer (Grudin, 1990). Instead, from the person’s point of view, the interface includes all of the elements which compose its context of use. From this perspective, one can view the interface of a computer as more than the screens, buttons, and knobs, and also include any documentation, other people present, and past knowledge directly relevant to the situation. In these situations, HII needs to consider more than just the button-pushing or system response, but how people and information interact with the situation in building comprehension.
Article
A storyteller panorama tour, The Story of the Markam Car Collection, was developed as an example for museums and cultural institutions concerning the use of panoramas combined with multimedia to tell stories of specific large objects (antique cars). It was designed for multiple platforms to involve and engage audiences via large curved screens while allowing for access via portable devices. Understanding users’ requirements is essential when designing and developing virtual museum tours. Measuring digital productions’ usability is an ongoing challenge that affects the improvement of user experience and the actual output. A variety of techniques and approaches are available to test digital productions’ usability and the related user experience. It is possible to measure and evaluate the production’s usefulness, including users’ engagement and understanding. However, the best method for assessing museum-related digital productions varies depending on aims, capacity, audience, and local context. In this paper, we demonstrate the strategy we employed to evaluate the particular storyteller panorama tour’s usability, user experience, engagement, and resulting audience understanding. The results of the evaluations showed that storyteller panorama tours could be an effective, attractive, and engaging storyteller method for cultural institutions. However, the findings also indicated that the users’ age, gender, and computer-related experience influence the use and enjoyment levels. We share our experiences and offer an example of how to evaluate a storyteller panorama tour. We believe that the presented evaluation strategy would be applicable to other museum-related projects, as well.
Chapter
This paper presents an evaluation of usability and user experience perceptions of two suicide prevention websites to explore ways these critical resources may be improved. Suicide is one of the leading causes of death in the world and is most prevalent in teen and young adult populations. Studies have provided evidence that the Internet can influence behavior related to suicide. Therefore, this paper explores usability and user experience to better understand how the language, design, and ease of navigation contribute to user perceptions.
Chapter
Current and future animations seek for more human-like motions to create believable animations for computer games, animated movies and commercial spots. A technology widely used technology is motion capture to capture actors' movements which enrich digital avatars motions and emotions. However, a motion capture environment poses challenges to actors such as short preparation times and the need to highly rely on their acting and imagination skills. To support these actors, we developed a mixed reality application that allows showing digital environments while performing and being able to see the real and virtual world. We tested our prototype with 6 traditionally trained theatre and TV actors. As a result, the actors indicated that our application supported them getting into the demanded acting moods with less unrequired emotions. The acting scenario was also better understood with less need of explanation than when just discussing the scenario, as commonly done in theatre acting.
Article
This article proposes a novel approach towards quick concept generation and validation of interactive-product behaviours. When designing for user-product interactions, designers have to consider spatial and behavioural elements besides form/tangible aspects and perform quick-validation of the generated concepts often done through functional prototyping at later design stages. As a result, the designed-outcomes often depend on parameters like designer's familiarity with the design tools used, the level of fidelity achieved while prototyping and the frequency of design-iterations, limiting a thorough-exploration of concept-space and outcomes' creativity at the early design stages. This research targets such dependencies and non-creative hindrances at concept generation stage through a Mixed Reality implementation. This work establishes requirements for creating a suitable design-tool and presents a proof-of-concept use-case. A design task to ideate, create and revise concepts of playful product-behaviours swiftly was performed to assess the impact of the implemented method. In an empirical study, a broader exploration of solution-space and an overall improvement in creative output-flow was observed when compared to the design-outcomes in the traditional storyboard design approach. Though the implemented design-tool's unfamiliarity and capability presented a challenge, a significant increase in usage of iterative concept-design behaviour was observed throughout the study.
Chapter
Evaluating the utility of Human-Machine Systems’ User Interfaces is not trivial. Several evaluation methods can be used to investigate if the behaviour of the user interface complies with best practices of Human-Machine Interface Design. Even when is possible to agree on which methods to use to conduct the evaluation, defining the utility requires evaluating the interface under analysis toward the company’s goals, or mission. This paper investigates how the utility, perceived by end users of interfaces, can be captured by a research instrument, as well as be represented by a structured approach based on Usability evaluation and Balanced Score Cards methodology. This is an alternative demarche for accessing the Usability of a Software System, and the main goal is helping designers and administrators to maintain and improve their systems.
Article
Whilst new patents and announcements advertise the technical availability of foldable displays, which are capable to be folded to some extent, there is still a lack of fundamental and applied understanding of how to model, to design, and to prototype graphical user interfaces for these devices before actually implementing them. Without waiting for their off-the-shelf availability and without being tied to any physical foldable mechanism, Flecto defines a model, an associated notation, and a supporting software for prototyping graphical user interfaces running on foldable displays, such as foldable smartphone or assemblies of foldable surfaces. For this purpose, we use an extended notation of the Yoshizawa-Randlett diagramming system, used to describe the folds of origami models, to characterize a foldable display and define possible interactive actions based on its folding operations. A guiding method for rapidly prototyping foldable user interfaces is devised and supported by Flecto, a design environment where foldable user interfaces are simulated in 3D environment instead of in physical reality. We report on a case study to demonstrate Flecto in action and we gather the feedback from users on Flecto, using Microsoft Product Reaction Cards.
Chapter
Website design, analysis, and related critical thinking in a CALL context are rather unique. This chapter presents an in-depth exploration of how website analysis and design pedagogy could help support analytical thinking and English language production in an EFL context. The study investigated if students could analyze English websites and comprehend and produce responses in English for eight open-ended questions, divided into two sets of design and inference-based queries. Additionally, students answered a questionnaire on their own awareness about the use of metacognitive reading strategies during website analysis and questionnaire responses. Results have demonstrated the reasonable ability for students to answer most design and inference set queries. This chapter also introduces the idea of a project-based CALL environment. This was created through 3D-printing-related processes and in-class design, and manufacturing of digital prototypes of products, that culminated in procedural documentation in a computer-mediated collaborative environment and with related website production.
Article
Health and medical contexts have emerged as an important area of inquiry for researchers at the intersection of user experience and technical communication. In addressing this intersection, this article advocates and extends patient experience design or PXD ( Melonçon, 2017 ) as an important framework for user experience research within health and medicine. Specifically, this article presents several PXD insights from a task-based usability study that examined an online intervention program for people with voice problems. We respond to Melonçon's call ( 2017 ) to build PXD as a framework for user experience and technical communication research by describing ways traditional usability methods can provide PXD insights and asking the following question: What insights can emerge from combining traditional usability methods and PXD research? In addressing this question, we outline two primary methodological and practical considerations we found central to conducting PXD research: (1) engaging patients as participants, and (2) leveraging multidisciplinary collaboration.
Article
In contrast to existing, static audience controls that map poorly onto users' ideal audiences on social networking sites, dynamic audience selection (DAS) controls can make intelligent inferences to help users select their ideal audience given context and content. But does this potential utility outweigh its potential intrusiveness? We surveyed 250 participants to model users' ideal versus their chosen audiences with static controls and found a significant misalignment, suggesting that DAS might provide utility. We then designed a sensitizing prototype that allowed users to select audiences based on personal attributes, content, or context constraints. We evaluated DAS vis-a-vis Facebook's existing audience selection controls through a counterbalanced summative evaluation. We found that DAS's expressiveness, customizability, and scalability made participants feel more confident about the content they shared on Facebook. However, low transparency, distrust in algorithmic inferences, and the emergence of privacy-violating side channels made participants find the prototype unreliable or intrusive. We discuss factors that affected this trade-off between DAS's utility and intrusiveness and synthesize design implications for future audience selection tools.
Article
Full-text available
The experiment was targeted to develop design strategies and methods by testing the complex interplay between the dynamics of daylight and electrical lighting in an office. The double dynamic lighting design concept is based on the idea of adding task lighting, with a directionality referring to the daylight inflow and a variation on direct/diffuse lighting and respective changes in colour temperature respond to sky conditions and daylight levels. The experiment was conducted in an office space at Aalborg University in Copenhagen from September to December 2019. Four participants moved in and worked in the office with four-week periods of respective standard static lighting as a baseline, and dynamic lighting. In a parallel mixed method approach with interviews and questionnaires, the dynamic lighting was compared to the baseline and to a control group. The results indicate that the dynamic lighting periods had a positive effect on visual comfort, perceived atmosphere and work engagement. The studies helped to develop the definition of five dynamic light settings. Seasonal changes, time of day, dynamic sunscreens and individual needs for task lighting can be implemented in future field experiments as additional dynamic parameters to meet individual needs and circadian potentials for double dynamic light.
ResearchGate has not been able to resolve any references for this publication.