No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
The Role and Potentials of Field User Interaction Data in the Automotive UX Development Lifecycle: An Industry Perspective
Abstract
We are interested in the role of field user interaction data in the development of In-Vehicle Information Systems (IVISs), the potentials practitioners see in analyzing this data, the concerns they share, and how this compares to companies with digital products. We conducted interviews with 14 UX professionals, 8 from automotive and 6 from digital companies, and analyzed the results by emergent thematic coding. Our key findings indicate that implicit feedback through field user interaction data is currently not evident in the automotive UX development process. Most decisions regarding the design of IVISs are made based on personal preferences and the intuitions of stakeholders. However, the interviewees also indicated that user interaction data has the potential to lower the influence of guesswork and assumptions in the UX design process and can help to make the UX development lifecycle more evidence-based and user-centered. CCS CONCEPTS • General and reference → Surveys and overviews; • Human-centered computing → HCI design and evaluation methods; Empirical studies in HCI. KEYWORDS interview study, user experience, in-vehicle information systems ACM Reference Format:
... Currently, the automotive industry primarily relies on empirical user studies to evaluate in-vehicle interfaces for usability and distraction potential. However, the complexity of modern infotainment systems makes these studies resourceintensive and limits their ability to provide holistic distraction evaluations [16]. A promising solution are computational models that can automate (parts of) the interface evaluation and thus help designers to evaluate designs not only more efficiently, but also earlier in the design process and on a larger scale. ...
... Product Development (16) Crash Risk Prediction (1) Others (2) Distraction Detection (8) Human Behavior Analysis (7) ...
... No Traffic (11) Traffic (16) No Information (7) Mobile Phone (10) Conversation (5) Other (1) ...
In this review, we analyze the current state of the art of computational models for in-vehicle User Interface (UI) design. Driver distraction, often caused by drivers performing Non Driving Related Tasks (NDRTs), is a major contributor to vehicle crashes. Accordingly, in-vehicle User Interfaces (UIs) must be evaluated for their distraction potential. Computational models are a promising solution to automate this evaluation, but are not yet widely used, limiting their real-world impact. We systematically review the existing literature on computational models for NDRTs to analyze why current approaches have not yet found their way into practice. We found that while many models are intended for UI evaluation, they focus on small and isolated phenomena that are disconnected from the needs of automotive UI designers. In addition, very few approaches make predictions detailed enough to inform current design processes. Our analysis of the state of the art, the identified research gaps, and the formulated research potentials can guide researchers and practitioners toward computational models that improve the automotive User Interface (UI) design process.
... Currently, the automotive industry primarily relies on empirical user studies to evaluate in-vehicle interfaces for usability and distraction potential. However, the complexity of modern infotainment systems makes these studies resourceintensive and limits their ability to provide holistic distraction evaluations [16]. A promising solution are computational models that can automate (parts of) the interface evaluation and thus help designers to evaluate designs not only more efficiently, but also earlier in the design process and on a larger scale. ...
... Product Development (16) Crash Risk Prediction (1) Others (2) Distraction Detection (8) Human Behavior Analysis (7) ...
... No Traffic (11) Traffic (16) No Information (7) Mobile Phone (10) Conversation (5) Other (1) ...
In this review, we analyze the current state of the art of computational models for in-vehicle User Interface (UI) design. Driver distraction, often caused by drivers performing Non Driving Related Tasks (NDRTs), is a major contributor to vehicle crashes. Accordingly, in-vehicle UI s must be evaluated for their distraction potential. Computational models are a promising solution to automate this evaluation, but are not yet widely used. limit their real-world impact. We systematically review the existing literature on computational models for NDRTs to analyze why current approaches have not yet found their way into practice. We found that while many models are intended for UI evaluation, they focus on small and isolated phenomena that are disconnected from the needs of automotive UI designers. In addition, very few approaches make predictions detailed enough to inform current design processes. Our analysis of the state of the art, the identified research gaps, and the formulated research potentials can guide researchers and practitioners towards computational models that improve the automotive UI design process.
... The growing number of features of modern touchscreen-based IVISs and the need to evaluate them with respect to the driving context [25] makes it increasingly complex to design IVISs that meet user needs and are safe to use. To date, the usability and distraction evaluation of IVISs is mostly based on qualitative feedback and small-scale user studies [16]. However, these approaches do not scale with the increasing complexity of the design task and the increasing number of features that need to be evaluated. ...
... Automotive UX professionals report that they lack the tools to access and analyze relevant data, even though modern cars collect large amounts of driving and interaction-related data. To gain insights from customer data, UX experts often have to submit requests to data scientists and involve other departments [16]. As a result, the problem that traditional methods are slow is only shifted, not solved. ...
... Our approach aims to improve the industrial design process of IVISs. Designers and UX researchers report that they are often forced to neglect design evaluation due to time constraints and data accessibility issues [16]. To develop a solution that meets the needs of automotive UX experts and improves their design and evaluation process, we followed a mixed methods User-Centered Design (UCD) approach [3,46]. ...
... The growing number of features of modern touchscreen-based IVISs and the need to evaluate them with respect to the driving context [25] makes it increasingly complex to design IVISs that meet user needs and are safe to use. To date, the usability and distraction evaluation of IVISs is mostly based on qualitative feedback and small-scale user studies [16]. However, these approaches do not scale with the increasing complexity of the design task and the increasing number of features that need to be evaluated. ...
... Automotive UX professionals report that they lack the tools to access and analyze relevant data, even though modern cars collect large amounts of driving and interaction-related data. To gain insights from customer data, UX experts often have to submit requests to data scientists and involve other departments [16]. As a result, the problem that traditional methods are slow is only shifted, not solved. ...
... Our approach aims to improve the industrial design process of IVISs. Designers and UX researchers report that they are often forced to neglect design evaluation due to time constraints and data accessibility issues [16]. To develop a solution that meets the needs of automotive UX experts and improves their design and evaluation process, we followed a mixed methods User-Centered Design (UCD) approach [3,46]. ...
User Experience (UX) professionals need to be able to analyze large amounts of usage data on their own to make evidence-based design decisions. However, the design process for In-Vehicle Information Systems (IVIS) lacks data-driven support and effective tools for visualizing and analyzing user interaction data. Therefore, we propose ICEBOAT, an interactive visualization tool tailored to the needs of automotive UX experts to effectively and efficiently evaluate driver interactions with IVISs. ICEBOAT visualizes telematics data collected from production line vehicles, allowing UX experts to perform task-specific analyses. Following a mixed methods User-centered design (UCD) approach, we conducted an interview study (N=4) to extract the domain specific information and interaction needs of automotive UX experts and used a co-design approach (N=4) to develop an interactive analysis tool. Our evaluation (N=12) shows that ICEBOAT enables UX experts to efficiently generate knowledge that facilitates data-driven design decisions.
... Since eyesoff-road durations longer than two seconds are proven to increase the crash risk [18], the evaluation of touchscreen-based IVISs also becomes a safety-related aspect apart from developing a system that satisfies the user needs in the best possible way. This added complexity makes it even harder to evaluate IVISs, which is why UX experts require support from data-driven methods [4]. ...
... However, the analysis and visualization of big interaction data can significantly benefit user behavior evaluation [17,38] and offers great potential for the automotive domain [25]. Ebel et al. [4] state that, currently, automotive interaction data is not used to its full potential. They describe that experts need aggregations of the large amounts of data and visualizations that allow deriving insights into user and driving behavior. ...
... Lamm and Wolff [19] describe that user behavior evaluations based on implicit data, generated from field usage, currently, do not play an important role in automotive UX development. On the other hand, Ebel et al. [4] found that automotive UX experts are in need of data-driven methods and visualizations that benefit a holistic system understanding based on data retrieved from production line vehicles. The authors argue that experts need tool support to understand what features are being used in which situations, how long it takes users to complete certain tasks, and how the interactions with IVISs affect the driving behavior. ...
... Digital companies recognized early that data-driven methods and big data analytics can have great value for their UX design process. In contrast, automotive OEMs are currently unable to fully exploit the possibilities and potentials of those methods (Ebel et al., 2020a) for the Product Development (PD) life cycle. This is due to organizational, legal, or technical restrictions. ...
... Additionally, the study leads to a deeper understanding of automotive-related limitations and builds the foundation for further investigation on how those limitations might be overcome. The study design and outcome are precisely described in Ebel et al. (2020a). ...
... However, a differentiation between the different types of studies has to be made. With Study 2 (Orlovska et al., 2020c) and Study 3 (Ebel et al., 2020a) being qualitative user studies, Maxwell's five threats to validity (Maxwell, 2012) apply. Maxwell (2012) elaborates on the flaws that can occur during study execution and data collection, and on the threat of deliberately or accidentally manipulating the collected data to fit a certain theory. ...
The development and evaluation of In-Vehicle Information Systems (IVISs) is strongly based on insights from qualitative studies conducted in artificial contexts (e.g., driving simulators or lab experiments). However, the growing complexity of the systems and the uncertainty about the context in which they are used, create a need to augment qualitative data with quantitative data, collected during real-world driving. In contrast to many digital companies that are already successfully using data-driven methods, Original Equipment Manufacturers (OEMs) are not yet succeeding in releasing the potentials such methods offer. We aim to understand what prevents automotive OEMs from applying data-driven methods, what needs practitioners formulate, and how collecting and analyzing usage data from vehicles can enhance UX activities. We adopted a Multiphase Mixed Methods approach comprising two interview studies with more than 15 UX practitioners and two action research studies conducted with two different OEMs. From the four studies, we synthesize the needs of UX designers, extract limitations within the domain that hinder the application of data-driven methods, elaborate on unleveraged potentials, and formulate recommendations to improve the usage of vehicle data. We conclude that, in addition to modernizing the legal, technical, and organizational infrastructure, UX and Data Science must be brought closer together by reducing silo mentality and increasing interdisciplinary collaboration. New tools and methods need to be developed and UX experts must be empowered to make data-based evidence an integral part of the UX design process.
... Since eyesoff-road durations longer than two seconds are proven to increase the crash risk [18], the evaluation of touchscreen-based IVISs also becomes a safety-related aspect apart from developing a system that satisfies the user needs in the best possible way. This added complexity makes it even harder to evaluate IVISs, which is why UX experts require support from data-driven methods [4]. ...
... However, the analysis and visualization of big interaction data can significantly benefit user behavior evaluation [17,38] and offers great potential for the automotive domain [25]. Ebel et al. [4] state that, currently, automotive interaction data is not used to its full potential. They describe that experts need aggregations of the large amounts of data and visualizations that allow deriving insights into user and driving behavior. ...
... Lamm and Wolff [19] describe that user behavior evaluations based on implicit data, generated from field usage, currently, do not play an important role in automotive UX development. On the other hand, Ebel et al. [4] found that automotive UX experts are in need of data-driven methods and visualizations that benefit a holistic system understanding based on data retrieved from production line vehicles. The authors argue that experts need tool support to understand what features are being used in which situations, how long it takes users to complete certain tasks, and how the interactions with IVISs affect the driving behavior. ...
With modern IVIS becoming more capable and complex than ever, their evaluation becomes increasingly difficult. The analysis of large amounts of user behavior data can help to cope with this complexity and can support UX experts in designing IVIS that serve customer needs and are safe to operate while driving. We, therefore, propose a Multi-level User Behavior Visualization Framework providing effective visualizations of user behavior data that is collected via telematics from production vehicles. Our approach visualizes user behavior data on three different levels: (1) The Task Level View aggregates event sequence data generated through touchscreen interactions to visualize user flows. (2) The Flow Level View allows comparing the individual flows based on a chosen metric. (3) The Sequence Level View provides detailed insights into touch interactions, glance, and driving behavior. Our case study proves that UX experts consider our approach a useful addition to their design process.
... To keep pace with this rapid development and to continuously develop interfaces that are well received by the customers, there is an increased need for data-driven support in the automotive UX design [2,4,9]. Whereas websites and apps track every interaction UIST a user makes and design decisions are made in consideration of conversion rates, time on task, or error rates [6], the decision-making process for the design of IVISs is more complex. ...
... In previous research [2,4], we identified needs and potentials to support the automotive design process by providing feature usage statistics, user flow analyses, and context-dependent visualizations. Based on these findings, we developed three standalone hierarchical user behavior visualizations that proved their usefulness in a user study [3]. ...
... To keep pace with this rapid development and to continuously develop interfaces that are well received by the customers, there is an increased need for data-driven support in the automotive UX design [2,4,9]. Whereas websites and apps track every interaction UIST a user makes and design decisions are made in consideration of conversion rates, time on task, or error rates [6], the decision-making process for the design of IVISs is more complex. ...
... In previous research [2,4], we identified needs and potentials to support the automotive design process by providing feature usage statistics, user flow analyses, and context-dependent visualizations. Based on these findings, we developed three standalone hierarchical user behavior visualizations that proved their usefulness in a user study [3]. ...
In this work, we present ICEBOAT an interactive tool that enables automotive UX experts to explore how users interact with In-vehicle Information Systems. Based on large naturalistic driving data continuously collected from production line vehicles, ICEBOAT visualizes drivers' interactions and driving behavior on different levels of detail. Hence, it allows to easily compare different user flows based on performance- and safety-related metrics.
... As stated by Yerram et al. [44], UX design can improve quality of products, the developers can create effective and enjoyable solutions when they understand the objectives, and preferences of users. In reference [45], a cases study was performed in automotive industry who is the leader in the field of product line engineering. They interviewed UX professionals who stressed the need of statistical support based on user interaction data to leverage feature elicitation, and their prioritization. ...
... Data-driven approaches have gained traction in various fields, including automotive design, where they enable designers to make informed decisions based on empirical data [39], [40], [41]. Researchers have used data-driven models for various aspects of automobiles, such as improving the braking control systems [42], or evaluating the health of electronic systems on board [43]. ...
Haptic affection plays a crucial role in user experience, particularly in the automotive industry where the tactile quality of components can influence customer satisfaction. This study aims to accurately predict the affective property of a car door by only watching the force or torque profile of it when opening. To this end, a deep learning model is designed to capture the underlying relationships between force profiles and user-defined adjective ratings, providing insights into the door-opening experience. The dataset employed in this research includes force profiles and user adjective ratings collected from six distinct car models, reflecting a diverse set of door-opening characteristics and tactile feedback. The model's performance is assessed using Leave-One-Out Cross-Validation, a method that measures its generalization capability on unseen data. The results demonstrate that the proposed model achieves a high level of prediction accuracy, indicating its potential in various applications related to haptic affection and design optimization in the automotive industry.
... Despite these challenges, designers in the automotive industry have expressed particular interest towards integrating data into the design process (Ebel et al., 2021). User interaction data could decrease the impact of conjecture and guesswork in the consumer vehicle design process (Ebel et al., 2020) through data-driven personas, content-dependent design evaluations, and user flow visualizations (Ebel et al., 2021). The value of ML In extracting valuable insights from data to support decision-making in the automotive industry has been recognized but practiced to a limited extent. ...
Data-driven design is believed to be empowered by machine learning (ML) with advanced pattern classification and prediction. However, research on how ML can be used to support automotive human-machine interface (HMI) design is lacking. We presented a case study of truck HMI design to understand the current data use and expectations of ML in the design process. Findings show decentralized data practices, the role of expertise in decision-making, and the envisioned reactive use of ML, where we underscore the implications for advancing human-ML collaboration in designing future truck HMI systems.
... Furthermore, to evaluate a new IVISs design in a user study, all relevant features need to be implemented in a functional prototype. Although such measures will remain necessary, automotive UX experts require explainable evaluation methods (Ebel et al., 2020) allowing them to identify potentially distracting interaction patterns already in the early design stages. Such automated methods can facilitate the development of interaction concepts that are safe by design and, therefore, less likely to fail final evaluations. ...
With modern infotainment systems, drivers are increasingly tempted to engage in secondary tasks while driving. Since distracted driving is already one of the main causes of fatal accidents, in-vehicle touchscreen Human-Machine Interfaces (HMIs) must be as little distracting as possible. To ensure that these systems are safe to use, they undergo elaborate and expensive empirical testing, requiring fully functional prototypes. Thus, early-stage methods informing designers about the implication their design may have on driver distraction are of great value. This paper presents a machine learning method that, based on anticipated usage scenarios, predicts the visual demand of in-vehicle touchscreen interactions and provides local and global explanations of the factors influencing drivers' visual attention allocation. The approach is based on large-scale natural driving data continuously collected from production line vehicles and employs the SHapley Additive exPlanation (SHAP) method to provide explanations leveraging informed design decisions. Our approach is more accurate than related work and identifies interactions during which long glances occur with 68 % accuracy and predicts the total glance duration with a mean error of 2.4 s. Our explanations replicate the results of various recent studies and provide fast and easily accessible insights into the effect of UI elements, driving automation, and vehicle speed on driver distraction. The system can not only help designers to evaluate current designs but also help them to better anticipate and understand the implications their design decisions might have on future designs.
... To enable such predictions, we envision a system leveraging driving and interaction data, automatically collected from a large amount of production line vehicles. Having access to such large-scale data makes it possible to generate insights that go beyond the detail of current, mostly qualitative or relatively small-scale naturalistic driving studies [5]. Additionally, as soon as a software update is deployed to the fleet, the changes that were made can directly be assessed. ...
... Manuscript submitted to ACM production line vehicles. Having access to such large-scale data makes it possible to generate insights that go beyond the detail of current, mostly qualitative or relatively small-scale naturalistic driving studies [5]. Additionally, as soon as a software update is deployed to the fleet, the changes that were made can directly be assessed. ...
Center touchscreens are the main HMI (Human-Machine Interface) between the driver and the vehicle. They are becoming, larger, increasingly complex and replace functions that could previously be controlled using haptic interfaces. To ensure that touchscreen HMI can be operated safely, they are subject to strict regulations and elaborate test protocols. Those methods and user trials require fully functional prototypes and are expensive and time-consuming. Therefore it is desirable to estimate the workload of specific interfaces or interaction sequences as early as possible in the development process. To address this problem, we envision a model-based approach that, based on the combination of user interactions and UI elements, can predict the secondary task load of the driver when interacting with the center screen. In this work, we present our current status, preliminary results, and our vision for a model-based system build upon large-scale natural driving data.
With modern infotainment systems, drivers are increasingly tempted to engage in secondary tasks while driving. Since distracted driving is already one of the main causes of fatal accidents, in-vehicle touchscreens must be as little distracting as possible. To ensure that these systems are safe to use, they undergo elaborate and expensive empirical testing, requiring fully functional prototypes. Thus, early-stage methods informing designers about the implication their design may have on driver distraction are of great value. This paper presents a machine learning method that, based on anticipated usage scenarios, predicts the visual demand of in-vehicle touchscreen interactions and provides local and global explanations of the factors influencing drivers' visual attention allocation. The approach is based on large-scale natural driving data continuously collected from production line vehicles and employs the SHapley Additive exPlanation (SHAP) method to provide explanations leveraging informed design decisions. Our approach is more accurate than related work and identifies interactions during which long glances occur with 68% accuracy and predicts the total glance duration with a mean error of 2.4s. Our explanations replicate the results of various recent studies and provide fast and easily accessible insights into the effect of UI elements, driving automation, and vehicle speed on driver distraction. The system can not only help designers to evaluate current designs but also help them to better anticipate and understand the implications their design decisions might have on future designs.
Automated Driver Assistance Systems (ADAS) are designed to support the driver and enhance the driving experience. Due to ADAS limitations associated with the driving context, the intended use of ADAS functions is often non-transparent for the end-user. The system performance capabilities affected by the continuously changing driving context influence ADAS usage. However, the cumulative effect of the driving context on driver behavior and ADAS usage is insufficiently covered in the ongoing research. This paper aims to investigate and understand how the driving context affects the use of ADAS. Throughout this research, data from a Naturalistic Driving (ND) study was collected and analyzed. The analysis of the ND data helped to register how drivers use ADAS in different driving conditions and indicated several issues associated with ADAS usage. To be able to clarify the outcomes of quantitative sensor-based data analysis, an explanatory sequential mixed-method design was implemented. The method facilitated the subsequent design of qualitative in-depth interviews with the drivers. The combined data analysis allowed a holistic interpretation and evaluation of the findings regarding the effect of the driving context on ADAS usage. The findings warrant consideration of the driving context as a key factor enabling the effective development of ADAS functions.
The lack of automated cars above SAE level 3 raises challenges for conducting User Experience Design (UXD) and behaviour research for automated driving. User-centred methods are critical to ensuring a human-friendly progress of vehicle automation. This work introduces the Immersive Video-based Automated Driving (IVAD) Simulator. It uses carefully recorded 180/360° videos that are played back in a driving simulator. This provides immersive driving experiences in visually realistic and familiar environments. This paper reports learnings from an iterative development of IVAD, and findings of two user studies: One simulator study (N=15) focused on the immersive experience; and one VR study (N=16) focused on rapid prototyping and the evaluation of Augmented Reality (AR) concepts. Overall, we found the method to be a useful, versatile and low budget UXD tool with a high level of immersion that is uniquely aided by the familiarity of the environment. IVAD's limitations and future improvements are discussed in relation to research applications within AutoUI.
In the evolution of technical systems, freedom from error and early adoption plays a major role for market success and to maintain competitiveness. In the case of automated driving, we see that faulty systems are put into operation and users trust these systems, often without any restrictions. Trust and use are often associated with users' experience of the driver-vehicle interfaces and interior design. In this work, we present the results of our investigations on factors that influence the perception of automated driving. In a simulator study, N=48 participants had to drive a SAE level 2 vehicle with either perfect or faulty driving function. As a secondary activity, participants had to solve tasks on an infotainment system with varying aesthetics and usability (2x2). Results reveal that the interaction of conditions significantly influences trust and UX of the vehicle system. Our conclusion is that all aspects of vehicle design cumulate to system and trust perception.
In this paper, we focus on exploiting the vehicle kinematic signals from a naturalistic driving dataset to estimate motor control difficulty associated with the driving environment (i.e., curvature and poor surface condition), and detect whether the driver was engaged in a secondary task or not. Advanced driver assistance systems can exploit such driver behavior models to better support the driving task and improve safety. Hidden Markov models were built from sequential data; lateral (x-axis) and longitudinal (y-axis) acceleration were used to classify motor control difficulty (lower vs. higher), whereas GPS speed and steering wheel position were used to classify secondary task engagement (yes vs. no). The resulting accuracy for lower motor control difficulty classification was 72.03%, whereas 71.27% was achieved for higher motor control difficulty. Cases of engagement in secondary task vs. not were classified with 84.4% and 74.0% accuracy, respectively.
This article studied and compared the two nonprobability sampling techniques namely, Convenience Sampling and Purposive Sampling. Convenience Sampling and Purposive Sampling are Nonprobability Sampling Techniques that a researcher uses to choose a sample of subjects/units from a population. Although, Nonprobability sampling has a lot of limitations due to the subjective nature in choosing the sample and thus it is not good representative of the population, but it is useful especially when randomization is impossible like when the population is very large. It can be useful when the researcher has limited resources, time and workforce. It can also be used when the research does not aim to generate results that will be used to create generalizations pertaining to the entire population. Therefore, there is a need to use nonprobability sampling techniques. The aim of this study is to compare among the two nonrandom sampling techniques in order to know whether one technique is better or useful than the other. Different articles were reviewed to compare between Convenience Sampling and Purposive Sampling and it is concluded that the choice of the techniques (Convenience Sampling and Purposive Sampling) depends on the nature and type of the research.
Online services are increasingly dependent on user participation. Whether it's online social networks or crowdsourcing services, understanding user behavior is important yet challenging. In this paper, we build an unsupervised system to capture dominating user behaviors from clickstream data (traces of users' click events), and visualize the detected behaviors in an intuitive manner. Our system identifies "clusters" of similar users by partitioning a similarity graph (nodes are users; edges are weighted by clickstream similarity). The partitioning process leverages iterative feature pruning to capture the natural hierarchy within user clusters and produce intuitive features for visualizing and understanding captured user behaviors. For evaluation, we present case studies on two large-scale clickstream traces (142 million events) from real social networks. Our system effectively identifies previously unknown behaviors, e.g., dormant users, hostile chatters. Also, our user study shows people can easily interpret identified behaviors using our visualization tool.
With the proliferation of the touchscreen technology, interactive displays are becoming an integrated part of the modern vehicle environment. However, due to road and driving conditions, the user input on such displays can be perturbed resulting in erroneous selections. This paper describes an evaluative study of the usability and input performance of in-vehicle touchscreens. The analysis is based on data collected in instrumented cars driven under various road/driving conditions. We assess the frequency of failed selection attempts , distances by which users miss the intended on-screen target and the durations of undertaken free hand pointing gestures to accomplish the selection tasks. It is shown that the road/driving conditions can notably undermine the usability of an interactive display when the user input is perturbed, e.g. due to the experienced vibrations and lateral accelerations in the vehicle. The distance between the location of an erroneous on-screen selection and the intended endpoint on the display, is closely related to the level of present in-vehicle noise. The conducted study can advise graphical user interfaces design for the vehicle environment where the user free hand pointing gestures can be subject to varying levels of perturbations .
Explicit user testing tends to be costly and time-consuming from a company’s point of view. Therefore, it would be desirable to infer a quantitative usability score directly from implicit feedback, i.e., the interactions of users with a web interface. As a basis for this, we require an adequate usability instrument whose items form a usability score and can be meaningfully correlated with such interactions. Thus, we present INUIT, the first instrument consisting of only seven items that have the right level of abstraction to directly reflect user behavior on the client. It has been designed in a two-step process involving usability guideline reviews and expert interviews. A confirmatory factor analysis shows that our model reasonably well reflects real-world perceptions of usability
In this article the author addresses the history of reliability and validity in qualitative research as this method of inquiry has progressed through various paradigms. The importance of the concepts of reliability and validity in research findings is traced from the traditional era, where there was only a modest distinction between qualitative and quantitative researchers involving their definitions of research reliability and validity, through the current era, where some researchers question the need to be restricted in their research by attempting to control for or account for the reliability and validity of their research findings. The author rejects a strict need for reliability and validity as traditionally defined in quantitative research and outlines a less restrictive approach to ensuring reliability and validity in qualitative research.
There has been increasing interest in the adoption of UX within corporate environments, and what competencies translate into effective UX design. This paper addresses the space between pedagogy and UX practice through the lens of competence, with the goal of understanding how students are initiated into the practice community, how their perception of competence shifts over time, and what factors influence this shift. A 12-week longitudinal data collection, including surveys and interviews, documents this shift, with participants beginning internships and full-time positions in UX. Students and early professionals were asked to assess their level of competence and factors that influenced competence. A co-construction of identity between the designer and their environment is proposed, with a variety of factors relating to tool and representational knowledge, complexity, and corporate culture influencing perceptions of competence in UX over time. Opportunities for future research, particularly in building an understanding of competency in UX based on this preliminary framing of early UX practice are addressed.
Increased competition between manufacturers made it necessary
for them to create products that provide great experiences in order
to stand out from the rest of the market. For a positive product
evaluation by the user, not only usability, but also user experience
plays an important role. Recent research revealed that the most
common method of using post-test questionnaires to assess user
experience as a remembered episode could miss information about
the experience [19]. This study presents a method for measuring
user experience through fulfillment of psychological needs
momentarily (‘momentary UX’) in situations instead of in a whole
interaction episode. 28 participants used a novel navigation device
that is designed to help explore the environment during a leisure
car ride as a team together with other passengers. A story
approach was used to create a context for the experimental
product interaction. The results show that psychological needs
relatedness and stimulation addressed by design could be
measured specifically and momentarily for relevant events during
the interaction. Further, a positive relationship to positive feelings
in the interaction was found. Beyond that, a similar questionnaire
for the whole episode is validated in this interaction and
comparison between episodic and momentary UX is made. It is
demonstrated that the method for measuring momentary UX is
specific for needs and situations, and possible use cases are
discussed.
Guidelines for determining nonprobabilistic sample sizes are virtually nonexistent. Purposive samples are the most commonly used form of nonprobabilistic sampling, and their size typically relies on the concept of “saturation,” or the point at which no new information or themes are observed in the data. Although the idea of saturation is helpful at the conceptual level, it provides little practical guidance for estimating sample sizes, prior to data collection, necessary for conducting quality research. Using data from a study involving sixty in-depth interviews with women in two West African countries, the authors systematically document the degree of data saturation and variability over the course of thematic analysis. They operationalize saturation and make evidence-based recommendations regarding nonprobabilistic sample sizes for interviews. Based on the data set, they found that saturation occurred within the first twelve interviews, although basic elements for metathemes were present as early as six interviews. Variability within the data followed similar patterns.
Over the last decade, 'user experience' (UX) became a buzzword in the field of human – computer interaction (HCI) and interaction design. As technology matured, interactive products became not only more useful and usable, but also fashionable, fascinating things to desire. Driven by the impression that a narrow focus on interactive products as tools does not capture the variety and emerging aspects of technology use, practitioners and researchers alike, seem to readily embrace the notion of UX as a viable alternative to traditional HCI. And, indeed, the term promises change and a fresh look, without being too specific about its definite meaning. The present introduction to the special issue on 'Empirical studies of the user experience' attempts to give a provisional answer to the question of what is meant by 'the user experience'. It provides a cursory sketch of UX and how we think UX research will look like in the future. It is not so much meant as a forecast of the future, but as a proposal – a stimulus for further UX research.
Understanding how users behave when they connect to social networking sites creates opportunities for better interface design, richer studies of social interactions, and improved design of content distribution systems. In this paper, we present a rst of a kind analysis of user workloads in on- line social networks. Our study is based on detailed click- stream data, collected over a 12-day period, summarizing HTTP sessions of 37,024 users who accessed four popular social networks: Orkut, MySpace, Hi5, and LinkedIn. The data were collected from a social network aggregator web- site in Brazil, which enables users to connect to multiple social networks with a single authentication. Our analysis of the clickstream data reveals key features of the social net- work workloads, such as how frequently people connect to social networks and for how long, as well as the types and sequences of activities that users conduct on these sites. Ad- ditionally, we crawled the social network topology of Orkut, so that we could analyze user interaction data in light of the social graph. Our data analysis suggests insights into how users interact with friends in Orkut, such as how frequently users visit their friends' or non-immediate friends' pages. In summary, our analysis demonstrates the power of using clickstream data in identifying patterns in social network workloads and social interactions. Our analysis shows that browsing, which cannot be inferred from crawling publicly available data, accounts for 92% of all user activities. Con- sequently, compared to using only crawled data, considering silent interactions like browsing friends' pages increases the measured level of interaction among users.
Despite the growing interest in user experience (UX), it has been hard to gain a common agreement on the nature and scope of UX. In this paper, we report a survey that gathered the views on UX of 275 researchers and practitioners from academia and industry. Most respondents agree that UX is dynamic, context-dependent, and subjective. With respect to the more controversial issues, the authors propose to delineate UX as something individual (instead of social) that emerges from interacting with a product, system, service or an object. The draft ISO definition on UX seems to be in line with the survey findings, although the issues of experiencing anticipated use and the object of UX will require further explication. The outcome of this survey lays ground for understanding, scoping, and defining the concept of user experience.
In this paper, we investigate how detailed tracking of user interaction can be monitored using standard web technolo- gies. Our motivation is to enable implicit interaction and to ease usability evaluation of web applications outside the lab. To obtain meaningful statements on how users inter- act with a web application, the collected information needs to be more detailed and fine-grained than that provided by classical log files. We focus on tasks such as classifying the user with regard to computer usage proficiency or making a detailed assessment of how long it took users to fill in fields of a form. Additionally, it is important in the context of our work that usage tracking should not alter the user's ex- perience and that it should work with existing server and browser setups. We present an implementation for detailed tracking of user actions on web pages. An HTTP proxy modifies HTML pages by adding JavaScript code before de- livering them to the client. This JavaScript tracking code collects data about mouse movements, keyboard input and more. We demonstrate the usefulness of our approach in a case study.
Driver distraction is a major issue in manual driving, causing more than 30'000 fatal crashes on US roadways in 2015 only [11]. As such, it is widely studied in order to increase driving safety. Many studies show how to detect driver distraction using Machine Learning algorithms and driver psychophysiological data. In this study, we investigate the trade-off between efficiency and privacy while predicting driver distraction. Specifically, we want to assess the impact on the estimation of the driver state without access to his/her psychophysiological data. Different Machine Learning models (Convolutional Neural Networks, K-NN and Random forest) are implemented to evaluate the validity of the distraction detection with and without access to psychophysiological data. The results show that a Convolutional Neural Network model is still able to detect driver distraction without access to psychophysiological features, with an f1-score of 97.11%, losing only 1.37% in the process.
An exploratory literature review method was applied to publications from several sources on Human-Computer Interaction (HCI) for In-Vehicle Information Systems (IVIS). The novel approach for bibliographic classification uses a graph database to investigate connections between authors, papers, used methods, and investigated interface types. This allows the application of algorithms to find similarities between different publications and overlaps between different usability evaluation methods. Through community detection algorithms, the publications can be clustered based on similarity relationships. For the proposed approach several thousand papers were systematically filtered, classified, and stored in a graph database. The survey shows a trend for usability assessment methods with direct involvement of users, especially the observation of users and performance-related measurements, as well as questionnaires and interviews. However, especially methods usually applied in early stages of development based on the assessment through models or experts, as well as collaborative and creativity methods do not seem very popular in automotive HCI research.
To evaluate the performance of mobile app designs, designers and researchers employ techniques such as A/B, usability, and analytics-driven testing. While these are all useful strategies for evaluating known designs, comparing many divergent solutions to identify the most performant remains a costly and difficult problem. This paper introduces a design performance testing approach that leverages existing app implementations and crowd workers to enable comparative testing at scale. This approach is manifest in ZIPT, a zero-integration performance testing platform that allows designers to collect detailed design and interaction data over any Android app -- including apps they do not own and did not build. Designers can deploy scripted tests via ZIPT to collect aggregate user performance metrics (e.g., completion rate, time on task) and qualitative feedback over third-party apps. Through case studies, we demonstrate that designers can use ZIPT's aggregate data and visualizations to understand the relative performance of interaction patterns found in the wild, and identify usability issues in existing Android apps.
Informing a driver of a vehicle’s changing state and environment is a major challenge that grows with the introduction of in-vehicle assistant and infotainment systems. Even in the age of automation, the human will need to be in the loop for monitoring, taking over control, or making decisions. In these cases, poorly designed systems could lead to needless attentional demands imparted on the driver, taking it away from the primary driving task. Existing systems are offering simple and often unspecific alerts, leaving the human with the demanding task of identifying, localizing, and understanding the problem. Ideally, such systems should communicate information in a way that conveys its relevance and urgency. Specifically, information useful to promote driver safety should be conveyed as effective calls for action, while information not pertaining to safety (therefore less important) should be conveyed in ways that do not jeopardize driver attention. Adaptive ambient displays and peripheral interactions have the potential to provide superior solutions and could serve to unobtrusively present information, to shift the driver’s attention according to changing task demands, or enable a driver to react without losing the focus on the primary task. In order to build a common understanding across researchers and practitioners from different fields, we held a “Workshop on Adaptive Ambient In-Vehicle Displays and Interactions” at the AutomotiveUI‘15 conference. In this chapter, we discuss the outcomes of this workshop, provide examples of possible applications now or in the future and conclude with challenges in developing or using adaptive ambient interactions.
In-vehicle experiences are made up mainly of mundane small moments, repeated practices, and taken-for-granted decisions that make up daily experiences in and around private passenger vehicles. Understanding what those experiences are for drivers around the world presents an opportunity for designing novel interactive experiences, technologies, and user interfaces for vehicles. In this chapter, we present a set of tools, methodologies, and practices that will help reader create a holistic design space for future mobility. Transitioning between ethnography, insights, prototyping, experience design, and requirements decomposition is a challenging task even for experienced UX professionals. This chapter provides guidance in this matter with practical examples.
Given the rapid advancement of technologies in the automotive domain, driver--vehicle interaction has recently become more and more complicated. The amount of research applied to the vehicle cockpit is increasing, with the advent of (highly) automated driving, as the range of interaction that is possible in a driving vehicle expands. However, as opportunities increase, so does the number of challenges that automotive user experience designers and researchers will face. This chapter focuses on the instrumentation of sensing and displaying techniques and technologies to make better user experience while driving. In the driver--vehicle interaction loop, the vehicle can sense driver states, analyze, estimate, and model the data, and then display it through the appropriate channels for intervention purposes. To improve the interaction, a huge number of new/affordable sensing (EEG, fNIRS, IR imaging) and feedback (head-up displays, auditory feedback, tactile arrays, etc.) techniques have been introduced. However, little research has attempted to investigate this area in a systematic way. This chapter provides an overview of recent advances of input and output modalities to be used for timely, appropriate driver--vehicle interaction. After outlining relevant background, we provide information on the best-known practices for input and output modalities based on the exchange results from the workshop on practical experiences for measuring and modeling drivers and driver--vehicle interactions at AutomotiveUI 2015. This chapter can help answer research questions on how to instrument a driving simulator or realistic study to gather data and how to place interaction outputs to enable appropriate driver interactions.
After explaining the motivation and presenting related experiences, an extended Usage-Centered Design approach that integrates standardized process activities from User-Centered Design approach (defined in ISO 9241-210) and uses cultural models is suggested and simultaneously it is also adapted to ASPICE Standard so that the approach is suitable for the design of intercultural user interfaces/experiences in the automotive context. This agile oriented approach makes it possible to track and trace both the culture specific requirements and the design decisions for internationalized HCI in order to produce adequate cultural interaction experiences for users of automotive user interfaces in the car.
In this paper we develop a model capable of classifying drivers from their driving behaviors sensed by only low level sensors. The sensing platform consists of data available from the diagnostic outlet (OBD) of the car and smartphone sensors. We develop a window based support vector machine model to classify drivers. We test our model with two datasets collected under both controlled and naturalistic conditions. Furthermore, we evaluate the model using each sensor source (car and phone) independently and combining both the sensors. The average classification accuracies attained with data collected from three different cars shared between couples in a naturalistic environment were 75.83%, 85.83% and 86.67% using only phone sensors, only cars sensors and combined car and phone sensors respectively.
Modern web clickstream data consists of long, high-dimensional sequences of multivariate events, making it difficult to analyze. Following the overarching principle that the visual interface should provide information about the dataset at multiple levels of granularity and allow users to easily navigate across these levels, we identify four levels of granularity in clickstream analysis: patterns, segments, sequences and events. We present an analytic pipeline consisting of three stages: pattern mining, pattern pruning and coordinated exploration between patterns and sequences. Based on this approach, we discuss properties of maximal sequential patterns, propose methods to reduce the number of patterns and describe design considerations for visualizing the extracted sequential patterns and the corresponding raw sequences. We demonstrate the viability of our approach through an analysis scenario and discuss the strengths and limitations of the methods based on user feedback.
After conducting explorative user studies in the beginning of product development process, there are often more alternative product concepts than it is possible to develop further. The best ideas for production need to be selected by investigating the technical feasibility and profitability of each product concept, but also by evaluating attractiveness and value of the concept proposals for the target user group. This paper discusses means for and challenges in user experience evaluation in the early phases of product development, when only rough product concept descriptions exist. We investigate two lightweight evaluation methods in more detail: expert evaluation and remote online evaluation, and analyse how they could help in identifying the best concepts from the user experience perspective.
Driving a car is becoming increasingly complex. Many new features (e.g., for communication or entertainment) that can be used in addition to the primary task of driving a car increase the driver's workload. Assessing the driver's workload, however, is still a challenging task. A variety of means are explored which rather focus on experimental conditions than on real world scenarios (e.g., questionnaires). We focus on physiological data that may be assessed in an non-obtrusive way in the future and is therefore applicable in the real world.
Hence, we conducted a real world driving experiment with 10 participants measuring a variety of physiological data as well as a post-hoc video rating session. We use this data to analyze the differences in the workload in terms of road type as well as especially important parts of the route such as exits and on-ramps. Furthermore, we investigate the correlation between the objective assessed and subjective measured data.
Digital information that is place- and time-specific, is increasingly becoming available on all aspects of the urban landscape. People (cf. the Social Web), places (cf. the Geo Web), and physical objects (cf. ubiquitous computing, the Internet of Things) are increasingly infused with sensors, actuators, and tagged with a wealth of digital information. Urban informatics research explores these emerging digital layers of the city at the intersection of people, place and technology. However, little is known about the challenges and new opportunities that these digital layers may offer to road users driving through today's mega cities. We argue that this aspect is worth exploring in particular with regards to Auto-UI's overarching goal of making cars both safer and more enjoyable. This paper presents the findings of a pilot study, which included 14 urban informatics research experts participating in a guided ideation (idea creation) workshop within a simulated environment. They were immersed into different driving scenarios to imagine novel urban informatics type of applications specific to the driving context.
The usability of mobile applications is critical for their adoption because of the relatively small screen and awkward (sometimes virtual) keyboard, despite the recent advances of smartphones. Traditional laboratory-based usability testing is often tedious, expensive, and does not reflect real use cases. In this paper, we propose a toolkit that embeds into mobile applications the ability to automatically collect user interface (UI) events as the user interacts with the applications. The events are fine-grained and useful for quantified usability analysis. We have implemented the toolkit on Android devices and we evaluated the toolkit with a real deployed Android application by comparing event analysis (state-machine based) with traditional laboratory testing (expert based). The results show that our toolkit is effective at capturing detailed UI events for accurate usability analysis.
Previous studies have used mouse tracking as a tool to measure usability of webpages, user attention and search relevance. In this paper, we go beyond measurement of user behavior to prediction of the resulting user experience from mouse patterns alone. Specifically, we identify mouse markers that can predict user frustration and reading struggles at reasonably high accuracy. We believe that mouse-based prediction of user experience is an important advance, and could potentially offer a scalable way to infer user experience on the web. In addition, we demonstrate that mouse tracking could be used for applications such as evaluating content layout and content noticeability; we apply this in particular to advertisements. More generally, it could be used to infer user attention in complex webpages containing images, text and varied content, including how attention patterns vary with page layout and user distraction.
Implementing controls in the car becomes a major challenge: The use of simple physical buttons does not scale to the increased number of assistive, comfort, and infotainment functions. Current solutions include hierarchical menus and multi-functional control devices, which increase complexity and visual demand. Another option is speech control, which is not widely accepted, as it does not support visibility of actions, fine-grained feedback, and easy undo of actions. Our approach combines speech and gestures. By using speech for identification of functions, we exploit the visibility of objects in the car (e.g., mirror) and simple access to a wide range of functions equaling a very broad menu. Using gestures for manipulation (e.g., left/right), we provide fine-grained control with immediate feedback and easy undo of actions. In a user-centered process, we determined a set of user-defined gestures as well as common voice commands. For a prototype, we linked this to a car interior and driving simulator. In a study with 16 participants, we explored the impact of this form of multimodal interaction on the driving performance against a baseline using physical buttons. The results indicate that the use of speech and gesture is slower than using buttons but results in a similar driving performance. Users comment in a DALI questionnaire that the visual demand is lower when using speech and gestures.
In recent years, the issue of usability of in-vehicle devices has received growing attention. This is in line with the increase in functionality of these devices, which has been accompanied by the introduction of various new interfaces to facilitate the user–device interaction. The complexity and diversity of the driving task presents a unique challenge in defining usability: user interaction with in-vehicle devices creates a ‘dual task’ scenario, in which conflicts can arise between primary and secondary driving tasks. This, and the safety-critical nature of driving, must be accounted for in defining and evaluating the usability of in-vehicle devices. It is evident that defining usability depends on the context of use of the device in question. The aim of this review therefore is to define usability for in-vehicle devices by selecting a set of criteria to describe the various factors which contribute to usability in this specific context of use.
The amount of software in cars grows exponentially. Driving forces of this development are cheaper and more powerful hardware and the demand for innovations by new functions. The rapid increase of software and software based functionality brings various challenges (see [21], [23], [25], [26]) for the automotive industries, for their organization, key competencies, processes, methods, tools, models, product structures, division of work, logistics, maintenance, and long term strategies. From a software engineering perspective, the automotive industry is an ideal and fascinating application domain for advanced techniques. Although the automotive industry may adopt general results and solutions from the software engineering body of knowledge gained in other domains, the specific constraints and domain specific requirements in the automotive industry ask for individual solutions and bring various challenges for automotive software engineering. In cars we find literally all interesting problems and challenging issues of software and systems engineering.
In this editorial to the special issue we first introduce the field of driver support system (DSS) design and evaluation, frame it into the larger context of human–computer interaction research, and highlight some of its specificities. We then proceed to briefly present the selection of articles that compose this issue. Finally, we put the contributions to the special issue into perspective along a number of dimensions to show how they represent the diversity of current DSS research.
Usability must be defined specifically for the context of use of the particular system under investigation. This specific context of use should also be used to guide the definition of specific usability criteria and the selection of appropriate evaluation methods. There are four principles which can guide the selection of evaluation methods, relating to the information required in the evaluation, the stage at which to apply methods, the resources required and the people involved in the evaluation. This paper presents a framework for the evaluation of usability in the context of In-Vehicle Information Systems (IVISs). This framework guides designers through defining usability criteria for an evaluation, selecting appropriate evaluation methods and applying those methods. These stages form an iterative process of design-evaluation-redesign with the overall aim of improving the usability of IVISs and enhancing the driving experience, without compromising the safety of the driver.
A practical usability engineering process that can be incorporated into the software product development process to ensure the usability of interactive computer products is presented. It is shown that the most basic elements in the usability engineering model are empirical user testing and prototyping, combined with iterative design. Usability activities are presented for three main phases of a software project: before, during, and after product design and implementation. Some of the recommended methods are not really single steps but should be used throughout the development process.< >
User experience evaluation in an automotive context
- Moritz Körber
- Armin Eichinger
- Klaus Bengler
- Cristina Olaverri-Monreal
Moritz Körber, Armin Eichinger, Klaus Bengler, and Cristina Olaverri-Monreal.
2013. User experience evaluation in an automotive context. In 2013 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops). Institute of Electrical and
Electronics Engineers, Gold Coast City, Australia, 13-18. https://doi.org/10.1109/
IVWorkshops.2013.6615219
User Experience White Paper - Bringing clarity to the concept of user experience
- Virpi Roto
- Effie Lai-Chong Law
- Arnold P O S Vermeeren
- Jettie Hoonhout
- Roto Virpi
Virpi Roto, Effie Lai-Chong Law, Arnold P. O. S. Vermeeren, and Jettie Hoonhout.
2011. User Experience White Paper -Bringing clarity to the concept of user
experience. In Dagstuhl Seminar on Demarcating User Experience. Dagstuhl reports,
Dagstuhl, Germany, 1-26.
Towards Practical User Experience Evaluation Methods. Meaningful measures: Valid useful user experience measurement (VUUM)
- Kaisa Väänänen
- Virpi Roto
- Marc Hassenzahl
- Väänänen Kaisa
Kaisa Väänänen, Virpi Roto, and Marc Hassenzahl. 2008. Towards Practical
User Experience Evaluation Methods. Meaningful measures: Valid useful user
experience measurement (VUUM) (01 2008), 1-4.
Towards Practical User Experience Evaluation Methods. Meaningful measures: Valid useful user experience measurement (VUUM) (01 2008) 1-4. Kaisa Väänänen Virpi Roto and Marc Hassenzahl
- Virpi Kaisa Väänänen
- Marc Roto
- Hassenzahl