No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Description-Experience Gap: The Role of Feedback and Description in Human Trust in Automation
Abstract
Human trust in automation is widely studied because the level of trust influences the effectiveness of the system (Muir, 1994). It is vital to examine the role that the people play and how they interact with the system (Hoff & Bashir, 2015). In the decision-making literature, an interesting phenomenon is the description-experience gap, with a typical finding that experience-based choices underweight small probabilities, whereas description-based choices overweight small probabilities (Hertwig, Barron, Weber, & Erev, 2004; Hertwig & Erev, 2009; Jessup, Bishara, & Busemeyer, 2008). We applied this description-experience gap concept to the study of human-automation interaction and had Amazon Mechanical Turk workers evaluate emails as legitimate or phishing. An anti-phishing warning system provided recommendations to the user with a reliability level of 60%, 70%, 80%, or 90%. Additionally, the way in which reliability information was conveyed was manipulated with two factors: (1) whether the reliability level of the system was stated explicitly (i.e., description); (2) whether feedback was provided after the user made each decision (i.e., experience). Our results showed that as the reliability of the warning system increased, so did decision accuracy, agreement rate, self-reported trust, and perceived system reliability, consistent with prior research (Lee & See, 2004; Rice, 2009; Sanchez, Fisk, & Rogers, 2004). The increase in performance and trust with the increase in reliability indicates that participants were paying attention to and using the automation to make decisions. Feedback was also highly influential in performance and establishing trust, but description only affected self-reported trust. The effect of feedback strengthened at the higher levels of reliability, showing that individuals benefited the most from feedback when the automated warning system was more reliable. Additionally, unlike prior studies that manipulated description and experience/feedback separately (Hertwig, 2012), we varied description and feedback conditions systematically and discovered an interaction between the two factors. Our results show that feedback is more helpful in situations that do not provide an explicit description of the system reliability, compared to those who do. An implication of the current results for system design is that feedback should be provided whenever possible. This recommendation is based on the finding that providing feedback benefited both users’ performance and trust in the system, and on the hope that the systems in use are mostly of high reliability (e.g., > .80). A note for researchers in the field of human trust in automation is that, if only subjective measures of trust are used in a study, providing description of the system reliability will likely cause an inflation in the trust measures.
... In contrast, Seppelt and Lee (2019) showed the superiority of continuous feedback in visual displays relative to discrete warnings in fostering drivers' situational awareness and performance. Another study showed that Mechanical Turk workers' trust in automation and their performance is enhanced with feedback, especially when the reliability of the system is unknown (Mishler et al., 2017). ...
... This description-experience gap is a robust phenomenon that has been established through a substantial volume of literature (see [28] for a review). When investigating these two different methods of communicating systemreliability information in the applied area of human-automation interaction, critical differences can emerge, which researchers in this field need to consider [22], [29]. ...
Antiphishing aid systems, among other automated systems, are not perfectly reliable. Automated systems can make errors, thereby resulting in false alarms or misses. An automated system's capabilities need to be communicated to the users to maintain proper user trust. System capabilities can be learned through an explicit description or from experience. Using a phishing-detection system as a testbed in this article, we systematically varied automation error type and the method of communicating system reliability in a factorial design and measured their effects on human performance and trust in the automation. Participants were asked to classify emails as legitimate or phishing with assistance from the phishing-detection system. The results from 510 participants suggest that learning through experience with feedback improved trust calibration for both objective and subjective trust measures in most conditions. Moreover, false alarms lowered trust more than misses for both unreliable and reliable systems, and false alarms turned out to be beneficial for proper trust calibration when using unreliable systems. Design implications of the results include using feedback whenever possible and choosing false alarms over misses for unreliable systems.
Background
Emails have become an integral part of our daily life and work. Phishing emails are often disguised as trustworthy ones and attempt to obtain sensitive information for malicious reasons (Egelman, Cranor, Hong, 2008;). Anti-phishing tools have been designed to help users detect phishing emails or websites (Egelman, et al., 2008; Yang, Xiong, Chen, Proctor, & Li, 2017). However, like any other types of automation aids, these tools are not perfect. An anti-phishing system can make errors, such as labeling a legitimate email as phishing (i.e., a false alarm) or assuming a phishing email as legitimate (i.e., a miss). Human trust in automation has been widely studied as it affects how the human operator interacts with the automation system, which consequently influences the overall system performance (Dzindolet, Peterson, Pomranky, Pierce, & Beck, 2003; Lee & Moray, 1992; Muir, 1994; Sheridan & Parasuraman, 2006). With interacting with an automation system, the human operator should calibrate his or her trust level to trust a system that is capable but distrust a system that is incapable (i.e., trust calibration; Lee & Moray, 1994; Lee & See, 2004; McGuirl & Sarter, 2006). Among the various system capabilities, automation reliability is one of the most important factors that affect trust, and it is widely accepted that higher reliability levels lead to higher trust levels (Desai et al., 2013; Hoff & Bashir, 2015). How well these capabilities are conveyed to the operator is essential (Lee & See, 2004). There are two general ways of conveying the system capabilities: through an explicit description of the capabilities (i.e., description), or through experiencing the system (i.e., experience). These two ways of conveying information have been studied widely in human decision-making literature (Wulff, Mergenthaler-Canseco, & Hertwig, 2018). Yet, there has not been systematic investigation on these different methods of conveying information in the applied area of human-automation interaction (but see Chen, Mishler, Hu, Li, & Proctor, in press; Mishler et al., 2017). Furthermore, trust and reliance on automation is not only affected by the reliability of the automation, but also by the error types, false alarms and misses (Chancey, Bliss, Yamani, & Handley, 2017; Dixon & Wickens, 2006). False alarms and misses affect human performance in qualitatively different ways, with more serious damage being caused by false-alarmprone automation than by miss-prone automation (Dixon, Wickens, & Chang, 2004). In addition, false-alarm-prone automation reduces compliance (i.e., the operator’s reaction when the automation presents a warning); and miss-prone automation reduces reliance (i.e., the operator’s inaction when the automation remains silent; Chancey et al., 2017).
Current Study
The goal of the current study was to examine how the methods of conveying system reliability and automation error type affect human decision making and trust in automation. The automation system was a phishing-detection system, which provided recommendations to users as to whether an email was legitimate or phishing. The automation reliability was defined as the percentage of correct recommendations (60% vs. 90%). For each reliability level, there were a false-alarm condition, with all the automation errors being false alarms, and a miss condition, with all the errors being misses. The system reliability was conveyed through description (with an exact percentage described to the user) or experience (with immediate feedback to help the user learn; Barron, & Erev, 2003). A total of 510 participants were recruited and completed the experiment online through Amazon Mechanical Turk. The experimental task consisted of classifying 20 emails as phishing and legitimate, with a phishing-detection system providing recommendations. At the end of the experiment, participants rated their trust in this automated aid system. The measures included a performance measure (the decision accuracy made by the participants), as well as two trust measures (participants’ agreement rate with the phishing-detection system, and their self-reported trust in the system). Our results showed that higher system reliability and feedback increased accuracy significantly, but description or error type alone did not affect accuracy. In terms of the trust measures, false alarms led to lower agreement rates than did misses. With a less reliable system, though, the misses caused a problem of inappropriately higher agreement rates; this problem was reduced when feedback was provided for the unreliable system, indicating a trust-calibration role of feedback. Self-reported trust showed similar result patterns to agreement rates. Performance was improved with higher system reliability, feedback, and explicit description. Design implications of the results included that (1) both feedback and description of the system reliability should be presented in the interface of an automation aid whenever possible, provided that the aid is reliable, and (2) for systems that are unreliable, false alarms are more desirable than misses, if one has to choose between the two.
Scientists Making a Difference is a fascinating collection of first-person narratives from the top psychological scientists of the modern era. These readable essays highlight the most important contributions to theory and research in psychological science, show how the greatest psychological scientists formulate and think about their work, and illustrate how their ideas develop over time. In particular, the authors address what they consider their most important scientific contribution, how they got the idea, how the idea matters for the world beyond academic psychology, and what they would like to see as the next steps in research. The contributors, who were chosen from an objectively compiled list of the most eminent psychological scientists, provide a broad range of insightful perspectives. This book is essential reading for students, researchers and professionals interested in learning about the development of the biggest ideas in modern psychological science, described firsthand by the scientists themselves.
In lottery gambling, the common phenomenon of risk aversion shows up as preference of the option with the higher win probability, even if a riskier alternative offers a greater expected value. Because riskier choices would optimize profitability in such cases, the present study investigates the visual format, with which lotteries are conveyed, as potential instrument to modulate risk attitudes. Previous research has shown that enhanced attention to graphical compared to numerical probabilities can increase risk aversion, but evidence for the reverse effect — reduced risk aversion through a graphical display of outcomes — is sparse. We conducted three experiments, in which participants repeatedly selected one of two lotteries. Probabilities and outcomes were either presented numerically or in a graphical format that consisted of pie charts (Experiment 1) or icon arrays (Experiment 2 and 3). Further, expected values were either higher in the safer or in the riskier lottery, or they did not differ between the options. Despite a marked risk aversion in all experiments, our results show that presenting outcomes as graphs can reduce — albeit not eliminate — risk aversion (Experiment 3). Yet, not all formats prove suitable, and non-intuitive outcome graphs can even enhance risk aversion (Experiment 1). Joint analyses of choice proportions and response times (RTs) further uncovered that risk aversion leads to safe choices particularly in fast decisions. This pattern is expressed under graphical probabilities, whereas graphical outcomes can weaken the rapid dominance of risk aversion and the variability over RTs (Experiment 1 and 2). Together, our findings demonstrate the relevance of information format for risky decisions.
Two sets of design principles for analogical visual displays, based on the concepts of emergent features and perceptual objects, are described. An interpretation of previous empirical findings for three displays (bar graph, polar graphic, alphanumeric) is provided from both perspectives. A fourth display (configural coordinate) was designed using principles of ecological interface design (i.e. direct perception). An experiment was conducted to evaluate performance (accuracy and latency of state identification) with these four displays. Numerous significant effects were obtained and a clear rank ordering of performance emerged (from best to worst): configural coordinate, bar graph, alphanumeric and polar graphic. These findings are consistent with principles of design based on emergent features; they are inconsistent with principles based on perceptual objects. Some limitations of the configural coordinate display are discussed and a redesign is provided.
Objective: We systematically review recent empirical research on factors that influence trust in automation to present a three-layered trust model that synthesizes existing knowledge.
Background: Much of the existing research on factors that guide human-automation interaction is centered around trust, a variable that often determines the willingness of human operators to rely on automation. Studies have utilized a variety of different automated systems in diverse experimental paradigms to identify factors that impact operators’ trust.
Method: We performed a systematic review of empirical research on trust in automation from January 2002 to June 2013. Papers were deemed eligible only if they reported the results of a human-subjects experiment in which humans interacted with an automated system in order to achieve a goal. Additionally, a relationship between trust (or a trust-related behavior) and another variable had to be measured. All together, 101 total papers, containing 127 eligible studies, were included in the review.
Results: Our analysis revealed three layers of variability in human–automation trust (dispositional trust, situational trust, and learned trust), which we organize into a model. We propose design recommendations for creating trustworthy automation and identify environmental conditions that can affect the strength of the relationship between trust and reliance. Future research directions are also discussed for each layer of trust.
Conclusion: Our three-layered trust model provides a new lens for conceptualizing the variability of trust in automation. Its structure can be applied to help guide future research and develop training interventions and design procedures that encourage appropriate trust.
Smartphone users are often unaware of the data collected by apps running on their devices. We report on a study that evaluates the benefits of giving users an app permission manager and sending them nudges intended to raise their awareness of the data collected by their apps. Our study provides both qualitative and quantitative evidence that these approaches are complementary and can each play a significant role in empowering users to more effectively control their privacy. For instance, even after a week with access to the permission manager, participants benefited from nudges showing them how often some of their sensitive data was be-ing accessed by apps, with 95% of participants reassessing their permissions, and 58% of them further restricting some of their permissions. We discuss how participants interacted both with the permission manager and the privacy nudges, analyze the effective-ness of both solutions, and derive some recommendations.
IT security systems often attempt to support users in taking a decision by communicating associated risks. However, a lack of efficacy as well as problems with habituation in such systems are well known issues. In this paper, we propose to leverage the rich set of personal data available on smartphones to communicate risks using personalized examples. Examples of private information that may be at risk can draw the users' attention to relevant information for a decision and also improve their response. We present two experiments that validate this approach in the context of Android app permissions. Private information that becomes accessible given certain permissions is displayed when a user wants to install an app, demonstrating the consequences this installation might have. We find that participants made more privacy-conscious choices when deciding which apps to install. Additionally, our results show that our approach causes a negative affect in participants, which makes them pay more attention.
Smartphone security research has produced many useful tools to analyze the privacy-related behaviors of mobile apps. However, these automated tools cannot assess people's perceptions of whether a given action is legitimate, or how that action makes them feel with respect to privacy. For example, automated tools might detect that a blackjack game and a map app both use one's location information, but people would likely view the map's use of that data as more legitimate than the game. Our work introduces a new model for privacy, namely privacy as expectations. We report on the results of using crowdsourcing to capture users' expectations of what sensitive resources mobile apps use. We also report on a new privacy summary interface that prioritizes and highlights places where mobile apps break people's expectations. We conclude with a discussion of implications for employing crowdsourcing as a privacy evaluation technique.
In this paper, we discuss the use of visual representations to assist people in understanding complex information about sea level rise and climate change. We report on the results of a 2011 study in which we conducted plus-minus document usability evaluations of documents describing the mechanisms and consequences of sea-level rise in coastal areas. The protocol included 40 participant interviews and post interview quizzes. We tested with three documents, one that presented information for the U.S. southeastern coastal region and two that presented information "localized" for the two areas in which we conducted the research. Findings indicate that participants had difficulty with information presented in graphs and maps and that, while they indicated preferences for localized information, localized images did not improve understanding of complex information.
Depictions, such as maps, that portray visible things are ancient whereas graphics, such as charts and diagrams, that portray things that are inherently not visible, are relatively modern inventions. An analysis of historical and developmental graphic inventions suggests that they convey meaning by using elements and space naturally. Elements are based on likenesses, "figures of depiction" and analogs to physical devices. Spatial relations are used metaphorically to convey other relations, based on proximity, at nominal, ordinal, and interval levels. Graphics serve a variety of functions, among them, attracting attention, supporting memory, providing models, and facilitating inference and discovery.
Usability is most often defined as the acceptability of a system for a particular class of users carrying out specific tasks in specific environment. Ease of use affects performance of the user and his satisfaction, which finally regulates the wish of the user to use the product or not. Thus, it is of great importance for every software practitioner that he should not only be aware of varius usability methods, but also be able to determine the best method suited to a software project. In case of human-computer interaction (HCI), it is a must to determine the usability of the product before prototyping it.
The concept of making security decisions fundamental to design security features used by the users, is described. A series of decision-making scenarios were designed to systematically vary by decision domain, risk, and gain-to-loss ratio in an effort to determine how computer users might respond to potential security decisions. Fifty-six students enrolled at a public university volunteered to participate in a study that used a 2×2×3 repeated measures factorial design. The study used performance on a scenario-based decision task to draw conclusions about how risk and gain-to-loss ratio might affect decision-making within the domains of computing and non-computing security decisions. Combining the evaluation approach with potential alterations of security warnings should allow designers to improve security systems.
Displays comparing the performance of healthcare providers are largely based on commonsense. To review the literature on the impact of compositional format and content of quantitative data displays on people's comprehension, choice and preference.
Ovid databases, expert recommendations and snowballing techniques.
Evaluations of the impact of different formats (bar charts, tables and pictographs) and content (ordering, explanatory visual cues, etc.) of quantitative data displays meeting defined quality criteria. Data extraction Type of decision; decision-making domains; audiences; formats; content; methodology; findings.
Most of the 30 studies used quantitative (n= 26) methods with patients or public groups (n= 28) rather than with professionals (n= 2). Bar charts were the most frequent format, followed by pictographs and tables. As regards format, tables and pictographs appeared better understood than bar charts despite the latter being preferred. Although accessible to less numerate and older populations, pictographs tended to lead to more risk avoidance. Tables appeared accessible to all. Aspects of content enhancing the impact of data displays included giving visual explanatory cues and contextual information while still attempting simplicity ('less is more'); ordering data; consistency. Icons rather than numbers were more user-friendly but could lead to over-estimation of risk. Uncertainty was not widely understood, nor well represented.
Though heterogeneous and limited in scope, there is sufficient research evidence to inform the presentation of quantitative data that compares the performance of healthcare providers. The impact of new formats, such as funnel plots, needs to be evaluated.
A recent study demonstrated that individuals making experience-based choices underweight small probabilities, in contrast to the overweighting observed in a typical descriptive paradigm. We tested whether trial-by-trial feedback in a repeated descriptive paradigm would engender choices more correspondent with experiential or descriptive paradigms. The results of a repeated gambling task indicated that individuals receiving feedback underweighted small probabilities, relative to their no-feedback counterparts. These results implicate feedback as a critical component during the decision-making process, even in the presence of fully specified descriptive information. A model comparison at the individual-subject level suggested that feedback drove individuals' decision weights toward objective probability weighting.
This paper 1) provides reasons why graphics should be effective aids to communicate risk; 2) reviews the use of visuals, especially graphical displays, to communicate risk; 3) discusses issues to consider when designing graphs to communicate risk; and 4) provides suggestions for future research. Key articles and materials were obtained from MEDLINE(R) and PsychInfo(R) databases, from reference article citations, and from discussion with experts in risk communication. Research has been devoted primarily to communicating risk magnitudes. Among the various graphical displays, the risk ladder appears to be a promising tool for communicating absolute and relative risks. Preliminary evidence suggests that people understand risk information presented in histograms and pie charts. Areas that need further attention include 1) applying theoretical models to the visual communication of risk, 2) testing which graphical displays can be applied best to different risk communication tasks (e.g., which graphs best convey absolute or relative risks), 3) communicating risk uncertainty, and 4) testing whether the lay public's perceptions and understanding of risk varies by graphical format and whether the addition of graphical displays improves comprehension substantially beyond numerical or narrative translations of risk and, if so, by how much. There is a need to ascertain the extent to which graphics and other visuals enhance the public's understanding of disease risk to facilitate decision-making and behavioral change processes. Nine suggestions are provided to help achieve these ends.
When people have access to information sources such as newspaper weather forecasts, drug-package inserts, and mutual-fund brochures, all of which provide convenient descriptions of risky prospects, they can make decisions from description. When people must decide whether to back up their computer's hard drive, cross a busy street, or go out on a date, however, they typically do not have any summary description of the possible outcomes or their likelihoods. For such decisions, people can call only on their own encounters with such prospects, making decisions from experience. Decisions from experience and decisions from description can lead to dramatically different choice behavior. In the case of decisions from description, people make choices as if they overweight the probability of rare events, as described by prospect theory. We found that in the case of decisions from experience, in contrast, people make choices as if they underweight the probability of rare events, and we explored the impact of two possible causes of this underweighting--reliance on relatively small samples of information and overweighting of recently sampled information. We conclude with a call for two different theories of risky choice.
This article describes a model of communication known as crisis and emergency risk communication (CERC). The model is outlined as a merger of many traditional notions of health and risk communication with work in crisis and disaster communication. The specific kinds of communication activities that should be called for at various stages of disaster or crisis development are outlined. Although crises are by definition uncertain, equivocal, and often chaotic situations, the CERC model is presented as a tool health communicators can use to help manage these complex events.
There have long been speculations that graphical and numerical presentations of risk statistics differ in their impact on people's wilingness to pursue actions that could harm or even kill them. But research has been unclear about the processes whereby the pictorial character of graphical displays per se might affect those risky decisions or even whether such effects actually occur. In two studies, we demonstrate that the pictorial nature of a graphical risk display can, indeed, increase risk avoidance. This increase is associated with a heightened impression of the riskiness of less safe alternatives. The results suggest that this picture-driven, intensified sense of riskiness, in turn, rests on two kinds of mechanisms: one cognitive, the other affective. Cognitively, pictorial presentations impose weaker upper bounds on people's internal representations of the chances that riskier alternatives will bring about actual harm. Affectively, pictures ignite stronger, more aversive negative associations with riskier options and their outcomes.
Automation is often problematic because people fail to rely upon it appropriately. Because people respond to technology socially, trust influences reliance on automation. In particular, trust guides reliance when complexity and unanticipated situations make a complete understanding of the automation impractical. This review considers trust from the organizational, sociological, interpersonal, psychological, and neurological perspectives. It considers how the context, automation characteristics, and cognitive processes affect the appropriateness of trust. The context in which the automation is used influences automation performance and provides a goal-oriented perspective to assess automation characteristics along a dimension of attributional abstraction. These characteristics can influence trust through analytic, analogical, and affective processes. The challenges of extrapolating the concept of trust in people to trust in automation are discussed. A conceptual model integrates research regarding trust in automation and describes the dynamics of trust, the role of context, and the influence of display characteristics. Actual or potential applications of this research include improved designs of systems that require people to manage imperfect automation. Copyright © 2004, Human Factors and Ergonomics Society. All rights reserved.
Individuals communicate and form relationships through Internet social networking websites such as Facebook and MySpace. We study risk taking, trust, and privacy concerns with regard to social networking websites among 205 college students using both reliable scales and behavior. Individuals with profiles on social networking websites have greater risk taking attitudes than those who do not; greater risk taking attitudes exist among men than women. Facebook has a greater sense of trust than MySpace. General privacy concerns and identity information disclosure concerns are of greater concern to women than men. Greater percentages of men than women display their phone numbers and home addresses on social networking websites. Social networking websites should inform potential users that risk taking and privacy concerns are potentially relevant and important concerns before individuals sign-up and create social networking websites.
We conducted three experiments with participants recruited on Amazon’s Mechanical Turk to examine the influence on app-installation decisions of summary risk information derived from the app permissions. This information can be framed negatively as amount of risk or positively as amount of safety, which was varied in all the experiments. In Experiments 1 and 2, the participants performed tasks in which they selected two Android apps from a list of six; in Experiment 3, the tasks were to reject two apps from the list. This summary information influenced the participants to choose less risky alternatives, particularly when it was framed in terms of safety and the app had high user ratings. Participants in the safety condition reported that they attended more to the summary score than did those in the risk condition. They also showed better comprehension of what the score was conveying, regardless of whether the task was to select or reject. The results imply that development of a valid risk/safety index for apps has the potential to improve users’ app-installation decisions, especially if that information is framed as amount of safety.
A central question within the domain of human cognition is whether or not the ability to replace a current action with a new one (i.e., cognitive control) depends on a conscious appreciation of the environmental change that necessitates the new behavior. Specifically, it is not yet known if non-consciously perceived stimuli can trigger the modification of a currently ongoing action. We show for the first time that individuals are able to use non-consciously perceived information to modify the course and outcome of an ongoing action. Participants were presented with a masked (i.e., subliminal) 'stop' or 'go-on' prime stimulus whilst performing a routine reach-to-touch action. Despite being invisible to participants, the stop primes produced more hesitations mid-flight and more movement reversals than the go-on primes. This new evidence directly establishes that cognitive control (i.e., the ability to modify a currently ongoing action) does not depend on a conscious appreciation of the environmental trigger.
Smartphones have unprecedented access to sensitive personal information. While users report having privacy concerns, they may not actively consider privacy while downloading apps from smartphone application marketplaces. Currently, Android users have only the Android permissions display, which appears after they have selected an app to download, to help them understand how applications access their information. We investigate how permissions and privacy could play a more active role in app-selection decisions. We designed a short "Privacy Facts' display, which we tested in a 20-participant lab study and a 366-participant online experiment. We found that by bringing privacy information to the user when they were making the decision and by presenting it in a clearer fashion, we could assist users in choosing applications that request fewer permissions.
Traditional user-based permission systems assign the user's full privileges to all applications. Modern platforms are transitioning to a new model, in which each application has a different set of permissions based on its requirements. Application permissions offer several advantages over traditional user-based permissions, but these benefits rely on the assumption that applications generally require less than full privileges. We explore whether that assumption is realistic, which provides insight into the value of application permissions. We perform case studies on two platforms with application permissions, the Google Chrome extension system and the Android OS. We collect the permission requirements of a large set of Google Chrome extensions and Android applications. From this data, we evaluate whether application permissions are effective at protecting users. Our results indicate that application permissions can have a positive impact on system security when applications' permission requirements are declared up-front by the developer, but can be improved.
The popularity and advanced functionality of mobile devices has made them attractive targets for malicious and intrusive applications (apps). Although strong security measures are in place for most mobile systems, the area where these systems often fail is the reliance on the user to make decisions that impact the security of a device. As our prime example, Android relies on users to understand the permissions that an app is requesting and to base the installation decision on the list of permissions. Previous research has shown that this reliance on users is ineffective, as most users do not understand or consider the permission information. We propose a solution that leverages a method to assign a risk score to each app and display a summary of that information to users. Results from four experiments are reported in which we examine the effects of introducing summary risk information and how best to convey such information to a user. Our results show that the inclusion of risk-score information has significant positive effects in the selection process and can also lead to more curiosity about security-related information.
We evaluated alternative scrolling methods on non-touch screen computer operating systems by comparing human performance in different scrolling conditions.
The scrolling directions on current operating systems are discrepant. Few researchers have investigated how scrolling method influences users performance. The response-effect (R-E) compatibility principle can be used as a theoretical guide.
Experiments 1 and 2 involved two successive tasks (scrolling and target content judgment) to simulate how people scroll to acquire and use off-screen information. Performance in R-E compatible and incompatible conditions was compared. Experiment 3 involved a location judgment task to test the influence of target location. Experiments 4 and 5 included a scrolling effect following the location judgment task to test the sufficient role of the scrolling effect.
Overall, responses were facilitated when the response direction was compatible with the forthcoming display-content movement direction (an R-E compatibility effect), when the scrolling effect was task relevant or task irrelevant. A spatial stimulus-response (S-R) compatibility effect attributable to target location was also found. When the scrolling effect was present, there were both R-E and S-R components; the R-E effect was the larger of the two.
Scrolling in the direction of content movement yielded the best performance, and the scrolling effect was the main source of the R-E compatibility effect.
These findings suggest that (a) the R-E compatibility principle may be used as a general design guideline for scrolling and (b) a consistent scrolling method should be available on various operating systems.
Each time a user installs an application on their Android phone they are presented with a full screen of information describing what access they will be granting that application. This information is intended to help them make two choices: whether or not they trust that the application will not damage the security of their device and whether or not they are willing to share their information with the ap- plication, developer, and partners in question. We performed a series of semi-structured interviews in two cities to determine whether people read and understand these permissions screens, and to better understand how people perceive the implications of these decisions. We find that the permissions displays are generally viewed and read, but not understood by Android users. Alarmingly, we find that people are unaware of the security risks associated with mobile apps and believe that app mar- ketplaces test and reject applications. In sum, users are not currently well prepared to make informed privacy and security decisions around installing applications.
Trust has been identified by previous research as a key determinant of automation reliance and usage (Lee & Moray, 1992). One factor that may affect trust and reliance on automation is the reliability of the automation (Parasuraman 1993; Riley, 1996). The effects of automation reliability and age on perceived reliability, trust, and reliance were investigated. A driving-like task was created and the reliability of the automation was manipulated by generating three levels (100%, 80% and 60%). Automation was present in the form of a decision support system that indicated the state of the gauges. Results indicated that high levels of automation reliability lead to increased reliance and higher subjective levels of trust. There were age-related effects on the ability to perceive the reliability of the automation and levels of trust where older adults were more sensitive to the change between 80% and 60% reliability than the younger adults.
This study examines Wickens' compatibility of proximity hypothesis of visual display design and proposes that an emergent-features approach might carry more explanatory power. Two studies show that a bar graph display is superior to an object display in an integration task if the bar graph has a strong emergent feature that maps directly onto a goal-relevant task invariant. Earlier results by Wickens and colleagues showing an object display advantage could not be replicated or generalized. These new findings suggest that object display advantages occur only under limited conditions and that the exploitation of emergent features may more predictably lead to an advantage of one display over another.
Smartphone users increasingly download and install third-party applications from official application repositories. Attackers may use this centralized application delivery architecture as a security and privacy attack vector. This risk increases since application vetting mechanisms are often not in place and the user is delegated to authorize which functionality and protected resources are accessible by third-party applications. In this paper, we mount a survey to explore the security awareness of smartphone users who download applications from official application repositories (e.g. Google Play, Apple's App Store, etc.). The survey findings suggest a security complacency, as the majority of users trust the app repository, security controls are not enabled or not added, and users disregard security during application selection and installation. As a response to this security complacency we built a prediction model to identify users who trust the app repository. The model is assessed, evaluated and proved to be statistically significant and efficient.
Preference can be inferred from direct choice between options or from a matching procedure in which the decision maker adjusts one option to match another. Studies of preferences between two-dimensional options (e.g., public policies, job applicants, benefit plans) show that the more prominent dimension looms larger in choice than in matching. Thus, choice is more lexicographic than matching. This finding is viewed as an instance of a general principle of compatibility: The weighting of inputs is enhanced by their compatibility with the output. To account for such effects, we develop a hierarchy of models in which the trade-off between attributes is contingent on the nature of the response. The simplest theory of this type, called the contingent weighting model, is applied to the analysis of various compatibility effects, including the choice-matching discrepancy and the preference-reversal phenomenon. These results raise both conceptual and practical questions concerning the nature, the meaning and the assessment of preference. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
In 3 experiments, the authors examined alternative formats for displaying low-probability risk information and the effects of these formats on professed risk-taking behavior. In Experiment 1, participants who were presented with numerically displayed risk information stated that they would be willing to pay less money to reduce a risk than were participants given the identical information by means of stick figures. Experiments 2 and 3 evaluated 3 potential explanations for this finding by including additional formats where the risk information was displayed as asterisks, bar graphs, or faces. The data did not support explanations focusing on the discrete character of stick figures or their possible humanizing nature but instead suggested that the graphical nature of stick figures increased risk avoidance. These results suggest that depicting risk information graphically as opposed to numerically is a potentially useful technique for decreasing risk-taking behavior. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Conducted 3 experiments in which undergraduate males (N = 261) chose their preferred bet from pairs of bets and later bid for each bet separately. In each pair, 1 bet had a higher probability of winning (P bet); the other offered more to win ($ bet). Bidding method (selling vs. buying) and payoff method (real-play vs. hourly wage) were varied. Results show that when the P bet was chosen, the $ bet often received a higher bid. It is concluded that these inconsistencies violate every risky decision model, but can be understood via information-processing considerations. In bidding, S starts with amount to win and adjusts it downward to account for other attributes of the bet. In choosing, there is no natural starting point: amount to win dominates bids but not choices. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
The study investigates the visual salience of information in making differential predictions for alpha-numeric versus graphic displays in consumer decision making. The experiment partially supports predictions that with alpha-numeric displays, information is acquired in correspondence with the importance weights of the attributes, whereas under graphic conditions, information is acquired in correspondence with the visual salience of the attributes. The information display form appears to have some effect not only on the temporal order in which information on attributes is acquired, but also on the relative attention given to information on attributes during the early phases of decision making.
Throughout social and cognitive psychology, participants are routinely asked to respond in some way to experimental stimuli that are thought to represent categories of theoretical interest. For instance, in measures of implicit attitudes, participants are primed with pictures of specific African American and White stimulus persons sampled in some way from possible stimuli that might have been used. Yet seldom is the sampling of stimuli taken into account in the analysis of the resulting data, in spite of numerous warnings about the perils of ignoring stimulus variation (Clark, 1973; Kenny, 1985; Wells & Windschitl, 1999). Part of this failure to attend to stimulus variation is due to the demands imposed by traditional analysis of variance procedures for the analysis of data when both participants and stimuli are treated as random factors. In this article, we present a comprehensive solution using mixed models for the analysis of data with crossed random factors (e.g., participants and stimuli). We show the substantial biases inherent in analyses that ignore one or the other of the random factors, and we illustrate the substantial advantages of the mixed models approach with both hypothetical and actual, well-known data sets in social psychology (Bem, 2011; Blair, Chapleau, & Judd, 2005; Correll, Park, Judd, & Wittenbrink, 2002).
This paper provides an introduction to mixed-effects models for the analysis of repeated measurement data with subjects and items as crossed random effects. A worked-out example of how to use recent software for mixed-effects modeling is provided. Simulation studies illustrate the advantages offered by mixed-effects analyses compared to traditional analyses based on quasi-F tests, by-subjects analyses, combined by-subjects and by-items analyses, and random regression. Applications and possibilities across a range of domains of inquiry are discussed.
To test optimal graphic risk communication formats for presenting small probabilities using graphics with a denominator of 1000 to adults with lower education and literacy.
A randomized experimental study, which took place in adult basic education classes in Sydney, Australia. The participants were 120 adults with lower education and literacy. An experimental computer-based manipulation compared 1) pictographs in 2 forms, shaded "blocks" and unshaded "dots"; and 2) bar charts across different orientations (horizontal/vertical) and numerator size (small <100, medium 100-499, large 500-999). Accuracy (size of error) and ease of processing (reaction time) were assessed on a gist task (estimating the larger chance of survival) and a verbatim task (estimating the size of difference). Preferences for different graph types were also assessed.
Accuracy on the gist task was very high across all conditions (>95%) and not tested further. For the verbatim task, optimal graph type depended on the numerator size. For small numerators, pictographs resulted in fewer errors than bar charts (blocks: odds ratio [OR] = 0.047, 95% confidence interval [CI] = 0.023-0.098; dots: OR = 0.049, 95% CI = 0.024-0.099). For medium and large numerators, bar charts were more accurate (e.g., medium dots: OR = 4.29, 95% CI = 2.9-6.35). Pictographs were generally processed faster for small numerators (e.g., blocks: 14.9 seconds v. bars: 16.2 seconds) and bar charts for medium or large numerators (e.g., large blocks: 41.6 seconds v. 26.7 seconds). Vertical formats were processed slightly faster than horizontal graphs with no difference in accuracy. Most participants preferred bar charts (64%); however, there was no relationship with performance.
For adults with low education and literacy, pictographs are likely to be the best format to use when displaying small numerators (<100/1000) and bar charts for larger numerators (>100/1000).
Sumario: Today many systems are highly automated. The human operator's role in these systems is to supervise the automation and intervene to take manual control when necessary. The operator's choice of automatic or manual control has important consequences for system performance, and therefore it is important to understand and optimize this decision process. In this paper a model of human trust in machines is developed, taking models of trust between people as a starting point, and extending them to the human-machine relationship
According to a common conception in behavioral decision research, two cognitive processes-overestimation and overweighting-operate to increase the impact of rare events on people's choices. Supportive findings stem primarily from investigations in which people learn about options via descriptions thereof. Recently, a number of researchers have begun to investigate risky choice in settings in which people learn about options by experiential sampling over time. This article reviews work across three experiential paradigms. Converging findings show that when people make decisions based on experience, rare events tend to have less impact than they deserve according to their objective probabilities. Striking similarities in human and animal experience-based choices, ways of modeling these choices, and their implications for risk and precautionary behavior are discussed.
The author examined the effects of human responses to automation alerts and nonalerts. Previous research has shown that automation false alarms and misses have differential effects on human trust (i.e., automation false alarms tend to affect operator compliance, whereas automation misses tend to affect operator reliance). Participants performed a simulated combat task, whereby they examined aerial photographs for the presence of enemy targets. A diagnostic aid provided a recommendation during each trial. The author manipulated the reliability and response bias of the aid to provide appropriate data for state-trace analyses. The analyses provided strong evidence that only a multiple-process theory of operator trust can explain the effects of automation errors on human dependence behaviors. The author discusses the theoretical and practical implications of this finding.
Communicating probability information about risks to the public is more difficult than might be expected. Many studies have examined this subject, so that their resulting recommendations are scattered over various publications, diverse research fields, and are about different presentation formats. An integration of empirical findings in one review would be useful therefore to describe the evidence base for communication about probability information and to present the recommendations that can be made so far. We categorized the studies in the following presentation formats: frequencies, percentages, base rates and proportions, absolute and relative risk reduction, cumulative probabilities, verbal probability information, numerical versus verbal probability information, graphs, and risk ladders. We suggest several recommendations for these formats. Based on the results of our review, we show that the effects of presentation format depend not only on the type of format, but also on the context in which the format is used. We therefore argue that the presentation format has the strongest effect when the receiver processes probability information heuristically instead of systematically. We conclude that future research and risk communication practitioners should not only concentrate on the presentation format of the probability information but also on the situation in which this message is presented, as this may predict how people process the information and how this may influence their interpretation of the risk.
To evaluate the ability of six graph formats to impart knowledge about treatment risks/benefits to low and high numeracy individuals.
Participants were randomized to receive numerical information about the risks and benefits of a hypothetical medical treatment in one of six graph formats. Each described the benefits of taking one of two drugs, as well as the risks of experiencing side effects. Main outcome variables were verbatim (specific numerical) and gist (general impression) knowledge. Participants were also asked to rate their perceptions of the graphical format and to choose a treatment.
2412 participants completed the survey. Viewing a pictograph was associated with adequate levels of both types of knowledge, especially for lower numeracy individuals. Viewing tables was associated with a higher likelihood of having adequate verbatim knowledge vs. other formats (p<0.001) but lower likelihood of having adequate gist knowledge (p<0.05). All formats were positively received, but pictograph was trusted by both high and low numeracy respondents. Verbatim and gist knowledge were significantly (p<0.01) associated with making a medically superior treatment choice.
Pictographs are the best format for communicating probabilistic information to patients in shared decision making environments, particularly among lower numeracy individuals.
Providers can consider using pictographs to communicate risk and benefit information to patients of different numeracy levels.
Integrative, objectlike displays have been advocated for presenting multidimensional system data. In this research two experiments assess the effect of uncertainty on the processing of integral and separable displays. In each experiment 30 subjects were trained to classify instances of system state into one of four state categories using a configural display, a bar graph display, or a digital display. In Experiment 1 the range of instances from the state categories was uniform; in Experiment 2 the distribution was biased toward those instances of highly uncertain state category membership. After training, subjects received extended practice classifying system data. In both experiments uncertainty was found to have the greatest effect on classification performance. In Experiment 1 the bar graph display was consistently superior; the configural display was superior to the digital display only under conditions of low uncertainty. In Experiment 2 the superiority of the bar graph display diminished, producing results equivalent to those of the digital display, with the configural display producing the worst performance. The effect of uncertainty on classification performance is discussed, with specific attention paid to the apparent configural and separable properties of the bar graph display.
The study was designed to determine which formats for displaying quantities, such as probabilities of treatment risks and benefits, are perceived most accurately and easily by patients. Accuracy and speed of processing were compared for six different presentation formats: pie charts, vertical bars, horizontal bars, numbers, systematic ovals, and random ovals. Quantities were used in two tasks: a choice task that required larger/smaller judgments and an estimate task that required more precise evaluation. The impacts of blue-yellow color and of a treatment-decision context on performance in the two tasks were also investigated. The study included four experiments. Taken together the results suggest that the formats best for making a choice differ from those best for estimating the size of an amount. For making a choice, vertical bars, horizontal bars, numbers, and systematic ovals were equally well perceived; pie charts and random ovals caused slower and less accurate performances. For estimating, numbers led to the most accurate estimates, followed by systematic ovals. The other four formats led to the least accurate estimates. Color and context did not alter which formats were best.