Content uploaded by Yeti Li
Author content
All content in this area was uploaded by Yeti Li on Dec 08, 2017
Content may be subject to copyright.
A preview of the PDF is not available
Exploring Automated Trading: Modeling a Financial System with a Variable Degree of Automation, Display Design, and Evaluation
Abstract and Figures
Automation has become more pervasive and started to replace human operation in all information-processing stages (e.g., machine-learning based systems). Typically, automated systems have a variable degree of automation (DOA) that challenges modeling and design. Cognitive Work Analysis (CWA) is a modeling approach to support analysts in coping with the complexity of socio-technical domains and has shown success in providing implications for developing ecological displays that are effective in unanticipated situations. CWA allows space for modeling automated systems but has not been well developed to describe variable DOAs. This dissertation explores this problem by focusing on the case of automated trading which is underexplored in both human factors research and finance research. This dissertation starts with developing a modeling approach then focuses on design and evaluation. Modeling. A DOA layering approach on the decision ladder was developed to serve as the theoretical foundation of this research. Two cases of automated trading – basket trading and trend following trading, each using a different DOA – were presented to facilitate the development of this approach. With this approach, the two most commonly used modeling tools in the CWA, the abstraction hierarchy, and the decision ladder each adopted an additional layer for representing human-administrated functions and automated functions. Also, the four information-processing stages in which the automation could take place were marked on the decision ladder to allow for a more detailed level mapping, which is unique in the CWA research. The DOA layering approach was demonstrated to extend the use case of CWA to include automated systems with a variable DOA and have important implications for ecological display design as well as automation design. Design and evaluation. The experimental approach presented later in this dissertation further explored automation and display design implications of the DOA layering approach using AUTRASS (AUtomated TRAding System Simulation), a simulation developed as part of this research program. Two experimental studies on trend following trading are reported in which the design concepts were evaluated. In Experiment 1, inspired by the CWA models, automation was designed as two configurations to represent distinct DOA situations. The moderate DOA configuration simulated a trading situation in which the participants performed a flexible trading task. The high DOA configuration represented a higher DOA situation where a trading algorithm that was unfamiliar to the participants traded in a similar market condition, and the participants monitored the automation and performed a fault detection task. Two types of displays were designed. Conventional displays were typical in information content and form to current trading displays and should support the basic use of the automation. Ecological displays were implemented from the CWA models to support monitoring for unanticipated situations for each DOA condition. Four scenario types were developed by combing the two DOA configurations with the two display types. Experiment 1 involving 24 participants was conducted to thoroughly examine the effectiveness of ecological displays with different DOAs. Based on the literature, the ecological displays were hypothesized to improve task performance and situation awareness and to trigger riskier actions without imposing higher workload. Results of Experiment 1showed that the ecological displays did not provide better support on either trading performance (moderate DOA) or fault detection performance (high DOA). Empirically, a trade-off of situation awareness and workload between the two DOA configurations seemed to exist without the influence of the ecological displays. Interestingly, the results of this study suggested a different pattern of risk preference compared to that in the literature. Specifically, the ecological displays imposed riskier financial trading decisions. The results of Experiment 1 provided implications for identifying system and contextual factors that could influence risk preference and demonstrated sufficient space for improving automation design. Experiment 2 is a follow-up to the first experimental study with separately recruited 24 participants. The high DOA configuration previously used in Experiment 1 was improved with its traits inherited and its flaws in automation design eliminated (i.e., the improved-high DOA configuration was expected to be better supported by the ecological displays). An adaptive configuration was introduced to simulate adaptive automation in the automated trading setting. The conventional displays and the ecological displays continued to be used without any modifications, and similar hypotheses were examined with the two new DOA configurations. Results of Experiment 2 showed that with the improved automation design in the high DOA configuration, the ecological display significantly improved fault detection performance. A consistent pattern of risk preference was found in this study as in Experiment 1. A comparison of the two experimental studies showed new opportunities to derive automation and display design from the DOA layering approach that can support fault detection performance in automated trading, and future research is warranted to explore the influence of ecological displays on risk preference. The development of AUTRASS also makes a unique contribution. The design of the different DOA configurations demonstrated the applicability of the DOA layering approach to guide automation design. Overall, the following conclusions were reached by conducting this dissertation research: CWA has been effective in characterizing the complexity in automated trading that is associated with the variable DOA, and it can further support the design of automation and the ecological displays. Ecological displays may foster risky operation with moderate DOA as well as performance improvement with high DOA. The applicability of the proposed approaches spanning modeling, design, and evaluation should go beyond the limit of automated trading to the brave new world of artificially intelligent automation.
Figures - uploaded by Yeti Li
Author content
All figure content in this area was uploaded by Yeti Li
Content may be subject to copyright.
ResearchGate has not been able to resolve any citations for this publication.
Laboratory studies have shown that ecological interfaces can enhance operator performance in process control. However, the nuclear industry needs empirical evaluation in representative settings in order to justify the adoption of the Ecological Interface Design (EID) framework. This report presents an empirical study as a first step towards the validation of EID in the nuclear domain. The empirical study compared the operator task and workload performance of ecological displays against mimic-based displays for the turbine side of a boiling water reactor plant. The results suggest that ecological displays have a marked advantage in supporting operator performance during monitoring for unanticipated events as compared to mimic-based displays. This study provides supporting or validation evidence that EID is effective at a scale and level of complexity that is representative of nuclear power plant operations.
Automation is often problematic because people fail to rely upon it appropriately. Because people respond to technology socially, trust influences reliance on automation. In particular, trust guides reliance when complexity and unanticipated situations make a complete understanding of the automation impractical. This review considers trust from the organizational, sociological, interpersonal, psychological, and neurological perspectives. It considers how the context, automation characteristics, and cognitive processes affect the appropriateness of trust. The context in which the automation is used influences automation performance and provides a goal-oriented perspective to assess automation characteristics along a dimension of attributional abstraction. These characteristics can influence trust through analytic, analogical, and affective processes. The challenges of extrapolating the concept of trust in people to trust in automation are discussed. A conceptual model integrates research regarding trust in automation and describes the dynamics of trust, the role of context, and the influence of display characteristics. Actual or potential applications of this research include improved designs of systems that require people to manage imperfect automation. Copyright © 2004, Human Factors and Ergonomics Society. All rights reserved.
Automation has been rapidly developing into a pervasive part of our every day lives. Although I agree with Kaber’s original article, I argue that human factors as a discipline is not keeping up with the pace of technological change. Human factors researchers must rapidly embrace the development of richer automation models, more complex laboratory studies, and naturalistic studies in the field to generate relevant insights into human automation interaction. The corresponding development of massive data collection presents an opportunity for a more data-driven approach to understanding human automation interaction and human factors in general.
The current cognitive engineering literature includes a broad range of models of human-automation interaction (HAI) in complex systems. Some of these models characterize types and levels of automation (LOAs) and relate different LOAs to implications for human performance, workload, and situation awareness as bases for systems design. However, some have suggested that the LOAs approach has overlooked key issues that need to be considered during the design process. Others are simply unsatisfied with the current state of the art in modeling HAI. In this paper, I argue that abandoning an existing framework with some utility for design makes little sense unless the cognitive engineering community can provide the broader design community with other sound alternatives. On this basis, I summarize issues with existing definitions of LOAs, including (a) presumptions of human behavior with automation and (b) imprecision in defining behavioral constructs for assessment of automation. I propose steps for advances in LOA frameworks. I provide evidence of the need for precision in defining behavior in use of automation as well as a need for descriptive models of human performance with LOAs. I also provide a survey of other classes of HAI models, offering insights into ways to achieve descriptive formulations of taxonomies of LOAs to support conceptual and detailed systems design. The ultimate objective of this line of research is reliable models for predicting human and system performance to serve as a basis for design.
Ecological Interface Design delivers the techniques and examples that provide you with a foundation to succeed in designing advanced display graphics. The opening chapters introduce the “art” of interface design by exposing the analytical methods behind designs, the most common graphical forms, and how these methods and forms are pulled together to create a complete design. The book then incorporates case studies that further emphasize techniques and results. Each example exemplifies a solution to a certain part of the EID puzzle. Some of the examples demonstrate the analysis phase, while others apply more scrutiny to graphical design. Each is unique, allowing allowing you to use them in the development of your own designs. The volume concludes with an analysis that connects ecological interface design with other common interface design methods, enabling you to better understand how to combine approaches in the creation of design solutions.
Cognitive work analysis is useful to develop displays for complex situations, but it has not been well explored in providing support for human-automation coordination. To fill this gap, we propose a degree of automation (DOA) layering approach, demonstrated by modeling an automated financial trading domain, with a goal of supporting interface design in this domain. The abstraction hierarchy and the decision ladder each adopted an additional layer, mapping functions allocated to the trader and to the automation. In addition to the mapping, we marked the four stages of automation on the decision ladder to provide guidance on representing the function allocation at the task level. Next, we compared the DOA layering approach to how automation was represented in the cognitive work analysis literature. We found that a DOA-layered decision ladder, which included well-developed knowledge of the stages and levels of automation, can be suited to modern automated systems with different DOAs. This study suggests that the DOA layering approach has important implications for designing automation displays and deciding stages and levels of automation and may be a useful approach for modeling adaptive automation.
Autonomous and semiautonomous vehicles are currently being developed by over14 companies. These vehicles may improve driving safety and convenience, or they may create new challenges for drivers, particularly with regard to situation awareness (SA) and autonomy interaction. I conducted a naturalistic driving study on the autonomy features in the Tesla Model S, recording my experiences over a 6-month period, including assessments of SA and problems with the autonomy. This preliminary analysis provides insights into the challenges that drivers may face in dealing with new autonomous automobiles in realistic driving conditions, and it extends previous research on human-autonomy interaction to the driving domain. Issues were found with driver training, mental model development, mode confusion, unexpected mode interactions, SA, and susceptibility to distraction. New insights into challenges with semiautonomous driving systems include increased variability in SA, the replacement of continuous control with serial discrete control, and the need for more complex decisions. Issues that deserve consideration in future research and a set of guidelines for driver interfaces of autonomous systems are presented and used to create recommendations for improving driver SA when interacting with autonomous vehicles.
This paper discusses the ways in which automation of industrial processes may expand rather than eliminate problems with the human operator. Some comments will be made on methods of alleviating these problems within the 'classic' approach of leaving the operator with responsibility for abnormal conditions, and on the potential for continued use of the human operator for on-line decision-making within human-computer collaboration.
As autonomous driving emerges, it is important to understand drivers' experiences with autonomous cars. We report the results of an online survey with Tesla owners using two autonomous driving features, Autopilot and Summon. We found that current users of these features have significant driving experience, high self-rated computer expertise and care about how automation works. Surprisingly, although automation failures are extremely common they were not perceived as risky. The most commonly occurring failures included the failure to detect lanes and uncomfortable speed changes of the vehicle. Additionally, a majority of the drivers emphasized the importance of being alert while driving with autonomous features and aware of the limitations of the current technology. Our main contribution is to provide a picture of attitudes and experiences towards semi-autonomous driving, revealing that some drivers adopting these features may not perceive autonomous driving as risky, even in an environment with regular automation failures.