No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Situation Awareness and campaign assessments
Abstract
Military-led nation building campaigns can face many challenges as advances in certain aspects such as the economy and infrastructure may well be accompanied by setbacks in other areas such as security, making it difficult to judge overall progress. In this context, campaign assessments provide vital decision support for adapting strategy and deploying resources to best effect. The value of such assessments is enhanced when they report not only what has happened but why and when they also indicate likely future events—mirroring the three levels in Endsley’s model of Situation Awareness. Our aim is to develop a process for evaluating changes to the complex organisational structure and processes used for producing assessment documents, even if it is difficult to understand the linkages between such changes and output quality. Using 2 years’ worth of Australian Defence Force quarterly campaign assessments, we study the effect of adoption of a new analysis framework in their production. Using the conventions of thematic analysis and Endsley’s model of Situation Awareness, we introduce the Situation Awareness Elicitation method for obtaining metrics pertaining to the value of assessments before and after introduction of the framework and perform statistical analysis on this data.
... Generally, the commanders can only monitor BS intermittently and periodically through sampling [16,17]. In the process of monitoring, if the changes of BS are beyond the range that the commanders can predict, the commanders need to shorten the monitoring interval time to improve the understanding of BS. ...
Commanders are aware of the battlefield situation (BS) by monitoring the battlefield periodically. When the monitoring interval period is too short, there will be a huge cognitive load, which seriously affects commanders’ situation awareness (SA). When the monitoring interval period is too long, there will be omission of important information, which also affects commanders’ SA. Therefore, it is necessary to set the reasonable interval period of monitoring battlefield. In this paper, BS signal is defined to represent BS changes, according to the characteristics of BS changes. Then Fourier transform (FT) is used to analyze frequency characteristics of BS signal in order to describe the complicatedness of BS changes. The discriminant of reasonable monitoring interval period (DRMIP) is derived, according to the commanders’ requirements accuracy of BS. When the complicatedness of BS changes cannot be obtained previously, the highest frequency iteration algorithm (HFIA) is proposed to obtain the best monitoring interval period according to iterative BS signal. In the case of setting the threshold of difference, the validity of proposed method was verified by simulation calculation method.
In order to realize autonomy and intelligence for situation awareness, this paper proposes an intelligent situation awareness model based on fractional dimension information mining and multi-dimensional information reconstruction. First, the spatial situation perception is established by 3D reconstruction of the input fusion information, 4D completes the situation comprehension and 5D reconstruction seeks the situation prediction. The three-level situation estimation model is optimized to a more robust situation estimation ternary model. Combined with the database system, the reasoning learning mechanism and the diversified human-machine interface concept, a basic framework of intelligent situation awareness is completed. Secondly, the system’s flexible configuration method is demonstrated. Thirdly, some basic evaluation methods are given for the intelligent situation awareness’s performance metrics and multi-node consistency. Fourthly, the paper gives a typical electromagnetic situation estimation example for a drilling platform to explain and validate the theory. Finally, several suggestions are put forward for the next construction of intelligent situation awareness system.
The purpose of this paper is to propose foundations for a theory of situation awareness based on the analysis of interactions between agents (i.e. both human and non-human) in subsystems. This approach may help to promote a better understanding of technology-mediated interaction in systems, as well as helping in the formulation of hypotheses and predictions concerning distributed situation awareness. It is proposed that agents within a system each hold their own situation awareness, which may be very different from (although compatible with) that of other agents. It is argued that we should not always hope for, or indeed want, sharing of this awareness, as different system agents have different purposes. This view marks situation awareness as a dynamic and collaborative process binding agents together on tasks on a moment-by-moment basis. Implications of this viewpoint for the development of a new theory of, and accompanying methodology for, distributed situation awareness are offered.
paper can be retrieved from:
http://www.inform.nu/Articles/Vol17/ISJv17p059-073Ali0476.pdf
Between good intentions and great results lies a program theory—not just a list of tasks but a vision of what needs to happen, and how. Now widely used in government and not-for-profit organizations, program theory provides a coherent picture of how change occurs and how to improve performance. Purposeful Program Theory shows how to develop, represent, and use program theory thoughtfully and strategically to suit your particular situation, drawing on the fifty-year history of program theory and the authors' experiences over more than twenty-five years.
"From needs assessment to intervention design, from implementation to outcomes evaluation, from policy formulation to policy execution and evaluation, program theory is paramount. But until now no book has examined these multiple uses of program theory in a comprehensive, understandable, and integrated way. This promises to be a breakthrough book, valuable to practitioners, program designers, evaluators, policy analysts, funders, and scholars who care about understanding why an intervention works or doesn't work." —Michael Quinn Patton, author, Utilization-Focused Evaluation
"Finally, the definitive guide to evaluation using program theory! Far from the narrow 'one true way' approaches to program theory, this book provides numerous practical options for applying program theory to fulfill different purposes and constraints, and guides the reader through the sound critical thinking required to select from among the options. The tour de force of the history and use of program theory is a truly global view, with examples from around the world and across the full range of content domains. A must-have for any serious evaluator." —E. Jane Davidson, PhD, Real Evaluation Ltd.
Preview of 61 pages available here https://books.google.com.au/books/about/Purposeful_Program_Theory.html?id=zBoGbWnVsFsC&printsec=frontcover&source=kp_read_button&redir_esc=y#v=onepage&q&f=false
We propose a simple checklist for the users of policy supporting research in order to decide whether a piece of research begs further study or can be dismissed right away. The checklist focusses on the quality of the research question (is it a research question, and is the research question answerable); the kind of knowledge along with the order, level and quality of data needed for answering the RQ; the methods of analysis used; the degree to which the research results support the conclusions; and whether the conclusions provide an answer to the research question.
One of the most problematic issues for researchers who conduct qualitative research using semi-structured, unstructured interviews or story telling data collection methods is the analysis of large quantities of rich data. In the past this has often led to fairly unmethodical approaches to analysis which in turn has led to qualitative business and management research being seen as insubstantial and unworthy of consideration. A relatively recent development in organisational research has been the application of Template Analysis to rich unstructured qualitative data following the primary data collection phase. Template Analysis appears to have emerged from the USA during the 1990s and academics familiar with the Grounded Theory approach to data analysis may see similarities in the techniques used. Nevertheless, it has gained credibility in the UK through the work of Nigel King and other colleagues researching in health and sociology related fields. This paper provides an overview of the origins of Template Analysis and discusses how it has been used to structure qualitative data. It then goes on to examine through the two case studies how Template Analysis has been extended and used by the authors in two different research projects. In the first case study the research team worked within a Primary Care Trust in the North East of England on a project that explored the Diffusion of Innovation of clinical and administrative computer systems across General Practice within the Trust. Seventeen Trust members were interviewed for approximately one hour and this led to over 85000 words of rich data. The second project focused on the NHS Secondary Care sector and examined IT project management practice related to the development of integrated pathology computing systems across eight separate laboratories in the North of England. Eight senior managers were interviewed and this, combined with participant observation and over 3 years of document collection, also resulted in a large volume of rich textual material. The use of template analysis, combined with a critical success factors methodology, resulted in a novel approach for learning about current IT project management practices. This paper critically examines these two case studies in terms of their particular research philosophy, epistemological approach and the lessons learnt from the techniques employed. The paper then provides a discussion of the principles and practicalities of template analysis and explores the benefits to the business and management research community at large.
How are decision practices fostered in organizations and how are they linked to decision outcomes? This study addresses these questions by examining one financial institution's efforts to standardize and control decision making across geographically separated organizational units. We argue that decision-maker behavior is situated and is not simply a function of individual choice. Rather, in organizational settings decision-makers are subject to a hierarchy of influences that affect the decision processes they use and their resulting decision choices. To test our ideas, we examined 900 borrower "risk rating" decisions and found general support for our hypotheses. Decision makers were more likely to use the prescribed practice when decisions were important, when the decision target was known, and when the decision maker was located in a larger subunit. Decision makers altered their decision practices in the short term, but in the long term they appeared to partially revert to their earlier practices. Reliance on prescribed practice fostered stability in decisions, but surprisingly appeared to negatively affect future judgments. The findings indicate that organizations can change the more microaspects of decision making, but these changes may be transitory. Moreover the results suggest that decision makers may become complacent when they rely on prescribed decision practices, a tendency that can have untoward consequences for the organizations in which they are embedded.
This paper presents a theoretical model of situation awareness based on its role in dynamic human decision making in a variety of domains. Situation awareness is presented as a predominant concern in system operation, based on a descriptive view of decision making. The relationship between situation awareness and numerous individual and environmental factors is explored. Among these factors, attention and working memory are presented as critical factors limiting operators from acquiring and interpreting information from the environment to form situation awareness, and mental models and goal-directed behavior are hypothesized as important mechanisms for overcoming these limits. The impact of design features, workload, stress, system complexity, and automation on operator situation awareness is addressed, and a taxonomy of errors in situation awareness is introduced, based on the model presented. The model is used to generate design implications for enhancing operator situation awareness and future directions for situation awareness research.
Situation awareness (SA) is an important component of pilot/system performance in all types of aircraft. It is the role of the human factors engineer to develop aircraft cockpits which will enhance SA. Research in the area of situation awareness is is vitally needed if system designers are to meet the challenge of providing cockpits which enhance SA. This paper presents a discussion of the SA construct, important considerations facing designers of aircraft systems, and current research in the area of SA measurement.
Methodologies for the empirical measurement of situation awareness are reviewed, including a discussion of the advantages and disadvantages of each method and the potential limitations of the measures from a theoretical and practical viewpoint. Two studies are presented that investigate questions of validity and intrusiveness regarding a query-based technique. This technique requires that a simulation of the operational tasks be momentarily interrupted in order to query operators on their situation awareness. The results of the two studies indicate that the query technique is not intrusive on normal subject behavior during the trial and does not suffer from limitations of human memory, which provides an indication of empirical validity. The results of other validity studies regarding the technique are discussed along with recommendations for its use in measuring situation awareness in varied settings.
Command and control is a generic activity involving the exercise of authority over assigned resources, combined with planning, coordinating and controlling how those resources are used. The challenge for understanding this type of activity is that it is not often amenable to the conventional experimental/methodological approach. Command and control tends to be multi-faceted (so requires more than one method), is made up of interacting socio and technical elements (so requires a systemic approach) and exhibits aggregate behaviours that emerge from these interactions (so requires methods that go beyond reductionism). In these circumstances a distributed cognition approach is highly appropriate yet the existing ethnographic methods make it difficult to apply and, for non-specialist audiences, sometimes difficult to meaningfully interpret. The Event Analysis for Systemic Teamwork method is put forward as a means of working from a distributed cognition perspective but in a way that goes beyond ethnography. A worked example from Air Traffic Control is used to illustrate how the language of social science can be translated into the language of systems analysis. Statement of Relevance: Distributed cognition provides a highly appropriate conceptual response to complex work settings such as Air Traffic Control. This paper deals with how to realise those benefits in practice without recourse to problematic ethnographic techniques.
C4i is defined as the management infrastructure needed for the execution of a common goal supported by multiple agents in multiple locations and technology. In order to extract data from complex and diverse C4i scenarios a descriptive methodology called Event Analysis for Systemic Teamwork (EAST) has been developed. With over 90 existing ergonomics methodologies already available, the approach taken was to integrate a hierarchical task analysis, a coordination demand analysis, a communications usage diagram, a social network analysis, and the critical decision method. The outputs of these methods provide two summary representations in the form of an enhanced operation sequence diagram and a propositional network. These offer multiple overlapping perspectives on key descriptive constructs including who the agents are in a scenario, when tasks occur, where agents are located, how agents collaborate and communicate, what information is used, and what knowledge is shared. The application of these methods to live data drawn from the UK rail industry demonstrates how alternative scenarios can be compared on key metrics, how multiple perspectives on the same data can be taken, and what further detailed insights can be extracted. The ultimate aim of EAST is, by applying it across a number of scenarios in different civil and military domains, to provide data to develop generic models of C4i activity and to improve the design of systems aimed at enhancing this management infrastructure.
This paper provides a review and evaluation of the use of problem structuring methods (PSMs) in practice. It starts by describing the origins of PSMs, the type of problem situation for which they are suitable, and the characteristics of some leading methods. An overview of the practice of PSMs is provided from a number of angles, including case studies and surveys of applications. A number of issues in the application of PSMs are discussed, in particular an account of the debate about evaluation of the success of PSMs; the selection of an appropriate method; multimethodology; and a variety of aspects of the maintenance of relationships with the client organisation(s). Finally, some possible future developments are suggested, especially through productive interactions with similar or related practices.
This book examines the theoretical and conceptual foundation of effective modern intelligence collectionâthe strategies required to support intelligence analysis of the modern, complex operational environments of today's military conflicts or competitive civilian situations such as business.
Just as the old rules of conventional warfare and intelligence analysis do not apply fully in the 21st-century environment, neither does the traditional methodology of collecting intelligence on these elusive, adapting foes operating as complex adaptive systems (CAS)âadversaries that excel in today's complex contexts.
Intelligence Collection: How To Plan and Execute Intelligence Collection In Complex Environments proposes substantive improvements in the way the U.S. national security system collects intelligence and supports intelligence analysis. The work draws on the groundbreaking work of a diverge group of theorists ranging from Carl von Clausewitz and Sun Tzu to M. Mitchell Waldrop, General David Petraeus, and Orson Scott Card, communicating a unifying theory and ontology of thought for how America's intelligence collection professionals must learn to collect data as our country faces elusive, determined, and smart adversaries in nonlinear, dynamic environments. The new ideas presented will help the nation's intelligence collection specialists to amass a formidable, cumulative intelligence power, regardless of the level of war or the type of operational environment.
Advances in military information and systems technologies have greatly improved warfighters' situational awareness on the battlefield and ability to make critical decisions in a timely manner. System integration is an important enabler for improving planning and execution of operations via availability of timely, relevant information and synchronization of activity. The Coalition Attack Guidance Experiment (CAGE) aims to assess potential joint fires capabilities and the resulting operational impacts, which can be effective to meet challenges from new and evolving threats. This paper highlights results from a recent CAGE activity and key lessons learnt to improve future endeavours. Of primary importance is the need to consider the human dimension and extant processes for effective system integration.
Successful implementation of change in a complex enterprise requires a shared understanding of system interdependency. Otherwise, the architecture of those changes risk the emergence of otherwise unforeseen obstacles. A fundamental element is understanding how the system responds to stimuli. Given the complexity of that system, no single model would adequately represent the totality of the enterprise. As such, we have employed a structured approach based on soft systems methodology and reference models to develop common pictures. These heuristic models act as anchor points for achieving a shared understanding and as a basis for the development of more detailed models. The approach has been applied to defence preparedness; a system containing many levels of inter-dependency, contested by a range of differing viewpoints, multilayered with decisions and activity at a number of levels, and often seeking to rapidly transition to solutions. We present some examples of distinct but inter-related reference models for defence preparedness.
David Snowden's Cynefin framework, introduced to articulate discussions of sense-making, knowledge management and organisational learning, has much to offer discussion of statistical inference and decision analysis. I explore its value, particularly in its ability to help recognise which analytic and modelling methodologies are most likely to offer appropriate support in a given context. The framework also offers a further perspective on the relationship between scenario thinking and decision analysis in supporting decision makers.
Despite well over two decades of academic research, situation awareness in complex collaborative environments remains ambiguous. Recent systems-based models of distributed situation awareness have attempted to address this; however, further effort is required in clarifying exactly how situation awareness operates in complex socio-technical systems. One pertinent issue regards what the ‘distributed’ situational picture looks like from the perspective of each component of the system and how it differs amongst team members. Is the same picture ‘shared’ across team members, or do they have different, but compatible, views on the situation? This article tackles this question by presenting a distributed cognition-inspired systems-based model of situation awareness in collaborative systems, along with an accompanying modelling approach, the propositional network methodology. Following this, the paper focuses explicitly on the model's sub-concept of compatible situation awareness and uses case study evidence to demonstrate how awareness is distributed between, and differs between, team members. This view is then compared to existing team situation awareness views and the implications for collaborative system design are discussed.
Situation awareness has long been recognized as an important variable in aviation performance. Research to date has focused on identifying characteristics of situation awareness for individuals, not on the behaviors and processes associated with team situation awareness. The purpose of this review is to delineate and identify characteristics of team situation awareness. In addition, implications are discussed and research questions are outlined that target the measurement and training of situation awareness in teams.
Our world is a myriad of ever-changing, interdependent variables whose courses we can never entirely predict. The strength of an effects-based approach to operations is that it squarely addresses these complexities by concentrating on their most nonlinear aspects: humans, their institutions, and their actions. Indeed, the entire effects-based approach can be characterized by four things: a focus on the human dimension of competition and conflict; the consideration of a full spectrum of actions whether in peace, crisis, or hostilities; a multifaceted, whole-of-nation concept of power; and the recognition of the complex interconnected nature of the actors and challenges involved. The human dimension arises because all effects-based approaches are ultimately about shaping human perceptions and behavior, and because they depend heavily on human beings to make the complex estimates and decisions involved. The focus on an entire spectrum of actions means thinking holistically across a peace-crisis-hostilities spectrum. Finally, any effects-based approach must proceed from the recognition that all actions and the reactions they provoke are inextricably linked in a system of ever-changing and adapting human systems whose complexity shapes both the nature of the problem and the task of assessing, planning, and executing any operation. The central tenet of an effects-based approach to operations is that we can somehow purposefully shape the interactions of the actors in this complex security environment. Living systems theory offers a way of approaching this complexity. It sees the world in biological and sociological terms as an interlocking multilevel system of complex adaptive systems from which no individual system can be extracted without changing both its character and that of the system as a whole. In the model, interactions occur simultaneously on many different levels with each interaction tending to proceed at a pace dictated by local circumstances.
The search for scientific bases for confronting problems of social policy is bound to fail, becuase of the nature of these problems. They are wicked problems, whereas science has developed to deal with tame problems. Policy problems cannot be definitively described. Moreover, in a pluralistic society there is nothing like the undisputable public good; there is no objective definition of equity; policies that respond to social problems cannot be meaningfully correct or false; and it makes no sense to talk about optimal solutions to social problems unless severe qualifications are imposed first. Even worse, there are no solutions in the sense of definitive and objective answers.
John Boyd is often known exclusively for the so-called 'OODA' loop model he developed. This model refers to a decision-making process and to the idea that military victory goes to the side that can complete the cycle from observation to action the fastest. This book aims to redress this state of affairs and re-examines John Boyd's original contribution to strategic theory. By highlighting diverse sources that shaped Boyd's thinking, and by offering a comprehensive overview of Boyd's work, this volume demonstrates that the common interpretation of the meaning of Boyd's OODA loop concept is incomplete. It also shows that Boyd's work is much more comprehensive, richer and deeper than is generally thought. With his ideas featuring in the literature on Network Centric Warfare, a key element of the US and NATO's so-called 'military transformation' programmes, as well as in the debate on Fourth Generation Warfare, Boyd continues to exert a strong influence on Western military thinking. Dr Osinga demonstrates how Boyd's work can helps us to understand the new strategic threats in the post- 9/11 world, and establishes why John Boyd should be regarded as one of the most important (post)modern strategic theorists.
A theory based framework for critical thinking in defence planning and assessment
- E Mazourenko
- M Jobst
Australian Defence Doctrine Publication (ADDP) 5.0-Joint Planning. Edition 2. Commonwealth of Australia
- Department Of Defence
Effects Based Operations. US Department of Defense
- E Smith
Logic assumption and risks framework applied to campaign planning and evaluation
- A Hickman
- R Karlsson
Australian Defence Doctrine Publication (ADDP) 3.0-Campaigns and Operations
- Defence Department
Clausewitzian Friction and Future War (revised edition) McNair Paper Series. Institute for National Strategic Studies
- B Watts
- Smith E
statistics and decision analysisJournal of the Operational Research
- Frenchscynefin
Institute for National Strategic Studies
- B Watts
- Endsley M