ArticlePDF Available

Abstract and Figures

In this paper we present three cognitive acts analysts use while solving criminal cases: intuition, leap of faith and insight. We used the Emergent Themes Analysis to find out how these cognitive acts help analysts’ inference making and what are their features. We analyzed the interviews with six analysts that were done with the Critical Decision Method. What we discovered is that intuition, leap of faith and insight relate to each other creating an integrated process while solving problems. We propose that a leap of faith occurs between intuition and insight. This is a preliminary study that we plan to develop further. However, current results are interesting enough that we hope the findings may help us design computer systems that will facilitate the process of solving criminal cases by analysts.Criminal intelligence analysis, analytical reasoning, inference making, intuition, leap of faith, insight, critical decision method.
Content may be subject to copyright.
How Analysts Think: Intuition, Leap of Faith and Insight
Matylda Gerber1, B.L. William Wong2, and Neesha Kodagoda2
1Warsaw School of Economics, 02-554 Warsaw, Poland
2Interaction Design Centre, Middlesex University
London NW4 4BT, United Kingdom
In this paper we present three cognitive acts analysts use while solving criminal cases: intuition, leap of
faith and insight. We used the Emergent Themes Analysis to find out how these cognitive acts help
analystsinference making and what are their features. We analyzed the interviews with six analysts that
were done with the Critical Decision Method. What we discovered is that intuition, leap of faith and insight
relate to each other creating an integrated process while solving problems. We propose that a leap of faith
occurs between intuition and insight. This is a preliminary study that we plan to develop further. However,
current results are interesting enough that we hope the findings may help us design computer systems that
will facilitate the process of solving criminal cases by analysts.
Criminal intelligence analysis, analytical reasoning,
inference making, intuition, leap of faith, insight, critical
decision method.
How do analysts think? In an earlier paper (Wong and
Kodagoda, 2015) we presented a number of inference making
strategies performed by criminal intelligence analysts when
solving crimes. We reported that analysts use and combine
abduction, induction and deduction methods depending on
data presented to them, their knowledge and goals at the time,
and in some cases, creating and using stories where data is
absent or unclear to progress a line of inquiry.
In this paper we describe three cognitive acts that help
analysts solve cases: intuition, leap of faith and insight. Their
importance have been considered in the process of decision-
making, especially in situations where there are only uncertain
data available and that decisions need to be made under high
time constraints (Klein 1997, Hogarth 2001, Merritt 2011).
Fautua & Schatz (2012) emphasized that intuition is a strategic
element of analysis. It indicates which elements the analysis
should focus on. Moreover, Klein (1997) in presenting the
Recognition Primed Decision (RPD) model, he showed the
importance of intuition in recognizing a pattern, from which
the most suitable behavior is recalled from previous
experience. Then by mental simulation, the person decides
whether to implement, modify or reject a given approach. The
leap of faith is perceived to be crucial in uncertain situations,
such as where an individual has no cues that could help him
understand a given problem nor foresee possible consequences
of actions taken (Moellering, 2007). Merritt (2011)
emphasized that the decision to use a machine under uncertain
conditions could not be done without a leap of faith that it will
work in the new situation the same way as it did in the old
situation. Contrary to intuition and leap of faith, insight is
derived not only from experience but also from collected data.
Insight is helpful to find a new solution when existing patterns
cannot be applied. In cases when there is not enough
information to find a solution deliberately, insight appears to
be the only process that can provide the answer (Sternberg &
Davidson 1996; Klein 2013).
According to the Data-Frame theory (Klein et al. 2007),
intuition allows one to respond quickly to a given situation by
drawing from one’s experience the most suitable frame. The
frame sets boundaries upon the direction of focus and allows
to interpret obtained information easily. The frame provides
initial comprehension of the situation and as more data are
collected, the frame can be confirmed, modified or replaced by
a new one. If we look closely at the Data-Frame model, we
can see when intuition, leap of faith and insight appear in the
decision making process. Intuition allows one to recognize a
situation as being similar to one of the frames stored in
memory. Since intuition communicates only through emotions
(Damasio, 1994) and it is acquired not only through explicit
but also implicit learning (Evans, 2008; Hogarth, 2001; Reber,
1993), there appears another process, which explains what
intuition might be about. Our observations suggest that this is
the leap of faith that interprets the understanding derived
through intuition and motivates the decision maker to provide
analysis in a particular direction. In the further process, when
more information is acquired, the leap of faith might be
confirmed, modified or contradicted. If the leap of faith is
modified or contradicted, it is a moment when a new frame is
created which appears in the form of insight. Insight arises on
the basis of a combination of one’s experience and newly
gathered data (Sternberg & Davidson 1996).
Intuition has been defined as "affectively charged
judgments that arise through rapid non-conscious and holistic
associations" (Dane, Pratt 2007, p.40). It has been found to be
a helpful process in finding important cues in uncertain
situations that lead to solving problems. Intuition has its roots
in emergency threat alert situations where there is a need for a
fast response at an exposure to risk. As a result of this, there is
a direct connection between the Long-Term Memory and the
body with avoidance of the conscious part of the brain
(LeDoux, 1994). It is perceived that while facing a problem
our brain unconsciously analyses our experience in search for
information about possible consequences the situation may
lead to. Then it provides information in the form of a high or
low frequency nerve impulse flow, called Somatic Markers.
As a result of the impulse flow, the decision maker receives
information by being stressed or relaxed (Damasio, 1994).
That is apparently why when intuition spots an interesting
pattern it communicates the finding directly through feelings.
There is a divergence of opinion about reliability of intuition
in experts’ decision making. Kahneman (2013), perceives
intuition to cause errors in decision making whereas Klein
(2004) sees it as fundamental and accurate strategy experts’
utilize in their everyday decision making. Kahneman and
Klein (2009) have tired to combine their findings. They came
to a common conclusion that, depending on the situation,
intuition may lead to errors, or provide a good solution. When
the environment is unpredictable (stock exchange, predicting
political changes or long term weather forecast) experts’
intuition might mislead them because there are to many factors
influencing the situation that one’s experience alone is not
enough to foresee the variety of possible outcomes. However,
in situations such as those faced by fire fighters, nuclear power
plant controllers, Navy or Army officers, one’s experience has
a higher value on assessing situation and intuition has been
found to be very helpful in making quick and accurate
decisions. Hogarth (2001) also agrees that to utilize one’s
intuition the decision makers need to have significant
experience to make decisions in environment that is similar to
situations already experienced. Moreover, he adds that to use
intuition reliably, the decision maker needs to obtain good
feedback about the consequences of his or her previous
decisions. Among scientists who do research about intuition,
there is an even bolder about the use of intuition. Gigerenzer
(2000) did a series of studies where he showed that in
uncertain situations where there are no reliable cues, intuition
might be the best form of decision making. Even if only based
on simple heuristics, decision making could be rational. The
number of pieces of information considered is not as important
as the accuracy of a given decision-making strategy to a
certain situation (Arkes, Gigerenzer, Herting, 2016). Less
processing while decision making in uncertain environment
seem to enable one to make accurate inferences and this is
crucial to adapting the mind to unfamiliar situations
(Gigerenzer, and Brighton, 2009).
The term ‘leap of faithhas been used in the scientific
world. However, to the best of our knowledge, there have been
no attempts to define its meaning and its role in the process of
decision-making. Leap of faith has been described as a belief
in something without evidence (Salas, 2006) that reflects
affective reactions (Merritt, 2011). Taking into account that
intuition appears to act before deliberate data collection and it
communicates its outcomes by feelings, the interpretation of
intuition should be done on the basis of a person’s experience.
As such we have formulated the definition of leap of faith as
an interpretation of intuitive judgments that arise from
experience consistent with perceptions of a current situation.
We suggest that a leap of faith provides a temporary
understanding of what might have happened or what is going
to happen. Since this belief is not supported by evidence, the
decision maker is aware that the solution is uncertain.
According to Ancona (2012) regardless of whether an initial
comprehension of a situation is right or wrong, this belief
helps one to start searching for a solution. We find this leap of
faith plays a crucial role in uncertain situations: regardless of
whether it is right or wrong, it enables one to start the analysis.
There is no doubt that insight is a very important cognitive
act in experts’ decision making. Insights lead to right solutions
and even though we may not know what exactly preceded the
gaining of insight, we can justify the resulting outcome from
the collected facts (Sternberg & Davidson, 1996). Insight has
been defined as "sudden unexpected thoughts that solve
problems" (Hogarth 2001, p. 251) and "an unexpected shift in
the way we understand things" (Klein, 2007). Insight provides
a comprehension of a situation as a result of the unconscious
merging of experience with newly collected data (Sternberg &
Davidson, 1996). According to Klein (2013) there are three
types of insight. The first one appears as we identify
inconsistency in gathered data; the second arises from
collecting data that are integrated into a solution; and the third
is the result of escaping an impasse by looking at the same
problem from a different perspective.
We used a qualitative data analysis approach called the
Emergent Themes Analysis (Wong, 2004) on transcripts of six
in-depth interviews with police criminal intelligence analysts
from the UK and Belgium. The analysts have an average
experience of 12 years. The interviews were conducted using
the Critical Decision Method (Klein, Calderwood et al. 1989),
a retrospective interview technique for eliciting participants’
knowledge, decision strategies and cues used in their work. At
the beginning of our data analysis we searched for two major
themes; intuition and insight. This enabled us to see what roles
intuition and insight played in the analysts’ inference making
process and when they appear. We then further categorized
them to reveal features and how they relate to each other.
Through this analysis, we discovered one more process
between intuition and insight. This inspired us to introduce the
notion of a leap of faith which allowed us to propose how
the three cognitive acts relate to each other. All the instances
when intuition, leap of faith and insight helped analysts in
decision making were extracted from the transcripts and
analysed in a table that contains following information: (i)
what data an investigator possessed at a given moment, (ii)
what information was used (iii) what goal was to be achieved,
(iv) what premises led to intuitions, leaps of faith or insights,
(v) how the current cognitive act relates to previous intuitions,
leaps of faith or insights (vi) to what claims intuition, leap of
faith or insight lead, (vii) what cues were considered after the
claim, (viii) what was further considered, and (ix) what were
the next steps.
To help structure our results, we present a framework in
Figure 1 that describes how criminal intelligence analysts
make decisions in situations of varying familiarity. For this
paper, we focus our discussion on the decision making path
where analysts invoke intuition, leap of faith and insight to
deal with the uncertainty faced in moderately familiar
situations. In such situations the conditions present incomplete
data, requiring the analyst to make suppositions to create data
to make the situation more recognisable. Intuition enables
them to identify a pattern, which becomes an anchor for
further analysis. Given the lack of evidence, the decision
maker takes a leap of faith to define a possible outcome based
on one’s experience. This enables the analyst to take the next
step, to deliberately collect and analyse further information.
In parallel with the deliberate analysis, there appears to be also
an unconscious analysis that connects gathered data with
previously identified patterns. This connection of new data
with the old pattern leads to the emergence of a new pattern in
form of insight (a fuller description of the model has been
reported in Gerber, Wong, Kodagoda, submitted).
Table 1 presents the characteristics of intuition, leap of
faith and insight, supported by excerpts from interviews with
the six analysts. The characteristics of those cognitive acts
were grouped into seven categories: 1) how do intuition, leap
of faith and insight, relate to a pattern recognition, 2) what is
their main function in the decision making process, 3) how do
they relate to a process of creating hypothesis, 4) what
influence they have on deliberate thinking 5) at which point of
decision making process do they appear, 6) how confident is
the decision maker that the information generated by those
cognitive acts is certain and 7) what are those cognitive acts
based on. We summarise the features of intuition, leap of faith
and insight below.
Intuition plays an important role when analysts deal with
uncertain data. It enables one to spot information that may be
helpful in solving a problem [see 1.1 in Table 1]. It provides
an anchor to start a line of inquiry using feelings [1.1.]
immediately after considering the data [4.1], [5.1]. It helps
analysts to start an investigation [2.1]. Despite it is often
difficult to explain why, the decision maker feels confident to
focus on it and to investigate the recognized pattern [3.1],
[6.1]. The reliability of intuition depends on the analysts’
expertise in a given domain, because they only recognize
patterns that are similar to the ones already experienced. That
is why some analysts can see important information within
data that are meaningless for others [7.1].
Leap of faith appears immediately after intuition [5.2],
where it asserts an interpretation what the feeling provided by
intuition could be about [1.2]. It creates opportnities to reach a
temporary comprehension of the situation [2.2] by providing
the most probable hypothesis for the current situation [3.2]. As
a leap of faith is based only on one’s experience [7.2], the
decision maker is aware that the solution is uncertain [6.2].
This, however, does not deny that the decision maker is
motivated to provide analysis towards leap of faith as it is
often the only way to deal with suppositions generated to fill
in for missing data. Its main role is not necessarily to give a
final comprehension of the problem but to indicate where to
direct analysis to find solution [4.2].
Insight appears in the form of a new pattern [1.3] as a
result of modification or contradiction of a leap of faith [3.3].
It occurs unexpectedly while collecting data towards and
beyond the leap of faith [5.3] by providing a comprehension of
the situation [2.3]. Insight is the outcome not only of
unconscious analysis but also of deliberate ones [4.3]. It is the
result of matching experience with newly acquired
information [7.3]. That is why, an analyst can support his
solution by previously collected data and he or she is
confident that the decision is right [6.3]. From these three
cognitive acts insight brings the most important outcome,
because the reasoning behind could be supported by gathered
information. However, it seems that insight cannot occur
without previous occurrence of intuition and leap of faith. The
new pattern that is created while gaining insight arises on the
basis of the old pattern that was recognized by intuition and
interpreted by leap of fait.
In this paper we have presented three cognitive acts used
by analysts while solving criminal cases that appear in a
sequence: intuition, leap of faith and insight. As intuition and
insight have previously been defined, it represents the first
attempt to define and isolate leap of faith.
Figure 1. The model of decision making as an adaptation of the RPD model (Klein, 1997) and the decision ladder (Rasmussen, 1990) to analysts’ environment
Table 1 Characteristic of intuition, leap of faith and insight
Categories of
Leap of faith
1. Relation to
a pattern
[1.1] pattern recognition communicated
by feelings
[1.2] interpretation of recognized pattern
[1.3] new pattern creation
550 "P1: I ... it's one of those ...! It's one of
those ... I don't know. I just saw it.
I: Where ...?
P1: I'm looking at the data, the patterns,
because we'd have like, the series sorts of
starts, and literally just outside the area
of where we were having the offences, that
night, we were having a VRM theft."
372 "P1: I thought about it, they came in and
hit us hard. They way I was thinking was
they were trying to set up shop, initially.
That they wanted quite a lot of satnavs to be
able to sell on. It is almost like building up
their stock for their market. That's the way I
was thinking they were working, when I
looked at it."
349 "P2: I looked up internet because I don’t know the
community [...] yes! there is something like a healing
and I said yes! this is possible that she has been healed
over here. […] Now I say knowing on she might be a
lesbian. The spiritual healer has been called by the
mother [...] to cleanse the girl. She has been cleansed
once, twice, three times, four times"
51 "P6:A lot of it comes from hunches. if
you dont follow your feeling of where it is
going you can tie your self down with tons
and tons of data in one dataset"
334 "P2: The lack of information is also
information. I said he didn’t want to be
disturbed over there."
109 "P1: So, it is probably better if you got an empty
car at that point. Because even if you get stopped by
police officers, and you haven't committed any offences,
that's ok, because, you still got your other areas to go
2. Main
[2.1] indicates where to focus on
[2.2] provides temporary comprehension
which helps to decide how to act
[2.3] provides comprehension of the situation
120 "P4: Okay, so that cluster I thought of
as being the most interesting one."
118 "P5: Okay, so that cluster I thought of
as being the most interesting one, because of
them all being pre-paid. So I checked
whether that group of numbers appeared
over the entire period the guy was there."
100 "P1: They'd start far away on the LPU because the
car's empty. Fill it a little bit there, come closer to
home, fill up the car some more with some goodies,
before they make a hasty exit back out to where they
are going to."
3. Relation to
[3.1] indicates area where a hypothesis
should be provided
[3.2] initial interpretation of recognized
pattern in form of hypothesis
[3.3] specifies or contradicts initial hypothesis
343 "P2: But then you see something and I
cannot make a hypothesis at this moment.
I see something strange"
305 "P5: Maybe that was what triggered me,
because there are four numbers, all of them
pre-paid, making a phone call at the
important moment in time and then being
there in that area over the period that the
victim was there also, and phoning each
378 "P2: Is the driver because I see him calling. I see
him coming and then I see him going up. He’s just a
4. Reference
to deliberate
[4.1] no deliberate thinking prior
[4.2] provides initial understanding as the
beginning of deliberate analysis
[4.3] outcome of deliberate thinking
387 "P1: So for us to have 5 or 6 with a
very similar MO, on one night,
immediately gets my radar going - I think I
have a problem here. "
129 "P3: Perhaps some one has mental
problems, perhaps they keep it somewhere in
a record in the plant."
495 "P1: I sort of put myself in their shoes, or try to
think the way they would think while they were doing
this. For example, if I was going to commit these
offences and do loads of them, what would be my best
was of operating? so why have they gone down there?
Oh, that's because the car's empty!"
5. Time of
[5.1] imediately after looking at data
[5.2] right after intuition
[5.3] unexpectedly while gathering data
243 "I: When did you realise it was a
specific MO?
P1: As soon as I started looking at it"
183 "P2:I say he communicates with mother
so I say it’s very strange and then you have
something. I need to know more about the
Moroccan community. I need to know more
about the mother and I ask the detective
squad how."
349 "P2: I looked up internet because I don’t know the
community [...] yes! there is something like a healing
and I said yes! this is possible that she has been healed
over here. […] Now I say knowing on she might be a
lesbian. The spiritual healer has been called by the
mother [...] to cleanse the girl. She has been cleansed
once, twice, three times, four times"
6. Level of
[6.1] high confidence
[6.2] low confidence
[6.3] high confidence
332 "P1: I've got this problem again.
Because I had seen it previously, it was
just ringing alarm bells to me as soon as I
saw it"
341 "P4: I found there was a, the talked
about taxi that was seen coming near to the
place […] So then perhaps the victim or the
shooters had come with the taxi."
498 "P1: why have they gone down there? Oh, that's
because the car's empty!"
7. What is
based on
[7.1] experience only
[7.2] experience only
[7.3] experience + gathered data
76 "P3: Saw things, strange things, that
weren't right. and they already looked at
and they didn't saw it. But it was obvious."
128 "P3: Perhaps someone has mental
problems, perhaps they keep it somewhere in
a record in the plant. A lot of investigators
don't ask those questions. "
155 "P5: So, and then later on, erm, er, the reports
came in from the lab guys and they found two
fingerprints, and they also matched with the guys that I
had already pulled from er, from the telephone, er,
analysis. So it confirmed everything"
We perceive leap of faith to be an initial interpretation of a
pattern recognized by intuition. It indicates direction to
provide analysis and, as it is based only on one’s experience, it
is treated rather as a general guidline than as a final solution:
488 „P1: I think, I personally, this is my opinion,
they did a test run with that. That was their test
vehicle. Well, we'll try a Skoda and see what we get
and see whether or not it is the same as the others."
What connects these three unconscious processes is that we
are aware of their outcome, however we cannot say what kind
of analysis are hidden within these outcomes. We found them
to be particulary helpful when analysts deal with uncertain
data. Intuition enables analyst to recognize a familiar pattern,
which provides a starting point for analysis:
322 “P1: My wave 1 was initially, how many was it…,
I’ve got this problem again. Because I had seen it
previously, it was just ringing alarm bells to me as
soon as I was it”
Leap of faith then provides an interpretation of intuition by
giving a general sense about what results the decision maker
might expect:
40: “P1: So I’m speaking to the vehicle team saying I
think this is the MO starting up again”
As intuition and leap of faith are based on past experience,
they allow only to recall already experienced patterns:
390: “I: So, was there a system to tell you “hey
there’s this stuff happening”? P1: No. My brain”
At the moment when the decision maker takes direction
indicated by leap of faith, he or she starts deliberate analysis.
Through a process of collecting more information and
merging them with previously recognized pattern, the decision
maker gains insight. Insight arises in a form of a new pattern,
which clarifies or contradicts the previous leap of faith:
495 "P1: I sort of put myself in their shoes, or try to
think the way they would think while they were doing
this. For example, if I was going to commit these
offences and do loads of them, what would be my best
way of operating? so why have they gone down there? Oh,
that's because the car's empty!"
The role of insight is to bring the solution and comprehension
of the situation that could be justified by collected
5: “P1: Audis and BMW vehicles. Entries were being made
and satnav or the whole centre console was being taken
out of vehicle”
The presented process of inference making looks very
orderly but this is due largely to providing retrospective
analysis, where participants of the study already knew the
solution to their cases. Because of that, we believe they
provided only the information that brought them closer to a
solution rather than those that misled them.
On the basis of our findings we propose that intelligence
analysis tools should support a spectrum of analytical
reasoning and a fluid movement among different decision
making strategies. The system should support not only
analysis characterized by critical and rigorous reasoning but
also the ones based on intuition. We find important to include
in a system design functions that support analysts when
dealing with uncertain data. The system needs to allow them
freely and unofficially manipulate data so as they can test their
suppositions. We hope that further research on a bigger
sample will allow us to develop our initial findings and will
help us to design a computer system that will support and
facilitate expert and insightful thinking and reasoning.
The research reported here has received funding from the
European Union Seventh Framework Programme (FP7/2007-
2013) through Project VALCRI, EC Grant Agreement N°
FP7-IP-608142, awarded to Middlesex University and
partners, and the Erasmus+ Traineeship Abroad Programme.
We also gratefully acknowledge the support from the West
Midlands Police Force, UK, the Lokale Politie Antwerpen,
and the Service Public Federal Interieur in Belgium,.
Ancona, D. (2012). Sensemaking. Framing and acting in the unknown. In S.
Snook, N. Nohira, R. Khorana. The handbook for teaching leadership.
Sage Publishing.
Arkes, H.,R., Gigerenzer, G., Herting, H. (2016). How bad is icoherence?
American Psychological Association.
Damasio, A. (1994). Descartes’ error. Emotion, reason, and the human brain.
New York: Pengiun Books.
Dane, E., Pratt M., (2007). Exploring Intuition and its role in managerial
decision making. Academy of Management.
Evans, J. St. B. T. (2008). Dual-processing accounts of reasoning, judgment
and social cognition. Annual Review of Psychology.
Fautua, D., & Schatz, S. (2012). Cognitive Readiness and the Challenge of
Institutionalizing the “New” Versus “News”. Journal of Cognitive
Engineering and Decision Making.
Gerber, Wong, Kodagoda (submitted for EISIC 2016 conference). How
analysts think: decision making in the absence of clear facts.
Gigerenzer, G. (2000). Adaptive thinking: rationality in the real world. Oxford
University Press.
Gigerenzer, G., Brighton, H. (2009). Homo Heuristicus: Why biased minds
make better inferences. Cognitive Science Society.
Hogarth, R. M. (2001). Educating Intuition. University of Chicago Press.
Kahneman, D. (2013). Thinking fast and slow. NY: Farrar, Straus and Giroux.
Kahneman, D., Klein, G. (2009). Conditions for intuitive expertise. A failure
to disagree. American Psychologist.
Klein, G. (1997). The recognition-primed decision (RPD) model: Looking
back, looking forward. In C. E. Zsambok & G. Klein (Eds.), NDM (pp.
285292). Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc.
Klein, G. (2004). The power of Intuition: How to use your gut feelings to
make better decisions at work. New York: Currency Doubledy.
Klein, G. A. (2013). Seeing What Others Don’t: The Remarkable Ways We
Gain Insights. New York, NY, USA: PublicAffairs, Perseus Book Group.
Klein, G., Phillips, J. K., Rall, E. L., & Peluso, D. A. (2007). A Data-Frame
Theory of Sensemaking. In R. R. Hoffman (Ed.), Expertise out of context
Proceedings of the Sixth International Conference on NDM, Erlbaum.
LeDoux, (1994). Emotions, memory and the brain. Scientific American.
Merritt, S. (2011). Affective Processes in Human-Automation Interactions.
Human Factors.
Moellering, G. (2007). Trust: Reason, Routine, Reflexivity. Proceedings of
the SCARR Conference on Risk & Rationalities, Queens’ College
Rasmussen, J. (1990). Taxonomy for cognitive work analysis. Riso National
Labolatory, DK-4000 Roskilde, Denmark.
Reber, A. S. (1993). Implicit learning and tacit knowledge: An essay on the
cognitive unconscious. New York: Oxford University Press.
Sternberg, R., Davidson, J. (1996). The Nature of Insight. MIT.
Salas, E., Wilson, K., Burke, S., Wightman, D. (2006). Does Crew Resource
Management Training Work? An Update, an Extenstion, and Some
Critical Needs. Human Factors
Wong, B. L. W. (2004). Critical Decision Method Data Analysis. In D. Diaper
& N. Stanton (Eds.), Task Analysis for Human-Computer Interaction (pp.
327-346). Mahwah, NJ: Lawrence Erlbaum Associates.
Wong, B. L. W., & Kodgoda, N. (2015). How analysts think: Inference
making strategies Proceedings of the Human Factors and Ergonomics
Society 59th Annual Meeting, 26-30October 2015, Los Angeles, USA
(pp. 269-273): SAGE Publications.
... A concern, however, is that AI technologies are relatively prescriptive in nature (i.e. they are built upon a relatively linear process of entering data, values, and weights), and are often designed, developed, and fielded without considering the cognitive work of analysts (Moon & Hoffman, 2005). There is a growing body of work suggesting that intelligence analysis is not a linear or prescriptive process of critical thinking, but rather an iterative sensemaking process, where analysts use a variety of abductive, inductive, and deductive reasoning strategies to make sense of disparate information (Hoffman, et al., 2012;Moon & Hoffman, 2005;Moore, 2011;Wong, 2014;Wong & Kodagoda, 2015;Gerber, et al., 2016). This mismatch of rigid tools with more fluid cognitive processes results in tools being misused or disused by analysts (Moon & Hoffman, 2005). ...
Full-text available
Artificial Intelligence (AI) is often viewed as the means by which the intelligence community will cope with increasing amounts of data. There are challenges in adoption, however, as outputs of such systems may be difficult to trust, for a variety of factors. We conducted a naturalistic study using the Critical Incident Technique (CIT) to identify which factors were present in incidents where trust in an AI technology used in intelligence work (i.e., the collection, processing, analysis, and dissemination of intelligence) was gained or lost. We found that explainability and performance of the AI were the most prominent factors in responses; however, several other factors affected the development of trust. Further, most incidents involved two or more trust factors, demonstrating that trust is a multifaceted phenomenon. We also conducted a broader thematic analysis to identify other trends in the data. We found that trust in AI is often affected by the interaction of other people with the AI (i.e., people who develop it or use its outputs), and that involving end users in the development of the AI also affects trust. We provide an overview of key findings, practical implications for design, and possible future areas for research.
In this paper, we present a proof-of-concept system to highlight the potential benefits of mimicking higher-order cognitive processes involved in ‘insight seeking’ to create the necessary context for expert sensemaking. We draw upon data from a realistic investigation exercise undertaken by 14 experienced intelligence analysts and use this to develop our prototype to mimic behaviours demonstrated by expert analysts. Our prototype system evaluates different strategies and provides recommendations for an analyst to explore, through a prototype user interface. The recommended strategies, and associated information retrieved, aligns with the actual investigations. We propose that our system presents a novel and promising approach to design AI support systems for tasks that typically require human expert cognitive processes.
The adoption of artificial intelligence (AI) systems in environments that involve high risk and high consequence decision-making is severely hampered by critical design issues. These issues include system transparency and brittleness, where transparency relates to (i) the explainability of results and (ii) the ability of a user to inspect and verify system goals and constraints; and brittleness, (iii) the ability of a system to adapt to new user demands. Transparency is a particular concern for criminal intelligence analysis, where there are significant ethical and trust issues that arise when algorithmic and system processes are not adequately understood by a user. This prevents adoption of potentially useful technologies in policing environments. In this article, we present a novel approach to designing a conversational agent (CA) AI system for intelligence analysis that tackles these issues. We discuss the results and implications of three different studies; a Cognitive Task Analysis to understand analyst thinking when retrieving information in an investigation, Emergent Themes Analysis to understand the explanation needs of different system components, and an interactive experiment with a prototype conversational agent. Our prototype conversational agent, named Pan, demonstrates transparency provision and mitigates brittleness by evolving new CA intentions. We encode interactions with the CA with human factors principles for situation recognition and use interactive visual analytics to support analyst reasoning. Our approach enables complex AI systems, such as Pan, to be used in sensitive environments, and our research has broader application than the use case discussed.
In this paper, we demonstrate how insight seeking strategies and rules can be captured from analyst interactions with a question-answer system, as they perform an investigation. We present our analysis of an interactive investigation exercise undertaken by 14 experienced intelligence analysts. We propose that our approach to model the abstract higher order cognition involved in insight seeking provides a means to design intelligent systems that can reward and optimise potential lines of inquiry, ultimately creating the environment from which insights can be derived.
AI/ML is often considered the means by which intelligence analysts will overcome challenges of data overload under time pressure; however, AI/ML tools are often data- or algorithm-centric and opaque, and do not support the complexities of analyst sensemaking. An exploratory sensitivity analysis was conducted with a simple Authorship Attribution (AA) task to identify the degree to which an analyst can apply their sensemaking outputs as inputs to affect the performance of AI/ML tools, which can then provide higher quality information for continued sensemaking. These results show that analysts may support the performance of AI/ML primarily by refinement of potential outcomes, refinement of data and features, and refinement of algorithms themselves. A notional model of collaborative sensemaking with AI/ML was developed to show how AI/ML can support analyst sensemaking by processing large amounts of data to assist with different inference-making strategies to build and refine frames of information. Designing tools to fit this framework will increase the performance of the AI/ML, the user’s understanding of the technology and outputs, and the efficiency of the sensemaking process.
Criminal investigations involve repetitive information retrieval requests in high risk, high consequence, and time pressing situations. Artificial Intelligence (AI) systems can provide significant benefits to analysts, by sharing the burden of reasoning and speeding up information processing. However, for intelligent systems to be used in critical domains, transparency is crucial. We draw from human factors analysis and a granular computing perspective to develop Human-Centered AI (HCAI). Working closely with experts in the domain of criminal investigations we have developed an algorithmic transparency framework for designing AI systems. We demonstrate how our framework has been implemented to model the necessary information granules for contextual interpretability, at different levels of abstraction, in the design of an AI system. The system supports an analyst when they are conducting a criminal investigation, providing (i) a conversational interface to retrieve information through natural language interactions, and (ii) a recommender component for exploring, recommending, and pursuing lines of inquiry. We reflect on studies with operational intelligence analysts, to evaluate our prototype system and our approach to develop HCAI through granular computing.
One research question guided the autoethnographic inquiry: “What is the experience of intuition and immersion in the Holotropic Breathwork community?” The experience of intuition and Holotropic Breathwork are subjective. An autoethnography is proposed to allow focus on personal and evocative narratives of the author. Specifically, Jones, Adams, and Ellis method for autoethnographic data analysis was followed to bring the reader into the experience while being vulnerable. Data analysis revealed (1) appropriate set and setting, (2) mutual support, and (3) self-trust as salient themes. Future considerations to inform possible alternatives and integrative methods for coping with anxiety, depression, and enhancing quality decision making given the experience of intuition and immersions in the Holotropic Breathwork community are discussed.
Full-text available
This paper presents the preliminary results of our initial, descriptive, practical, hybrid argumentation model, designed for the use by criminal intelligence analysts (from now on referred to as analysts) working with sophisticated visual analytical software in uncertain sense-making environments. Analysts are required to create exhibits (as evidence) for a court of law or as input for decision-making in intelligence-led policing. These exhibits are required to be accurate, relevant and unbiased. Eight experienced criminal intelligence analysts from West Midlands police and the Belgium police evaluated a low-fidelity prototype resembling the first-order argumentation concepts of our initial argumentation model. The evaluation was to assess the applicability and practicality of the first-order argumentation concepts within our model. The preliminary results presented in this paper indicate that most of the first-order argumentation concepts are both applicable and practical and that the participants would use such concepts to construct their rationale from the onset of an analytical activity, if it were included as part of a software application.
Conference Paper
Full-text available
In this paper we present findings from our research into how criminal intelligence analysts make decisions in the absence of clear facts. We used the Emergent Themes Analysis method to analyze interview transcripts of seven analysts to discover how intuition, leap of faith and insight help analysts use suppositions to deal with missing or ambiguous data. We discovered that the sequence of cognitive acts – intuition, leap of faith and insight – is a reliable and helpful strategy analysts utilize to make decisions in the absence of clear facts. This finding suggests the importance for supporting decision making based on those cognitive acts in the system design.
Full-text available
In this paper we present early observations of how seven criminal intelligence analysts think and how the make inferences. We used the Critical Decision Method to identify the causal mechanisms of how they think and reason, i.e. how they organize, structure and assemble their information, understandings and inferences. We envisaged that this would enable us to design software to support the structuring of arguments and the evidential reasoning process. Our early observations suggest that analytic reasoning is not straight-forward, but appears chaotic and haphazard, and sometimes cyclic; and that inference making – abduction, induction and deduction – are not independent processes, but are closely intertwined. These processes interact dynamically, each producing outcomes that become anchors used by the others.
Full-text available
Rationality is often defined in terms of coherence, assuming that a single syntactical rule such as consistency, transitivity, or Bayes’ rule suffices to evaluate behavior. Many normative claims made in psychological research follow this assumption. We argue that coherence-based norms are of limited value for evaluating behavior as rational. Specifically, we maintain that (a) there is little evidence that coherence violations are costly, or if they are, that people would fail to learn to avoid them; (b) adaptive rules of behavior can in fact imply incoherence; (c) computational intractability and conflicting goals can make coherence unattainable; and (d) coherence plays a key role in situations where it is instrumental in achieving functional goals. These observations lead to the conclusion that coherence cannot be a universal benchmark of rationality. We argue that smart choices need to be defined in terms of ecological rationality, which requires an analysis of the environmental structure and its match with cognitive strategies.
Where do new ideas come from? What is social intelligence? Why do social scientists perform mindless statistical rituals? This vital book is about rethinking rationality as adaptive thinking: to understand how minds cope with their environments, both ecological and social. The author proposes and illustrates a bold new research program that investigates the psychology of rationality, introducing the concepts of ecological, bounded, and social rationality. His path-breaking collection takes research on thinking, social intelligence, creativity, and decision-making out of an ethereal world where the laws of logic and probability reign, and places it into our real world of human behavior and interaction. This book is accessibly written for general readers with an interest in psychology, cognitive science, economics, sociology, philosophy, artificial intelligence, and animal behavior. It also teaches a practical audience, such as physicians, AIDS counselors, and experts in criminal law, how to understand and communicate uncertainties and risks.
As the military begins to formalize training and standards for cognitive readiness, it is fitting to mark potential barriers to its implementation. This article outlines three general challenges associated with the institutionalization of cognitive readiness: (a) that the training and education community must recognize that higher-order cognitive skills development (at least for lower echelons) is fundamentally new-not merely a slight deviation from the status quo; (b) that commonly discussed cognitive competencies can (and must) be better operationalized for instruction and measurement purposes; and (c) that achieving widespread cognitive readiness will be possible only if senior leaders recognize the importance of sustained support for these competencies. The critical thesis of this article is this: Military leadership tends to view cognitive readiness as an additive aspect ("news") to what is already known and accepted, instead of as a foundational competency ("new") that requires widespread transformation. Until the institutionalization of cognitive readiness is recognized as a fundamentally novel, leap-ahead innovation, the military community will struggle to accomplish it.