Modeling the situation awareness by the
analysis of cognitive process
Shuang Liu, Xiaoru Wanyan*and Damin Zhuang
School of Aeronautics Science and Engineering, Beihang University, No. 37 Xueyuan Road, Haidian
District, Beijing 100191, China
Abstract. To predict changes of situation awareness (SA) for pilot operating with different display interfaces and tasks, a
qualitative analysis and quantitative calculation joint SA model was proposed. Based on the situational awareness model
according to the attention allocation built previously, the pilot cognitive process for the situation elements was analyzed ac-
cording to the ACT-R (Adaptive Control of Thought, Rational) theory, which explained how the SA was produced. To verify
the validity of this model, 28 subjects performed an instrument supervision task under different experiment conditions. Situa-
tion Awareness Global Assessment Technique (SAGAT), 10-dimensional Situational Awareness Rating Technique (10-D
SART), performance measure and eye movement measure were adopted for evaluating SAs under different conditions. Sta-
tistical analysis demonstrated that the changing trend of SA calculated by this model was highly correlated with the experi-
mental results. Therefore the situational awareness model can provide a reference for designing new cockpit display interfac-
es and help reducing human errors.
Keywords: Situation awareness, ACT-R, analysis model, mathematical model, ergonomics
Safe and efficient task performance within complex systems relied on operators acquiring and main-
taining appropriate levels of Situation Awareness (SA). Therefore, a critical issue was how well the
flight deck could support pilots to acquire and maintain SA of relevant information in the environment
. Endsley defined SA as the perception of the elements in the environment (level 1 SA, SA1), the
comprehension of their meaning (level 2 SA, SA2), and the projection of their status in the near future
(level 3 SA, SA3). The higher level SA depended on the lower level SA. Within this taxonomy
framework of SA, a prior study showed that 71% of aviation accidents involved human errors, and
88% of these accidents involved the SA problems . Such study suggested that pilot SA modeling
could help predicting how pilot SA would respond to different encountered situations, which could
ultimately improve flight safe and performance.
Recently, the issue of SA modeling was getting more important in the field of ergonomics and hu-
man factors studies. For qualitative analysis, Endsley proposed an information processing model,
Neisse developed a perception/action loop model, and Flach analyzed the SA model from phenomeno-
*Corresponding author: Xiaoru Wanyan, School of Aeronautics Science and Engineering, Beihang University, No. 37
Xueyuan Road, Haidian District, Beijing 100191, China. Tel.: +8601082338163; E-mail: firstname.lastname@example.org.
0959-2989/14/$27.50 © 2014 – IOS Press and the authors.
Bio-Medical Materials and Engineering 24 (2014) 2311–2318
This article is published with Open Access and distributed under the terms of the Creative Commons Attribution and Non-Commercial License.
logical standpoint . While for quantitative analysis, Wickens developed an attention-situation
awareness (A-SA) model , Entin discussed a performance sensitivity model , and Hooney im-
proved a man–machine integration design and analysis SA model . Although these studies offered a
variety of ideas and methods for investigating SA, the combined application of qualitative analysis and
quantitative calculation were inadequate. In the present study, considering the three levels of SA, a
joint qualitative and quantitative model was established based on a previous SA model  that incor-
porating the ACT-R theory for analyzing pilot cognitive process for the situation elements (SEs), and
explaining how the SA was obtained.
2. Modeling situation awareness by ACT-R theory
2.1. Qualitative analysis model of SA
The relationship between how cognition was produced by the ACT–R theory and how the pilot ob-
tained three levels of SA was analyzed, as shown in the Figure 1 .
Fig. 1. Relationship between the ACT–R and
SA.ACT-R =Adaptive Control of Thought-
Rational theory. SA=Situation Awareness;
SA1=Level 1 of SA, perception; SA2=Level 2 of
SA, understanding the present; SA3=Level 3 of
SA, understanding the future; ķDetermine what
and where to see (Event i
a) ĸObtain the visual
information of situation element ĹRetrieve the
chunk (If successfully, Event i
b) ĺMatch the IF
side ĻSelect the Rule (If the Optimal rule,
c) ļExecute the THEN side ĽPrepare
for movement ľAct movement .
Fig. 2. Qualitative analysis model of pilot SA.SE=Situation Ele-
ment; Fact = Fact of recognizing SE; ()
a= Probability of paying
attention to SE; i
=Attention allocation proportion; (/)
pb a =
probability of chunk being retrieved successfully;
pc ba =probability of Optimal rule being selected;
C=Activation level of chunk; i
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2312
The vision module was used to determine what or where the ACT-R to “see” so that certain SEs
could be registered into the short term sensory store after being filtered by the selective attention. Then
the buffers obtained the visual information from SEs through the visual module and visited the declar-
ative memory to retrieve corresponding knowledge. Only when the level of activation of the chunk
was greater than a certain threshold, could the retrieval succeed and the perception been produced
(SA1, perception), which corresponded to Wickens’ attention module in the A-SA model . In addi-
tion, the procedural memory was production rules (IF–THEN Rules).When the condition (IF) was
matched against a set of buffers, the pattern matching would select the corresponding rule to fire from
it to execute the THEN side. With regard to the recognizing status generated by the production execu-
tion, there was a fuzzy boundary between understanding the present meaning (SA 2, understanding)
and understanding the future meaning (SA3, prediction) of the SE, since the former generally had di-
rect implications for the latter and both of them were equally relevant for the task, which corresponded
to belief module in the A-SA model . The specifics of the pilot SA qualitative analysis model were
presented in Figure 2, with more descriptions in Section 2.2.
2.2. Quantitative calculation model of SA
In a certain environment, the situation related to the current operation could be broken down into
several SEs. Assuming the attention resources obtained by the n SEs were 12
(, ,..., ,... )
AA A A= , and
allocation to the SEi was i
A, which could be defined as 1
=. For a certain SE i, i
the occurring frequency, i
Sa was the salient, i
Vmeant the information priority, iii
Vu=∂ , where i
the possibility of which potential cognitive status would be available, i
u was the importance, and thus
the attention allocation proportion i
of the SEicould be structured as
When the visual module determined to “see” the SE i as event i
a, the occurrence probability of i
should be equal to the attention allocation proportion, so
If the event i
ahad occurred, the buffers would have activated corresponding chunki to the SE i in
the declarative memory, and the activation level of chunk i(0i
C) could be defined as
AC AC W S=+
C was base-level activation of the chunk
, reflecting its general usefulness in the past,
Cc t≈+ , indicating the fact that recognizing SE i(Fact i) had been presented for
ttimes, and 0c=was chosen.
W was the attention weighting of the SE
at the current
Srepresented the strength of association from the current Fact i to the relational
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2313
an was the number of facts associated to the SE
, with ln( )
SS fan=− and S was esti-
mated to be 2 [6,7].
Only if the level of activation was over a threshold, could the chunk be retrieved successfully to
possess the perception of the SE (SA1), as event i
pb a e
Here scontrolled the noise in the activation levels typically set to 0.4, and
was set to 1.0 .
The key idea in ACT-R is that at any point in time multiple production rules (IF THEN) might be
executed, only one could be selected. As for Fact i, if the optimal Rule i production with the highest
U was chosen, the SEi would be fully comprehended either in the form of its current meaning
(SA2) or the future one (SA3), which could be recorded as event i
(/ ) il
pc ba e e
According to the previous work, the cognitive level i
of SE icould be set as three values to reflect
three cognition stages . At a certain moment, if the level of activation was lower than the threshold,
it wouldn’t be perceived (short term sensory store) at 0
P= with ()(/ )
ii i i i
ab p a p b a=˄˅ . If the level
of activation was greater than the threshold, it would be perceived (SA1) at 0.5
ii i i i
pab pa pba=˄˅ . And even if the optimal rule was selected, it would be understood (SA2 or
SA3) at 1.0
P=, along with (/ )(/)()
iii i ii i i i
abc p c ba p b a p a=˄˅ .Therefore, the mathematical expectancy
of cognitive level i
Pfor the SE icould be calculated by =( ) .5+( ) 1 (1-( )) 0
i ii iii i
Ppab pabc pa××+×, and
the level of SA could be stated as
11 2 2
= ... ( +0.5 1
SA e p e p e p e e e f u
eindicated the influence of the SE i on SA and i
umeant the importance of SE i(=
3. Experimental method
3.1. Materials and participants
The experiment display interfaces were designed referring to two typical of primary flight display
(PFD) interface formats with proper simplification and abstraction for the research needs, as shown in
Figure 3. In addition, the GL studio from DiSTI was used as the tool to develop the graphical model
for PFD and generate virtual instrumentation simulation procedure for the experiment in Microsoft
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2314
Visual Studio platform. The experiment interfaces were presented on a 19-inch Lenovo Monitor with
resolution of 1280×1024, and the average illumination was about 600Lx in the experiment environ-
ment. Smart Eye Pro 4.5 was used to track eye movements in a natural way.
3.2. Design and procedure
In this experiment, an indicator monitoring and identifying task was simulated, and 4 flight SEs
were set as the monitoring targets representing the optimal targets for human attention allocation, in-
cluding the rolling angle (SE1), indicated airspeed (SE2), barometric altitude (SE3) and heading angle
(SE4) . A two-factor completely within-subjects design was adopted in which factor 1 was the ab-
normal probability with two levels set by the frequency at which the SE was questioned randomly, and
factor 2 was the display interface with two levels shown in Figure 3. Task order was counterbalanced
across the subjects according to the Latin square design.
Prior to the experiment, all participants were guided through the requirements and instructions for
the procedure. In each monitoring task, a total of 32 questions with three types representing three lev-
els of SAGAT were presented at random orders for a random time limit in a single choice format. The
participants should answer within the time limit using the mouse to get the corresponding scores. As
soon as the monitoring task finished, the 10-D SART self-rating scale was required to be accom-
plished. The eye tracker was monitoring in real-time tracking state throughout the whole task.
The attribute values of 4 SEs as monitoring targets on the two display interfaces were calculated
respectively, as shown in Table 1. To be precise, the effort values were determined by the relative
normalized distances between SEs, and the salience value for SE i was determined by color
c, indicator size i
s and type of indicator i
t, expressed as
Sa c s t=++ .
Fig. 3. Experiment display interfaces including the Display A (Left) and Display B (Right).The display interfaces were de-
signed referring to typical of primary flight display (PFD) interface formats with proper simplification and abstraction.28
participants (20 males, 8 females, and mean age 23±0.99 year) from Beihang University were recruited in this study. All
participants were right-handed with normal or correct to normal vision, and were familiar with the basic computer operations
and aeronautics knowledge.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2315
Attribute values of the SEs
Rolling Angle (SE1) Indicated Airspeed (SE2) Barometric Altitude (SE3) Heading Angle (SE4)
Display format A B A B A B A B
Indicator size 0.1559 0.3341 0.0719 0.0717 0.0719 0.0717 0.1329 0.0900
Type of indicator 0.0833 0.0833 0.1667 0.1667 0.1667 0.1667 0.1250 0.0417
Color matching 0.1304 0.1304 0.0874 0.1387 0.0957 0.1387 0.1401 0.1387
Salience 0.1232 0.1826 0.1087 0.1257 0.1114 0.1257 0.1327 0.0901
Effort 0.1264 0.1001 0.1429 0.1120 0.1429 0.1120 0.1539 0.1096
Note: SE=Situation Element; A and B were two formats of experiment display interfaces; Attribute values in the table were
normalized to be dimensionless values.
Modeling and measuring results under two display formats and two task types
Display A Display B
Task 1 Task 2 Task 1 Task 2
Prediction of SA model: 0.3364 0.3860 0.3508 0.4077
Level 1 correct rate 0.60±0.17 0.65±0.20 0.68±0.23 0.67±0.19
Level 2 correct rate 0.73±0.10 0.74±0.10 0.70±0.09 0.75±0.11
Level 3 correct rate 0.79±0.17 0.76±0.17 0.83±0.13 0.81±0.18
Level 1 &2 correct rate 0.67±0.09 0.70±0.12 0.69±0.13 0.71±0.12
Overall correct rate 0.71±0.08 0.73±0.08 0.72±0.09 0.74±0.09
Correct response time (s) 2.73±0.39 2.63±0.45 2.68±0.36 2.57±0.43
Operation score(point) 72.23±8.43 72.26±8.83 73.67±9.37 75.79±10.27
Demand (point) 11.07±2.27 10.82±1.96 11.20±1.66 10.86±1.97
Supply (point) 17.14±2.30 17.92±3.05 17.39±3.04 16.93±3.66
Understanding (point) 13.46±2.91 13.82±2.55 13.32±2.93 15.07±2.43
Overall (point) 19.54±5.05 20.93±5.49 19.50±4.90 20.79±5.50
Pupil diameter (mm) 3.60±0.46 3.53±0.60 3.59±0.55 3.96±1.80
Blink frequency (times/s) 0.32±0.21 0.34±0.21 0.32±0.21 0.36±0.23
Ratio of saccades (times/s) 0.21±0.02 0.21±0.03 0.21±0.02 0.21±0.02
Note: SA=Situation Awareness; SAGAT=Situation Awareness Global Assessment Technique, with six indices; one index
was analyzed by the measure of Performance; SART=Situational Awareness Rating Technique, with four indices; three
indices were recorded by the measure of Eye movement. The measuring results in the table were shown as Mean±SD.
SA model predictions as well as the experiment results under the factors of two display interfaces
and two tasks are presented in Table 2.
To validate the SA model, Wickens used a protocol modeling the average pilot to analyze the corre-
lation between the predictions and the measurement indices [3,8]. This protocol was used in this study
for model validation purpose.
For SAGAT, SA model prediction was correlated with the correct rate for the sum of level 1& lev-
el2 SAGAT correct rate (r=0.94), and was also higher than any other SAGAT indices, such as the
overall (r=0.93), the level 1 (r=0.61), and the level 2 (r=0.57), but no correlation with the level 3 cor-
rect rate (r=-0.08) was found. Moreover, the SA model prediction showed a strongly negative correla-
tion with correct response time (r=-0.89). For performance measures, the operation score was weakly
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2316
correlated with the prediction (r=0.65). For 10-D SART, the prediction was highly correlated with
both overall SART rating (r=0.91) and understanding rating(r=0.88), as well as negatively correlated
with the demand rating (r=-0.81), but not correlated with supply rating (r=-0.007). For psychophysio-
logical measures, eye movements were recorded. Model prediction results demonstrated a strong cor-
relation with the blink frequency (r=0.98), and weak correlation with pupil diameter(r=0.648). How-
ever, no correlation was found with the ratio of mean number of saccades (r=-0.15).
5. Discussion and conclusion
In this study, four types of approaches with series of indices were applied to verify the SA model,
and were analyzed according to the results shown in the Section 4.
Previous studies reported that the SAGAT had some limitations on measuring the SA3 , and there
was a fuzzy boundary between SA2 and SA3 in the model. Therefore, it was reasonable to see that in
this study level 3 SAGAT was not correlated with the prediction. It was clear that the two factors (dis-
play and task) both had significant influence on the correct response time (p<0.05, paired samples).
And the results were strongly correlated with the model prediction, suggesting that correct response
time might be better in measuring the SA changes under different conditions than correct rate. A pos-
sible explanation for this finding could be that increase in speed of the cognitive processing could de-
crease response time, which is in an agreement with previous findings [10,11].
For performance measures, the participants should be instructed to maximize operation scores with
the appropriate attention allocation depending on the conditions. However, it was hard to avoid the
situation that some participants might misunderstand the requirements and focused only on acquiring
higher correct rate rather than higher performance score, which might lead to performances that could
not yield a high correlation with SA model. For 10D-SART, the self estimation of SA could be com-
puted by the algorithm SA=Understanding-(Demand -Supply), where the three indices were estimated
by self rating respectively. However, no correlation between the performance and the overall SART
(r=0.29) were found, since some subjects’ misunderstanding existed during the assessments due to the
overconfidence or excessive self-esteem .
For psychophysiological measurement, very few studies used this approach to investigate SAˈ
because it not clear that psychophysiological measure can directly tap the high level cognitive pro-
cesses involved in SA Therefore it was worth examining and exploring psychophysiological indices to
reflect SA for the relationship between SA and attention .
With regard to blink frequency, it was suggested to measure the SAs under different displays or
tasks, which was obviously influenced by the two factors (p<0.05, paired samples t test) and had a
strong correlation with prediction. This result was also consistent with previous finding . As a sen-
sitive index for mental workload, pupil diameter was positive correlated with the model calculation
but not consistent with the previous result . Since the previous study didn’t include SA3 in both the
SA model and the experiment, when the experiment task was more difficult they could not improve
the cognitive level but to obtain lower SA level, even if they put more effort to monitor the SEs with
the pupil diameter increasing. However in this study, with SA3 considered, the more effort they put in
the operation with pupil diameter increasing, the higher the cognitive level they could achieve. There-
fore, further studies were required as the relationship between pupil diameter and the SA was compli-
cated and uncertain, similarly to the relationship between the mental workload and the SA . How-
ever the results of the ratio of mean number of saccades indicated that it was not sensitive for measur-
ing SAs and more researches are needed.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2317
In conclusion, the current study introduced a qualitative analysis model to explain how the three
levels of SA produced with the ACT-R theory. Based on this model, the corresponding quantitative
mathematical model was built and its validation was verified by a comprehensive experiment. The
experimental results suggested that correct response time in SAGAT performed better than the correct
rate in measuring SAs and blink frequency could assist SA measurement as well. Overall, this model
could be applied to forecast SA changes during multi-tasking on one display interface or during differ-
ent types of display interfaces in one task. Such application may also contribute to the evaluation and
optimization design of human-machine interface as well as ergonomics studies in reducing and pre-
venting human errors.
This study was supported by National Basic Research Program of China (No. 2010 CB734104) and
Research Fund for the Doctoral Program of Higher Education of China (20121102120013).
 B.L. Hooey, B.F. Gore and C.D. Wickens, Modeling pilot situation awareness, in: Human Modeling in Assisted Trans-
portation, P.C. Cacciabue, M. Hjälmdahl, A. Luedtke et al., eds., Springer Milan, 2011, pp. 207–213.
 J.Z. Yang and K. Zhang, Situation awareness: approaches, measures and applications, Advances in Psychological Sci-
ence 12 (2004), 842–850 (in Chinese).
 C.D. Wickens, J. McCarley and L. Thomas, Attention-situation awareness (A-SA) model, in: NASA Aviation Safety
Program Conference on Human Performance Modeling of Approach and Landing with Augmented Displays, D.C.
Foyle, A. Goodman and B.L. Hooey, eds., NASA, 2003, pp. 189–205.
 E.B. Entin, D. Serfaty and E. Entin, Modeling and measuring situation awareness for target-identification performance,
in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Endsley, eds., Embry-
Riddle Aeronautical University Press, 1995, pp. 233–242.
 S. Liu, X.R. Wanyan and D.M. Zhuang, A quantitative situational awareness model of pilot, Proceedings of the
International Symposium of Human Factors and Ergonomics in Healthcare 3 ( 2014), 117-122.
 M.D. Fleetwood and M.D. Byrne, Modeling the visual search of displays: a revised ACT-R model of icon search based
on eye-tracking data, Human-Computer Interaction 21 (2006), 153–197.
 J.R. Anderson, D. Bothell, M.D. Byrne et al., An integrated theory of the mind, Psychological Review 111 (2004),
 C.D. Wickens, Display formatting and situation awareness model (DFSAM): An approach to aviation display design,
(AHFD-05-14/NASA-05-5), Aviation Human Factors Division Institute of Aviation, University of Illinois at Urbana-
Champaign, Savoy, Illinois, 2005, pp. 1–18.
 P.M. Salmon, N.A. Stanton, G.H. Walker et al., Measuring situation awareness in complex systems: comparison of
measures study, International Journal of Industrial Ergonomics 3 (2009), 490–500.
 F.T. Durso, T.R. Truitt, C.A. Hackworth et al., Manning expertise and chess: A pilot study comparing situation aware-
ness methodologies, in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Ends-
ley, eds., Embry-Riddle Aeronautical University Press, 1995, pp. 295–303.
 E. M. Rantanen, S. Butler. Measurement of air traffic controllers’ situation awareness and performance during simulator
training,in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Endsley, eds.,
Embry-Riddle Aeronautical University Press, 1995, pp. 203-209.
 M.R. Endsley and D.J. Garland, Situation Awareness Analysis and Measurement, Mahwah, New Jersey, 2000, pp. 133–
 M.R. Endsley, Toward a theory of situation awareness in dynamic systems, Human Factors 37 (1995), 32–64.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2318