ArticlePDF Available

Modeling the situation awareness by the analysis of cognitive process

Authors:

Abstract

To predict changes of situation awareness (SA) for pilot operating with different display interfaces and tasks, a qualitative analysis and quantitative calculation joint SA model was proposed. Based on the situational awareness model according to the attention allocation built previously, the pilot cognitive process for the situation elements was analyzed according to the ACT-R (Adaptive Control of Thought, Rational) theory, which explained how the SA was produced. To verify the validity of this model, 28 subjects performed an instrument supervision task under different experiment conditions. Situation Awareness Global Assessment Technique (SAGAT), 10-dimensional Situational Awareness Rating Technique (10-D SART), performance measure and eye movement measure were adopted for evaluating SAs under different conditions. Statistical analysis demonstrated that the changing trend of SA calculated by this model was highly correlated with the experimental results. Therefore the situational awareness model can provide a reference for designing new cockpit display interfaces and help reducing human errors.
Modeling the situation awareness by the
analysis of cognitive process
Shuang Liu, Xiaoru Wanyan*and Damin Zhuang
School of Aeronautics Science and Engineering, Beihang University, No. 37 Xueyuan Road, Haidian
District, Beijing 100191, China
Abstract. To predict changes of situation awareness (SA) for pilot operating with different display interfaces and tasks, a
qualitative analysis and quantitative calculation joint SA model was proposed. Based on the situational awareness model
according to the attention allocation built previously, the pilot cognitive process for the situation elements was analyzed ac-
cording to the ACT-R (Adaptive Control of Thought, Rational) theory, which explained how the SA was produced. To verify
the validity of this model, 28 subjects performed an instrument supervision task under different experiment conditions. Situa-
tion Awareness Global Assessment Technique (SAGAT), 10-dimensional Situational Awareness Rating Technique (10-D
SART), performance measure and eye movement measure were adopted for evaluating SAs under different conditions. Sta-
tistical analysis demonstrated that the changing trend of SA calculated by this model was highly correlated with the experi-
mental results. Therefore the situational awareness model can provide a reference for designing new cockpit display interfac-
es and help reducing human errors.
Keywords: Situation awareness, ACT-R, analysis model, mathematical model, ergonomics
1. Introduction
Safe and efficient task performance within complex systems relied on operators acquiring and main-
taining appropriate levels of Situation Awareness (SA). Therefore, a critical issue was how well the
flight deck could support pilots to acquire and maintain SA of relevant information in the environment
[1]. Endsley defined SA as the perception of the elements in the environment (level 1 SA, SA1), the
comprehension of their meaning (level 2 SA, SA2), and the projection of their status in the near future
(level 3 SA, SA3). The higher level SA depended on the lower level SA. Within this taxonomy
framework of SA, a prior study showed that 71% of aviation accidents involved human errors, and
88% of these accidents involved the SA problems [2]. Such study suggested that pilot SA modeling
could help predicting how pilot SA would respond to different encountered situations, which could
ultimately improve flight safe and performance.
Recently, the issue of SA modeling was getting more important in the field of ergonomics and hu-
man factors studies. For qualitative analysis, Endsley proposed an information processing model,
Neisse developed a perception/action loop model, and Flach analyzed the SA model from phenomeno-
*Corresponding author: Xiaoru Wanyan, School of Aeronautics Science and Engineering, Beihang University, No. 37
Xueyuan Road, Haidian District, Beijing 100191, China. Tel.: +8601082338163; E-mail: wanyanxiaoru@buaa.edu.cn.
0959-2989/14/$27.50 © 2014 – IOS Press and the authors.
DOI 10.3233/BME-141044
IOS Press
Bio-Medical Materials and Engineering 24 (2014) 2311–2318
This article is published with Open Access and distributed under the terms of the Creative Commons Attribution and Non-Commercial License.
2311
logical standpoint [2]. While for quantitative analysis, Wickens developed an attention-situation
awareness (A-SA) model [3], Entin discussed a performance sensitivity model [4], and Hooney im-
proved a man–machine integration design and analysis SA model [1]. Although these studies offered a
variety of ideas and methods for investigating SA, the combined application of qualitative analysis and
quantitative calculation were inadequate. In the present study, considering the three levels of SA, a
joint qualitative and quantitative model was established based on a previous SA model [5] that incor-
porating the ACT-R theory for analyzing pilot cognitive process for the situation elements (SEs), and
explaining how the SA was obtained.
2. Modeling situation awareness by ACT-R theory
2.1. Qualitative analysis model of SA
The relationship between how cognition was produced by the ACT–R theory and how the pilot ob-
tained three levels of SA was analyzed, as shown in the Figure 1 [6].
Fig. 1. Relationship between the ACT–R and
SA.ACT-R =Adaptive Control of Thought-
Rational theory. SA=Situation Awareness;
SA1=Level 1 of SA, perception; SA2=Level 2 of
SA, understanding the present; SA3=Level 3 of
SA, understanding the future; ķDetermine what
and where to see (Event i
a) ĸObtain the visual
information of situation element ĹRetrieve the
chunk (If successfully, Event i
b) ĺMatch the IF
side ĻSelect the Rule (If the Optimal rule,
Event i
c) ļExecute the THEN side ĽPrepare
for movement ľAct movement .
Fig. 2. Qualitative analysis model of pilot SA.SE=Situation Ele-
ment; Fact = Fact of recognizing SE; ()
i
p
a= Probability of paying
attention to SE; i
f
=Attention allocation proportion; (/)
ii
pb a =
probability of chunk being retrieved successfully;
(/ )
iii
pc ba =probability of Optimal rule being selected;
0i
A
C=Activation level of chunk; i
P=Cognitive level.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2312
The vision module was used to determine what or where the ACT-R to “see” so that certain SEs
could be registered into the short term sensory store after being filtered by the selective attention. Then
the buffers obtained the visual information from SEs through the visual module and visited the declar-
ative memory to retrieve corresponding knowledge. Only when the level of activation of the chunk
was greater than a certain threshold, could the retrieval succeed and the perception been produced
(SA1, perception), which corresponded to Wickens’ attention module in the A-SA model [3]. In addi-
tion, the procedural memory was production rules (IF–THEN Rules).When the condition (IF) was
matched against a set of buffers, the pattern matching would select the corresponding rule to fire from
it to execute the THEN side. With regard to the recognizing status generated by the production execu-
tion, there was a fuzzy boundary between understanding the present meaning (SA 2, understanding)
and understanding the future meaning (SA3, prediction) of the SE, since the former generally had di-
rect implications for the latter and both of them were equally relevant for the task, which corresponded
to belief module in the A-SA model [3]. The specifics of the pilot SA qualitative analysis model were
presented in Figure 2, with more descriptions in Section 2.2.
2.2. Quantitative calculation model of SA
In a certain environment, the situation related to the current operation could be broken down into
several SEs. Assuming the attention resources obtained by the n SEs were 12
(, ,..., ,... )
in
A
AA A A= , and
allocation to the SEi was i
A, which could be defined as 1
iiiii
A
VSaE
β
=. For a certain SE i, i
β
indicated
the occurring frequency, i
Sa was the salient, i
Vmeant the information priority, iii
Vu=∂ , where i
was
the possibility of which potential cognitive status would be available, i
u was the importance, and thus
the attention allocation proportion i
of the SEicould be structured as
1
n
ii i
i
f
AA
=
=¦ [5].
When the visual module determined to “see” the SE i as event i
a, the occurrence probability of i
a
should be equal to the attention allocation proportion, so
()
ii
p
af= (1)
If the event i
ahad occurred, the buffers would have activated corresponding chunki to the SE i in
the declarative memory, and the activation level of chunk i(0i
A
C) could be defined as
0
n
ii jji
j
AC AC W S=+
¦ (2)
Where 0i
A
C was base-level activation of the chunk
i
, reflecting its general usefulness in the past,
usually 00.5ln
i
A
Cc t≈+ , indicating the fact that recognizing SE i(Fact i) had been presented for
ttimes, and 0c=was chosen.
j
W was the attention weighting of the SE
j
at the current
Fact i(jj
Wf=); ji
Srepresented the strength of association from the current Fact i to the relational
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2313
SE
j
;
j
f
an was the number of facts associated to the SE
j
, with ln( )
j
ij
SS fan=− and S was esti-
mated to be 2 [6,7].
Only if the level of activation was over a threshold, could the chunk be retrieved successfully to
possess the perception of the SE (SA1), as event i
b,
()/
(/)11 i
AC s
ii
pb a e
τ
−−
=+ (3)
Here scontrolled the noise in the activation levels typically set to 0.4, and
τ
was set to 1.0 [7].
The key idea in ACT-R is that at any point in time multiple production rules (IF THEN) might be
executed, only one could be selected. As for Fact i, if the optimal Rule i production with the highest
utility i
U was chosen, the SEi would be fully comprehended either in the form of its current meaning
(SA2) or the future one (SA3), which could be recorded as event i
c,
//
(/ ) il
m
UU
iii
l
pc ba e e
θθ
=¦ (4)
According to the previous work, the cognitive level i
P
of SE icould be set as three values to reflect
three cognition stages [1]. At a certain moment, if the level of activation was lower than the threshold,
it wouldn’t be perceived (short term sensory store) at 0
i
P= with ()(/ )
ii i i i
p
ab p a p b a=˄˅ . If the level
of activation was greater than the threshold, it would be perceived (SA1) at 0.5
i
P=
with ()(/)
ii i i i
pab pa pba=˄˅ . And even if the optimal rule was selected, it would be understood (SA2 or
SA3) at 1.0
i
P=, along with (/ )(/)()
iii i ii i i i
p
abc p c ba p b a p a=˄˅ .Therefore, the mathematical expectancy
of cognitive level i
Pfor the SE icould be calculated by =( ) .5+( ) 1 (1-( )) 0
i ii iii i
Ppab pabc pa××+×, and
the level of SA could be stated as
// ()/
-1 1
11 2 2
1
= ... ( +0.5 1
li i
nm
UU ACs
nn ii
il
SA e p e p e p e e e f u
θθ τ
−−
=
+++= +
¦¦
˄˅ 
(5)
Where i
eindicated the influence of the SE i on SA and i
umeant the importance of SE i(=
ii
ue
) [5].
3. Experimental method
3.1. Materials and participants
The experiment display interfaces were designed referring to two typical of primary flight display
(PFD) interface formats with proper simplification and abstraction for the research needs, as shown in
Figure 3. In addition, the GL studio from DiSTI was used as the tool to develop the graphical model
for PFD and generate virtual instrumentation simulation procedure for the experiment in Microsoft
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2314
Visual Studio platform. The experiment interfaces were presented on a 19-inch Lenovo Monitor with
resolution of 1280×1024, and the average illumination was about 600Lx in the experiment environ-
ment. Smart Eye Pro 4.5 was used to track eye movements in a natural way.
3.2. Design and procedure
In this experiment, an indicator monitoring and identifying task was simulated, and 4 flight SEs
were set as the monitoring targets representing the optimal targets for human attention allocation, in-
cluding the rolling angle (SE1), indicated airspeed (SE2), barometric altitude (SE3) and heading angle
(SE4) [5]. A two-factor completely within-subjects design was adopted in which factor 1 was the ab-
normal probability with two levels set by the frequency at which the SE was questioned randomly, and
factor 2 was the display interface with two levels shown in Figure 3. Task order was counterbalanced
across the subjects according to the Latin square design.
Prior to the experiment, all participants were guided through the requirements and instructions for
the procedure. In each monitoring task, a total of 32 questions with three types representing three lev-
els of SAGAT were presented at random orders for a random time limit in a single choice format. The
participants should answer within the time limit using the mouse to get the corresponding scores. As
soon as the monitoring task finished, the 10-D SART self-rating scale was required to be accom-
plished. The eye tracker was monitoring in real-time tracking state throughout the whole task.
4. Results
The attribute values of 4 SEs as monitoring targets on the two display interfaces were calculated
respectively, as shown in Table 1. To be precise, the effort values were determined by the relative
normalized distances between SEs, and the salience value for SE i was determined by color
matching i
c, indicator size i
s and type of indicator i
t, expressed as
()
3
iiii
Sa c s t=++ [5].
Fig. 3. Experiment display interfaces including the Display A (Left) and Display B (Right).The display interfaces were de-
signed referring to typical of primary flight display (PFD) interface formats with proper simplification and abstraction.28
participants (20 males, 8 females, and mean age 23±0.99 year) from Beihang University were recruited in this study. All
participants were right-handed with normal or correct to normal vision, and were familiar with the basic computer operations
and aeronautics knowledge.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2315
Table 1
Attribute values of the SEs
Rolling Angle (SE1) Indicated Airspeed (SE2) Barometric Altitude (SE3) Heading Angle (SE4)
Display format A B A B A B A B
Indicator size 0.1559 0.3341 0.0719 0.0717 0.0719 0.0717 0.1329 0.0900
Type of indicator 0.0833 0.0833 0.1667 0.1667 0.1667 0.1667 0.1250 0.0417
Color matching 0.1304 0.1304 0.0874 0.1387 0.0957 0.1387 0.1401 0.1387
Salience 0.1232 0.1826 0.1087 0.1257 0.1114 0.1257 0.1327 0.0901
Effort 0.1264 0.1001 0.1429 0.1120 0.1429 0.1120 0.1539 0.1096
Note: SE=Situation Element; A and B were two formats of experiment display interfaces; Attribute values in the table were
normalized to be dimensionless values.
Table 2
Modeling and measuring results under two display formats and two task types
Display A Display B
Task 1 Task 2 Task 1 Task 2
Prediction of SA model: 0.3364 0.3860 0.3508 0.4077
SAGAT:
Level 1 correct rate 0.60±0.17 0.65±0.20 0.68±0.23 0.67±0.19
Level 2 correct rate 0.73±0.10 0.74±0.10 0.70±0.09 0.75±0.11
Level 3 correct rate 0.79±0.17 0.76±0.17 0.83±0.13 0.81±0.18
Level 1 &2 correct rate 0.67±0.09 0.70±0.12 0.69±0.13 0.71±0.12
Overall correct rate 0.71±0.08 0.73±0.08 0.72±0.09 0.74±0.09
Correct response time (s) 2.73±0.39 2.63±0.45 2.68±0.36 2.57±0.43
Performance:
Operation score(point) 72.23±8.43 72.26±8.83 73.67±9.37 75.79±10.27
SART:
Demand (point) 11.07±2.27 10.82±1.96 11.20±1.66 10.86±1.97
Supply (point) 17.14±2.30 17.92±3.05 17.39±3.04 16.93±3.66
Understanding (point) 13.46±2.91 13.82±2.55 13.32±2.93 15.07±2.43
Overall (point) 19.54±5.05 20.93±5.49 19.50±4.90 20.79±5.50
Eye movement:
Pupil diameter (mm) 3.60±0.46 3.53±0.60 3.59±0.55 3.96±1.80
Blink frequency (times/s) 0.32±0.21 0.34±0.21 0.32±0.21 0.36±0.23
Ratio of saccades (times/s) 0.21±0.02 0.21±0.03 0.21±0.02 0.21±0.02
Note: SA=Situation Awareness; SAGAT=Situation Awareness Global Assessment Technique, with six indices; one index
was analyzed by the measure of Performance; SART=Situational Awareness Rating Technique, with four indices; three
indices were recorded by the measure of Eye movement. The measuring results in the table were shown as Mean±SD.
SA model predictions as well as the experiment results under the factors of two display interfaces
and two tasks are presented in Table 2.
To validate the SA model, Wickens used a protocol modeling the average pilot to analyze the corre-
lation between the predictions and the measurement indices [3,8]. This protocol was used in this study
for model validation purpose.
For SAGAT, SA model prediction was correlated with the correct rate for the sum of level 1& lev-
el2 SAGAT correct rate (r=0.94), and was also higher than any other SAGAT indices, such as the
overall (r=0.93), the level 1 (r=0.61), and the level 2 (r=0.57), but no correlation with the level 3 cor-
rect rate (r=-0.08) was found. Moreover, the SA model prediction showed a strongly negative correla-
tion with correct response time (r=-0.89). For performance measures, the operation score was weakly
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2316
correlated with the prediction (r=0.65). For 10-D SART, the prediction was highly correlated with
both overall SART rating (r=0.91) and understanding rating(r=0.88), as well as negatively correlated
with the demand rating (r=-0.81), but not correlated with supply rating (r=-0.007). For psychophysio-
logical measures, eye movements were recorded. Model prediction results demonstrated a strong cor-
relation with the blink frequency (r=0.98), and weak correlation with pupil diameter(r=0.648). How-
ever, no correlation was found with the ratio of mean number of saccades (r=-0.15).
5. Discussion and conclusion
In this study, four types of approaches with series of indices were applied to verify the SA model,
and were analyzed according to the results shown in the Section 4.
Previous studies reported that the SAGAT had some limitations on measuring the SA3 [9], and there
was a fuzzy boundary between SA2 and SA3 in the model. Therefore, it was reasonable to see that in
this study level 3 SAGAT was not correlated with the prediction. It was clear that the two factors (dis-
play and task) both had significant influence on the correct response time (p<0.05, paired samples).
And the results were strongly correlated with the model prediction, suggesting that correct response
time might be better in measuring the SA changes under different conditions than correct rate. A pos-
sible explanation for this finding could be that increase in speed of the cognitive processing could de-
crease response time, which is in an agreement with previous findings [10,11].
For performance measures, the participants should be instructed to maximize operation scores with
the appropriate attention allocation depending on the conditions. However, it was hard to avoid the
situation that some participants might misunderstand the requirements and focused only on acquiring
higher correct rate rather than higher performance score, which might lead to performances that could
not yield a high correlation with SA model. For 10D-SART, the self estimation of SA could be com-
puted by the algorithm SA=Understanding-(Demand -Supply), where the three indices were estimated
by self rating respectively. However, no correlation between the performance and the overall SART
(r=0.29) were found, since some subjects’ misunderstanding existed during the assessments due to the
overconfidence or excessive self-esteem [12].
For psychophysiological measurement, very few studies used this approach to investigate SAˈ
because it not clear that psychophysiological measure can directly tap the high level cognitive pro-
cesses involved in SA Therefore it was worth examining and exploring psychophysiological indices to
reflect SA for the relationship between SA and attention [12].
With regard to blink frequency, it was suggested to measure the SAs under different displays or
tasks, which was obviously influenced by the two factors (p<0.05, paired samples t test) and had a
strong correlation with prediction. This result was also consistent with previous finding [5]. As a sen-
sitive index for mental workload, pupil diameter was positive correlated with the model calculation
but not consistent with the previous result [5]. Since the previous study didn’t include SA3 in both the
SA model and the experiment, when the experiment task was more difficult they could not improve
the cognitive level but to obtain lower SA level, even if they put more effort to monitor the SEs with
the pupil diameter increasing. However in this study, with SA3 considered, the more effort they put in
the operation with pupil diameter increasing, the higher the cognitive level they could achieve. There-
fore, further studies were required as the relationship between pupil diameter and the SA was compli-
cated and uncertain, similarly to the relationship between the mental workload and the SA [13]. How-
ever the results of the ratio of mean number of saccades indicated that it was not sensitive for measur-
ing SAs and more researches are needed.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process 2317
In conclusion, the current study introduced a qualitative analysis model to explain how the three
levels of SA produced with the ACT-R theory. Based on this model, the corresponding quantitative
mathematical model was built and its validation was verified by a comprehensive experiment. The
experimental results suggested that correct response time in SAGAT performed better than the correct
rate in measuring SAs and blink frequency could assist SA measurement as well. Overall, this model
could be applied to forecast SA changes during multi-tasking on one display interface or during differ-
ent types of display interfaces in one task. Such application may also contribute to the evaluation and
optimization design of human-machine interface as well as ergonomics studies in reducing and pre-
venting human errors.
Acknowledgement
This study was supported by National Basic Research Program of China (No. 2010 CB734104) and
Research Fund for the Doctoral Program of Higher Education of China (20121102120013).
References
[1] B.L. Hooey, B.F. Gore and C.D. Wickens, Modeling pilot situation awareness, in: Human Modeling in Assisted Trans-
portation, P.C. Cacciabue, M. Hjälmdahl, A. Luedtke et al., eds., Springer Milan, 2011, pp. 207–213.
[2] J.Z. Yang and K. Zhang, Situation awareness: approaches, measures and applications, Advances in Psychological Sci-
ence 12 (2004), 842–850 (in Chinese).
[3] C.D. Wickens, J. McCarley and L. Thomas, Attention-situation awareness (A-SA) model, in: NASA Aviation Safety
Program Conference on Human Performance Modeling of Approach and Landing with Augmented Displays, D.C.
Foyle, A. Goodman and B.L. Hooey, eds., NASA, 2003, pp. 189–205.
[4] E.B. Entin, D. Serfaty and E. Entin, Modeling and measuring situation awareness for target-identification performance,
in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Endsley, eds., Embry-
Riddle Aeronautical University Press, 1995, pp. 233–242.
[5] S. Liu, X.R. Wanyan and D.M. Zhuang, A quantitative situational awareness model of pilot, Proceedings of the
International Symposium of Human Factors and Ergonomics in Healthcare 3 ( 2014), 117-122.
[6] M.D. Fleetwood and M.D. Byrne, Modeling the visual search of displays: a revised ACT-R model of icon search based
on eye-tracking data, Human-Computer Interaction 21 (2006), 153–197.
[7] J.R. Anderson, D. Bothell, M.D. Byrne et al., An integrated theory of the mind, Psychological Review 111 (2004),
1036–1060.
[8] C.D. Wickens, Display formatting and situation awareness model (DFSAM): An approach to aviation display design,
(AHFD-05-14/NASA-05-5), Aviation Human Factors Division Institute of Aviation, University of Illinois at Urbana-
Champaign, Savoy, Illinois, 2005, pp. 1–18.
[9] P.M. Salmon, N.A. Stanton, G.H. Walker et al., Measuring situation awareness in complex systems: comparison of
measures study, International Journal of Industrial Ergonomics 3 (2009), 490–500.
[10] F.T. Durso, T.R. Truitt, C.A. Hackworth et al., Manning expertise and chess: A pilot study comparing situation aware-
ness methodologies, in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Ends-
ley, eds., Embry-Riddle Aeronautical University Press, 1995, pp. 295–303.
[11] E. M. Rantanen, S. Butler. Measurement of air traffic controllers’ situation awareness and performance during simulator
training,in: Experimental Analysis and Measurement of Situation Awareness, D.J. Garland and M.R. Endsley, eds.,
Embry-Riddle Aeronautical University Press, 1995, pp. 203-209.
[12] M.R. Endsley and D.J. Garland, Situation Awareness Analysis and Measurement, Mahwah, New Jersey, 2000, pp. 133–
136.
[13] M.R. Endsley, Toward a theory of situation awareness in dynamic systems, Human Factors 37 (1995), 32–64.
S. Liu et al. / Modeling the situation awareness by the analysis of cognitive process2318
... It has been well studied for several decades and is a valuable construct in cognitive engineering [35], used to explain human awareness of complex dynamic situations, such as monitoring and control of manufactural systems, aircrafts or vehicles. Current research approaches in psychological and technical fields attempt to model SA using, for example, production rules (ACT-R) [68]. Due to complexity of the construct and the different facets of SA, this is challenging and may require a combination of high-performance AI modelling methods. ...
... The proposed line of action is to integrate and assist cognitive processes such as learning, reasoning and decision making in HCPSand not necessarily substitute theseand to maintain human control [7] and facilitate human-machine-teaming. To this end, it is important to understand the necessity of cognitive and knowledge engineering methods [45,68], the stand-alone value of human expert knowledge, and how human knowledge differs from "data", "information" and "AI" [72,73]. To establish a close, efficient and successful interaction between HCPS subsystems, a combination of human knowledge and data-based approaches should be followed. ...
Article
Full-text available
The development of next-generation, intelligent manufacturing relies on the realization of human-cyber-physical systems. This key area of transdisciplinary research seeks to interrogate ways of supporting and integrating human cognition and expertise in complex cyber-physical systems, using modelling methods from artificial intelligence. This paper uses the exemplar of wear classification in thermal spraying to outline how relevant cognitive processes can be elicited and combined with technical variables in a singular, efficient and cognitively plausible modelling framework. To this end, eye tracking and high-resolution voltage measurements were performed in a pilot study, and a representative data set was generated for small data problems. Two multidimensional fuzzy pattern classification models were derived. Results show that both human and technical models are significant and complement one another. While the models still require optimizing, they clearly show that the integrative approach is practicable and fruitful. Our findings call for further transdisciplinary research of the development of cognitive assistance as well as predictive maintenance and quality assurance.
... Several studies [24,25] have also confirmed moderate correlations between saccade metrics and SA. Nevertheless, no study has found a significant correlation between pupil dilation or blink rate and SA [26]. It is noted that only a few studies [27] adopted multiple eye-tracking metrics, but not all had significant correlations with SA. ...
... In the five scenes, the eye-tracking metrics for AOI-1 are most associated with the SA in scene 2, scene 3, and scene 4, while the visual behaviors for AOI-2 are confirmed in scene 3 and scene 4. e correlation analysis of the eye-tracking features in each scene identifies the fixation count, fixation duration, and saccade count as eye-tracking metrics associated with pilots' SA levels in scene 3. e fixation count, fixation duration, and saccade duration are identified in scene 4. e findings for the fixation metrics agree with reports in previous studies [27,41]. Moreover, there was no significant correlation found between the pupil dilation and SA, which is similar to previous studies [26]. However, distinct from previous reports [42], these results demonstrate the significant correlation of the saccade metrics with SA. ...
Article
Full-text available
To maintain situation awareness (SA) when exposed to emergencies during pilotage, a pilot needs to selectively allocate attentional resources to perceive critical status information about ships and environments. Although it is important to continuously monitor a pilot’s SA, its relationship with attention is still not fully understood in ship pilotage. This study performs bridge simulation experiments that include vessel departure, navigation in the fairway, encounters, poor visibility, and anchoring scenes with 13 pilots (mean = 11.3 and standard deviation = 1.4 of experience). Individuals were divided into two SA group levels based on the Situation Awareness Rating Technology (SART-2) score (mean = 20.13 and standard deviation = 5.83) after the experiments. The visual patterns using different SA groups were examined using heat maps and scan paths based on pilots’ fixations and saccade data. The preliminary visual analyses of the heat maps and scan paths indicate that the pilots’ attentional distribution is modulated by the SA level. That is, the most concerning areas of interest (AOIs) for pilots in the high and low SA groups are outside the window (AOI-2) and electronic charts (AOI-1), respectively. Subsequently, permutation simulations were utilized to identify statistical differences between the pilots’ eye-tracking metrics and SA. The results of the statistical analyses show that the fixation and saccade metrics are affected by the SA level in different AOIs across the five scenes, which confirms the findings of previous studies. In encounter scenes, the pilots’ SA level is correlated with the fixation and saccade metrics: fixation count (p = 0.034
... In a previous study, the forming process of pilot SA was analyzed in combination with ACT-R cognitive theory, and an SA model covering three levels was proposed on the basis of attention allocation. The theoretical model was validated by several experimental measurements [40,41]. However, the experimental results indicated that the SA prediction value from the theoretical model was not correlated with the operator behavior performance. ...
... An indicator-monitoring task was performed in the experiment. The rolling angle (SE.1), indicated airspeed (SE.2), barometric altitude (SE.3), and heading angle (SE.4) presented on the interface simulation model were set as the monitoring targets, referring to the optimal and effective numbers of targets for human attention allocation [40]. Each participant was asked to perform four trials with different combinations of the interface simulation model and probability distributions for indicators' abnormal display. ...
Article
Full-text available
According to a previously built situation-awareness (SA) model based on attention allocation, with the ACT-R (Adaptive Control of Thought, Rational) theory for analyzing pilot cognitive processes for situation elements, an SA mathematical model was improved to predict pilot SA during exposure to different display interfaces and missions. An indicator-display monitoring task was performed under different experimental conditions for SA model verification, while the SA global assessment technique (SAGAT), performance measures, 10-dimensional SA rating technique (10-D SART), and eye movement measures were adopted to comprehensively assess the operator’s SA. The experimental results revealed that theoretical prediction values calculated using the improved SA model were strongly correlated with the operation performance, and thus confirming the model validation. The SAGAT was shown to be a more effective method than SART in this research, and the overall SAGAT accuracy rate, as well as the accuracy response time, are effective indices for SA measurement. Eye-movement indices, such as the fixation/saccade ratio, which corresponds to the mode of information perception and extraction, was examined to be sensitive to operator’s SA changes. The Advances of the improved SA model have been achieved in predicting and indicating SA by using human behaviors, including operation performance, SAGAT response behaviors, and visual behaviors. Thereby, it provides a new auxiliary tool for quantitative characterization of pilot’s SA during cockpit interface design optimization and ergonomic evaluation.
... Based on the findings of a recent literature review (Zhang et al., 2020), several eye-tracking metrics were selected as they illustrated potential correlations with direct SA measures. In this literature review, no significant correlation was found between direct SA measures and blink rate nor pupil size, but one study did suggest their potential to assess SA in conjunction with a direct SA score (Liu et al., 2014). The specific categorization of these eye-tracking measures was adopted from Moacdieh and Sarter (2015). ...
... This higher arousal was likely brought on by the demands of the PDT task, which required continuous scanning of the visual field to respond to the appearance of the "red dot" in a timely manner. This finding partially aligns with an aviation study that evaluated cockpit displays and SA (Liu et al., 2014). Here, a quantitative SA model was proposed that included both direct SA measures and eye-tracking measures as predictor variables. ...
Article
Automated driving systems are becoming increasingly prevalent throughout society. In conditionally automated vehicles, drivers may engage in non-driving-related tasks (NDRTs), which can negatively affect their situation awareness (SA) and preparedness to resume control of the vehicle, when necessary. Previous work has investigated engagement in NDRTs, but questions remain unanswered regarding its effect on drivers’ SA during a takeover event. The objective of the current study is to use eye-tracking to aid in understanding how visual engagement in NDRTs affects changes in SA of the driving environment after a takeover request (TOR) has been issued. Thirty participants rode in a simulated SAE Level 3 automated driving environment and engaged in three separate pre-TOR tasks (Surrogate Reference Task, Monitoring Task, and Peripheral Detection Task) until presented with a TOR. Situation Awareness Global Assessment Technique (SAGAT) scores and gaze behavior were recorded during the post-TOR segment. Overall, longer times spent viewing the driving scene, and more dispersed visual attention allocation, were observed to be associated with better overall SA. Also, location-based eye tracking metrics show most promise in differentiating between task conditions with significantly different SAGAT scores. Findings from this work can inform the development of real-time SA assessment techniques using eye movements and ultimately contribute to improved operator roadway awareness for next-generation automated transportation.
... Thus, drivers' attention allocation and workload play important roles in establishing their SA. Previous works have proposed quantitative methods to model the interplay between attention allocation, workload and SA [33]- [35]than on unhighlighted ones. Specifically, the attention allocation process can be largely influenced by the salience of an object and workload [36]. ...
Preprint
Although partially autonomous driving (AD) systems are already available in production vehicles, drivers are still required to maintain a sufficient level of situational awareness (SA) during driving. Previous studies have shown that providing information about the AD's capability using user interfaces can improve the driver's SA. However, displaying too much information increases the driver's workload and can distract or overwhelm the driver. Therefore, to design an efficient user interface (UI), it is necessary to understand its effect under different circumstances. In this paper, we focus on a UI based on augmented reality (AR), which can highlight potential hazards on the road. To understand the effect of highlighting on drivers' SA for objects with different types and locations under various traffic densities, we conducted an in-person experiment with 20 participants on a driving simulator. Our study results show that the effects of highlighting on drivers' SA varied by traffic densities, object locations and object types. We believe our study can provide guidance in selecting which object to highlight for the AR-based driver-assistance interface to optimize SA for drivers driving and monitoring partially autonomous vehicles.
... ese studies include qualitative analysis of situation elements, quantitative analysis and calculation of situation awareness [16][17][18], and the analysis of the relationship between situation awareness theory and cybernetics in the literature [19]. ...
Article
Full-text available
When unmanned underwater vehicles (UUVs) perform tasks, the marine environment situation information perceived by their sensors is insufficient and cannot be shared; moreover, the reasoning efficiency of the situation information is not high. To deal with these problems, this paper proposes an ontology-based situation awareness information expression method, using the Bayesian network method to reason about situation information. First, the situation awareness information is determined in uncertain events when performing tasks in the marine environment. The core and application ontologies of UUV situation awareness are also established. Subsequently, semantic rules are constructed, and uncertain events are identified through the corresponding semantic rules. The Jess inference engine is used to update the ontology model. Because the ontology cannot reason about uncertainty, it is transformed into the Bayesian network to reason about the impacts of uncertain events on tasks. Simulation experiments verify the effectiveness and accuracy of the situation awareness reasoning method that combines the ontology and the Bayesian network.
Article
Extant studies have increasingly investigated the impact of tour guide humor on tourist responses but paid insufficient attention to its effect on tourist pro-environmental behavior. Following the conservation of resources theory, this research examined how tour guide humor influences tourist pro-environmental behavior by identifying tourist relational energy as the mediator and tourist self-construal as the moderator. The theoretical model was tested using data from 476 tourist-matched surveys nested in 28 tour groups. Tour guide humor can stimulate tourists’ perception of relational energy and further motivate tourists to engage in pro-environmental behavior. Tourists’ self-construal can moderate the effect of tour guide humor on relational energy and the indirect effect of tour guide humor on tourist pro-environmental behavior via relational energy. Specifically, the positive association between tour guide humor and relational energy is stronger for tourists with more interdependent self-construal than for those with more independent self-construal. This study contributes to the tourist pro-environmental behavior literature by identifying tour guide humor as a prominent determinant. Valuable recommendations are also provided for destination management organizations to promote tourist pro-environmental behavior from the perspective of tour guide humor.
Article
The loss of situational awareness by pilots is one of the harshest risk factors leading to catastrophic accidents. In this paper a situational awareness analysis model based on multiple agents is presented to simulate the human-computer interaction process in the aircraft cockpit. From the perspective of human-in-loop, the integrated model consists of modularized pilot agent model, technical system agent model and environment agent model. Following the structured levels of situational awareness, the pilot's cognitive behaviors are modelled based on Adaptive Control of Thought-Rational (ACT-R) theory and Bayesian network, with the triggering inputs from the visual and auditory sensory channels. Considering the effects of overlapping information from different channels, the formation and evolution mechanism of the pilot's situational awareness is analyzed in a probabilistic inference manner. The tests of input-output characteristics prove that the model can well reflect the distribution and resist the uncertain fluctuation of different cognitive elements. The safety issues based on two simplified real risk scenarios, under the background of the incidents/accidents related with Boeing 737-8 (MAX), were analyzed. The mechanism of accident evolution along with the possible preventive measures were discussed. It is concluded that an early control priority transmission from the technical system to the pilots and the display of direct prompt messages could make a difference in avoiding the serious risk.
Article
Accurate prediction and evaluation of pilot's situation awareness (SA) are of great importance for ensuring flight safety. In the present study, an SA dynamic circulation (SADC) model is developed for describing the dynamic generation and update processes of pilot's SA in‐flight environments. In the SADC model, five cognitive statuses of SA, which constitute the dynamic circulation in a closed‐loop network, are put forward on a framework of adaptive control of thought rational‐queuing network. By introducing the progressive triggers of the low and high SA, pilot's SA covering three levels is computed using conditional probability. In addition, flight simulations based on four typical flight scenarios were performed to verify the usability of the SADC model, and multiple methods including the subjective evaluation, situation present assessment method (SPAM), and physiological measurement were adopted for SA measurement. According to the results, the rationality of the SADC model is validated due to the significant correlations between SA theoretical prediction value and SPAM response time as well as 10D‐SART score. Besides, in our study, the SPAM response time and 10D‐SART score show an acceptable sensitivity to SA measurement, however, physiological indicators of respiratory rate and standard deviation of normal to normal RR intervals are proved more sensitive to workload rather than SA. The SADC model is expected to be applied in the quantitative characterization of pilot's SA as a new auxiliary analysis tool during cockpit display interface design and flight training strategy optimization.
Article
Full-text available
Introduction The Man–machine Integration Design and Analysis (MIDAS) human performance model was augmented to improve predictions of multi-operator situation awareness (SA). In MIDAS, the environment is defined by situation elements (SE) that are processed by the modeled operator via a series of sub-models including visual attention, perception, and memory. Collectively, these sub-models represent the situation assessment process and determine which SEs are attended to, and comprehended by, the modeled operator. SA is computed as a ratio of the Actual SA (the number of SEs that are detected or comprehended) to the Optimal SA (the number of SEs that are required or desired to complete the task). Method A high-fidelity application model of a two-pilot commercial crew during the approach phase of flight was generated to demonstrate and verify the SA model. Two flight deck display configurations, hypothesized to support pilot SA at differing levels, were modeled. Results The results presented include the ratio of actual to optimal SA for three high-level tasks: Aviate, Separate, and Navigate. Conclusion The model results verified that the SA model is sensitive to scenario characteristics including display configuration and pilot responsibilities.
Article
Full-text available
Because of the visual nature of computer use, researchers and designers of com-puter systems would like to gain some insight into the visual search strategies of computer users. Icons, a common component of graphical user interfaces, serve as the focus for a set of studies aimed at (1) developing a detailed understanding of how people search for an icon in a typically crowded screen of other icons that vary in similarity to the target, and (2) building a cognitively plausible model that simulates the processes inferred in the human search process. An eye-tracking study of the task showed that participants rarely refixated icons that they had pre-viously examined, and that participants used an efficient search strategy of exam-ining distractor icons nearest to their current point of gaze. These findings were integrated into an ACT-R model of the task using EMMA and a "nearest" strat-egy. The model fit the response time data of participants as well as a previous model of the task, but was a much better fit to the eye movement data. Michael Fleetwood is an applied cognitive scientist with interests in human per-formance modeling and human vision; he is a PhD candidate in the psychology department at Rice University. Michael Byrne is an applied cognitive scientist with an interest in developing computational systems for application to human factors problems; he is an assistant professor in the psychology department at Rice University.
Article
Full-text available
This paper presents a theoretical model of situation awareness based on its role in dynamic human decision making in a variety of domains. Situation awareness is presented as a predominant concern in system operation, based on a descriptive view of decision making. The relationship between situation awareness and numerous individual and environmental factors is explored. Among these factors, attention and working memory are presented as critical factors limiting operators from acquiring and interpreting information from the environment to form situation awareness, and mental models and goal-directed behavior are hypothesized as important mechanisms for overcoming these limits. The impact of design features, workload, stress, system complexity, and automation on operator situation awareness is addressed, and a taxonomy of errors in situation awareness is introduced, based on the model presented. The model is used to generate design implications for enhancing operator situation awareness and future directions for situation awareness research.
Article
The pilot’s situational awareness (SA) mainly depends on the perception, understanding and prediction of the information obtained from the man-machine display interface, therefore there is an intrinsic link between the display interface design and the level of pilot’s SA. A quantitative SA model, which considers the influence of information importance on SA, the characteristics of human cognition and the Bayesian conditional probability theory, was proposed based on the attention allocation model built previously. This model was expected to be used to predict the pilot’s SA levels under different task conditions. In order to verify the validity of the SA model, 20 volunteers were recruited to perform the instrument supervision tasks in four kinds of task, at the same time, Situation Awareness Global Assessment Technique (SAGTA), 3-dimensional Situational Awareness Rating Technique (3-D SART), and eye movement measurement were adopted for the evaluation of SAs. The experimental results reveal that, the SAGAT is the most effective approach to measure the level 2 SA, showing that the correct rate of SAGAT has the most similar changing trend with the task performance, and the SA model is validated since the changing trend of SA predicted by the model is highly correlated with the measurement indices.
Article
Assessing operator situation awareness is a key component of sociotechnical system design and evaluation. This article describes a study that was undertaken in order to compare two different situation awareness measures (a freeze probe recall approach and a post trial subjective rating approach) when used to assess participant situation awareness during a military planning task. The results indicate that only the participant situation awareness scores derived via the freeze probe recall method produced a statistically significant correlation with performance on the planning task and also that there was no significant correlation between the two methods, which suggests that they were measuring different aspects of participant situation awareness during the trials. In conclusion, the findings, whilst raising doubts over the validity of the post trial subjective rating approach, offer validation evidence for the use of freeze probe recall approaches to measure situation awareness during simulated tasks. The findings are subsequently discussed with regard to their wider implications for the future measurement of situation awareness in complex collaborative systems.Relevance to industrySituation Awareness is a critical commodity for teams working in industrial systems. Accordingly, designers and analysts require reliable and valid methods for assessing the impact of new systems, interfaces, training programs and procedures on the level of situation awareness held by operators. This article presents a review and comparison of situation awareness measurement approaches for use in complex industrial systems and provides recommendations on the types of methods to use during future situation awareness assessments.