The Similar Situations Task: An Assessment of Analogical Reasoning in Healthy and Clinical Populations

Abstract

Analogical reasoning—the ability to understand and utilize relational similarities between entities despite surface-level differences—helps individuals solve problems and navigate through novel situations. This ability varies across healthy and clinical populations, yet current analogical reasoning tasks often fail to capture subtle performance variations across different populations. To address this problem, we developed the Similar Situations Task (SST), in which participants are presented 48 line-art scene analogy problems, with source and target scenes presented separately. In each source, two sets of items (humans, animals, or objects) interact in distinct areas within the scene. One or two arrows direct participants to encode and remember specific items and their relational roles. In each target, two matching items interact analogously to one set of items in the source, while two distractor items interact in a superficially similar manner to the alignable items. Participants are tasked with determining which item, if any, is in a similar situation as one of those pointed to in the source. SST problems were found to be reliable measures of performance and presented a range of challenges for both college students and chronic-phase traumatic brain injury patients. Moreover, SST performance correlated with neuropsychological cognitive measures, but notably did not correlate with measures of verbal working memory or intelligence. The SST appears to be a sensitive, reliable, and realistic test of analogical reasoning that captures the ability to discern analogous relations and roles across different situations. Importantly, SST results suggest this ability may be independent of other cognitive capacities.

Full text

Study 2
•15 participants (12 Female; age M= 21 years, SD = 3).
•Experimental procedure differed from Study 1: in target scenes,
participants had to confirm their initial choice by clicking on a blue dot
or cloud area.
The Similar Situations Task: An Assessment of
Analogical Reasoning in Healthy and Clinical Populations
Introduction
•Traditional assessments of analogical reasoning, such as 4-term
verbal analogy problems, are often insensitive at capturing reasoning
deficits in clinical populations (e.g., traumatic brain injury patients).
•A sufficiently challenging and sensitive analogical reasoning task
could help to better characterize executive function deficits in such
populations, as well as inform cognitive rehabilitation programs.
Method
•The Similar Situations Task (SST) presented participants with 48 line-
art scene analogy problems in source–target pairs.
•In each source scene one or two arrows directed participants to
encode and remember the relations and roles of the items pointed to.
•For each target scene, participants were tasked with identifying which
item, if any, was in a similar situation as one pointed to in the source.
•If no close analog was present, participants could click “No Match.”
•Participants then rated their confidence (from -4 to 4) in their answer.
•After the SST, various other cognitive and neuropsychological
measures (different between Studies 1 and 2) were administered.
•Four trial types (see columns) were presented in pseudorandom order.
Matthew J. Kmiecik1, Guido F. Schauer1, David Martinez1& Daniel C. Krawczyk1,2
1The University of Texas at Dallas, 2University of Texas Southwestern Medical Center at Dallas
Study 1
•38 participants (25 Female; age M = 23 years, SD = 7).
•SST Calibration—how accurate a person is in their confidence rating—was
calculated by multiplying each participant’s confidence rating by the
accuracy (1 if correct or -1 if incorrect) for each trial.
References
1. Abdi, H., & Williams, L. J. (2010). Principal component analysis. Wiley Interdisciplinary Reviews: Computational Statistics, 2(4), 433-459.
2. Beaton, D., Chin Fatt, C. R., & Abdi, H. (2014). An ExPosition of multivariate analysis with the singular value decomposition in R.
Computational Statistics & Data Analysis, 72, 176-189.
3. Burgess, P. W., & Shallice, T. (1996). Response suppression, initiation and strategy use following frontal lobe lesions. Neuropsychologia,
34(4), 263-272.
0
0.2
0.4
0.6
0.8
1
Match NoMatch
Proportion Correct
SST Accuracy
0
2
4
6
8
Match NoMatch
RT (seconds)
SST Correct RT
0
0.2
0.4
0.6
0.8
1
Match NoMatch
Proportion Correct
SST Accuracy
0
2
4
6
8
Match NoMatch
RT (seconds)
SST Correct RT
Match No Match
No Match
No Match
Low
Relational Load
12 Trials
No Match
No Match
High
Relational Load
24 Trials
No Match
No Match
Low
Relational Load
4 Trials
No Match
No Match
High
Relational Load
8 Trials
-4 -3 -2 -1 0 +1 +2 +3 +4
Wrong
for
Sure
Right
for
Sure
Unsure
Wrong
or Right
For SST accuracy, we observed main effects of relational load, F(1, 37) = 4.19, p= .048, and matchability,
F(1, 37) = 29.81, p< .001, but no interaction. Similarly, for SST RT for correct trials, we observed main
effects of relational load, F(1, 37) = 18.30, p< .001, and matchability, F(1, 37) = 8.26, p = .007, but not
their interaction. Error bars represent SEM.
For SST accuracy, we observed a main effect of matchability, F(1, 14) = 15.90, p= .001, but no main
effect of relational load nor their interaction. However, for SST RT for correct trials, we observed main
effects of relational load, F(1, 14) = 12.27, p= .004, and matchability, F(1, 14) = 7.72, p = .015, but not
their interaction. Error bars represent SEM.
Remote Associates Task
SST Spring - Measures
Component 1 variance: 32.71% p = 0.0005
Component 2 variance: 12.78% p = 0.73
SST Accuracy
SST Calibration
SST Correct RT
SST Confidence Rating
Hayling Section 1 RT
Symbol Span
Hayling Section 2 RT
Hayling Semantic Errors (A)
Letter Number Span
Stroop
Digit Span
Raven’s Matrices
Component 1 Variance 33% (p< .001)
Component 2 Variance 13% (p= .73)
Hayling Semantic Errors (B)
Principal Component Analysis 1
EDRG - Measures
Component 1 variance: 33.83% p = 0.0005
Component 2 variance: 23.19% p = 0.003
Component 1 Variance 34% (p< .001)
Component 2 Variance 23% (p= .003)
Symbol Span
Verbal Analogies Correct RT
Beck Depression Inventory
Beck Anxiety Inventory
SST Correct RT Abstraction RT
Abstraction+Memory RT
Abstraction+Memory Accuracy
Verbal Analogies Accuracy
SST Accuracy
SST Calibration
SST Confidence Rating Abstraction Accuracy
Principal Component Analysis 2
Inferential principal component analyses1,2 were performed using permutation and bootstrapping techniques across 2,000 iterations to calculate significance for components and measures, respectively. Significant
measures had bootstrap ratios greater than 2 (p < .05).Dot sizes signify contributions1, or how much variance each measure contributes to the components, with greater sizes signifying larger contributions.
Acknowledgements
Many thanks go to Pranali Kamat, Brandon Pires, Niki Allahyari, and
Rudy Perez for their help with data collection and Lara Jones for
allowing us to use her verbal analogy task in Study 2.
Low High
Relational Load
Component(s) on Which the Measure Was Significant
2 1 & 2 n.s.1
Time
Discussion
•Variability of performance on the SST across healthy individuals and how SST performance relates to performance on clinically relevant
cognitive measures suggests that the SST may be sensitive to executive function deficits in clinical populations.
•Future work will investigate the efficacy of the SST in predicting difficulties in everyday reasoning within various clinical groups.