Content uploaded by Jonas Jardim de Paula
Author content
All content in this area was uploaded by Jonas Jardim de Paula
Content may be subject to copyright.
International Psychogeriatrics: page 1 of 9
C
International Psychogeriatric Association 2013
doi:10.1017/S1041610213001282
The Stick Design Test on the assessment of older adults with
low formal education: evidences of construct, criterion-related
and ecological validity
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Jonas Jardim de Paula,
1,2
Mônica Vieira Costa,
1,2
Matheus Bortolosso Bocardi,
1,2
Mariana Cortezzi,
3
Edgar Nunes De Moraes
4
and Leandro Fernandes Malloy-Diniz
1,2,5
1
Laboratory of Neuropsychological Investigations (LIN), Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
2
INCT de Medicina Molecular, Faculdade de Medicina, Universidade Federal de Minas Gerais, Belo Horizonte-MG, Brazil
3
Programa de Pós-Graduação em Neurociências, Instituto de Ciências Biológicas, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
4
Departamento de Clínica Médica, Faculdade de Medicina, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
5
Departamento de Saúde Mental, Faculdade de Medicina, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
ABSTRACT
Background: The assessment of visuospatial abilities is usually performed by drawing tasks. In patients with
very low formal education, the use of these tasks might be biased by their cultural background. The Stick
Design Test was developed for the assessment of this population. We aim to expand the test psychometric
properties by assessing its construct, criterion-related and ecological validity in older adults with low formal
education.
Method: Healthy older adults (n
= 63) and Alzheimer’s disease patients (n = 92) performed the Stick Design
Test, Mini-Mental State Examination, Digit Span Forward and the Clock Drawing Test. Their caregivers
answered Personal Care and Instrumental Activities of Daily Living). Construct validity was assessed by
factor analysis, convergent correlations (with the Clock Drawing Test), and divergent correlations (with Digit
Span Forward); criterion-related validity by receiver operating characteristic curve analysis and binary logistic
regression; and Ecological validity by correlations with ADL.
Results: The test factor structure was composed by one component (R
2
= 64%). Significant correlations with
the Clock Drawing Test and Digit Span Forward were found, and the relationship was stronger with the first
measure. The test was less associated with formal education than the Clock Drawing Test. It classified about
76% of the participants correctly and had and additive effect with the Mini-Mental State Examination (84%
of correct classification). The test also correlated significantly with measures of ADL, suggesting ecological
validity.
Conclusions: The Stick Design Test shows evidence of construct, criterion-related and ecological validity. It is
an interesting alternative to drawing tasks for the assessment of visuospatial abilities.
Key words: Alzheimer’s disease (AD), cognitive assessment, education, neuropsychological testing, psychogeriatrics
Introduction
As life expectancy continuously lengthens, the
prevalence of chronic conditions associated with
age increases. Dementia, including that caused by
Alzheimer’s disease (AD), is mostly diagnosed in
patients above 60 years of age, and its prevalence
is growing in socioeconomically depressed regions
(Prince et al., 2013). Dementia involves progressive
Correspondence should be addressed to: Jonas Jardim de Paula, Av Alfredo Balena,
190, Belo Horizonte-MG, CEP 30130-100, Brazil. Phone: + 55-31-9990-
2760; Fax: + 55-31-9990-2760. Email: jonasjardim@gmail.com. Received
19 Mar 2013; revision requested 27 May 2013; revised version received 19 Jun
2013; accepted 27 Jun 2013.
cognitive impairment, typically starting with
impairments in declarative memory and daily living.
Although no treatment can currently control these
cognitive symptoms, pharmacological (Santo et al.,
2013) and non-pharmacological (Yamaguchi et al.,
2010) interventions may improve the patient’s
cognitive, behavioral, and functional symptoms, in
addition to improving the patient’s quality of life.
In this sense, the earlier a diagnosis is made, the
sooner the patient’s treatment will commence, and
the sooner the consequent benefits will occur.
For diagnosing dementia, at least two cognitive
domains must be impaired compared to the
patient’s previous level of performance. These
2 J. J. de Paula et al.
cognitive domains include memory, reason-
ing, visuospatial abilities, language, and the
mood/personality/behavior complex (McKhann
et al., 2011). Visuospatial abilities describe a
cognitive domain involved in several dementia
subtypes, such as Parkinson’s disease dementia,
Lewy body dementia, and AD (Salmon and Bondi,
2009). In patients with AD, this impairment
usually is reflected in a patient’s performance
on visuoconstructional tests, such as drawing
and object assembly (Tippet and Black, 2008).
Acceptable performance on these paradigms
requires that specific elements of the stimuli must
be integrated and transformed along goal-oriented
motor outputs; this process demands the association
of different brain regions including, the occipital
and parietal lobes (Guérin et al., 1999; Tippet and
Black, 2008). In AD, the functional integration of
these regions is usually altered, which may underlie
the visuospatial dysfunction (Prvulovic et al., 2002;
Tippet and Black, 2008).
Cognitive impairment can be objectively assessed
by means of neuropsychological assessment –
i.e. using structured tests to make inferences
about the patient’s actual cognitive integrity.
Because the concept of cognitive decline is pivotal
to making a dementia diagnosis, educational
background and other demographic factors (e.g.
age and gender) should be considered when
comparing present and past functional levels
(Salmon and Bondi, 2009). The role of educational
background in neuropsychological assessment is
crucial with respect to differential diagnosis
issues. This is of particular relevance in those
cases of cognitive complaints among elderly
individuals with low educational levels. Because
most neuropsychological assessment tasks are
influenced by the educational background of the
patient, those tasks that are less dependent on
formal education should preferentially be used.
The Rey-Osterrieth Complex Figure Test (Rey,
1941) is among the most traditional measures
of measuring constructional praxis. However, as
highlighted in a review by Rosselli and Ardila
(2003), this task exhibits a ceiling effect among
healthy adults and older adults with 12 or more
years of education (i.e. a performance between
30 and 36 points), whereas illiterates’ scores
usually exhibit less efficient performance (i.e.
between 15 and 20 points). In this sense, test
interpretation is markedly influenced by educational
level, reducing its validity for neuropsychological
assessment, especially when we consider criterion-
related validity. Despite the predominance of
drawing tasks in the assessment of constructional
praxis, some evidence suggests that classical
neuropsychological tasks such as the Rey-Osterrieth
Complex Figure Test seem to be highly related
to formal education (Rosselli and Ardila, 2003).
Even providing specific population norms for
subjects with low educational levels requires special
consideration of the important peripheral effects
of administering such paper-and-pencil tasks;
for example, patients may experience negative
emotional reactions while performing the tasks,
which may impair their performance (Guruje et al.,
1995; Unverzagt et al. 1999). Thus, although
drawing tasks are usually applied in the assessment
of visuospatial abilities, in contexts where the
subject shows little familiarity with paper-and-
pencil skills, model assembly procedures could be
an interesting alternative for patient evaluation.
Baiyewu et al. (2005) adapted a constructional
praxis test using sticks for the assessment of
dementia. The Stick Design Test was proposed
as an alternative for drawing tests that would
be suitable for elderly patients with low formal
education. They developed and validated the test
in a sample (N
= 724) with an average age
just over 78 years, a predominance of females
and little education (i.e. less than 10% of the
participants had any formal schooling). The results
for the Stick Design Test showed significant
effects for gender (viz., females did more poorly
than males), education (viz., better scores were
elicited from participants with any education
compared to those without formal education), and
age (viz., better scores for younger participants).
The Stick Design Test correctly classified 58%
of the Dementia group and detected cognitive
dysfunction of any type for 36% of a combined
group of participants with cognitive impairment
without dementia and patients with dementia.
This task presents, therefore, a potential solution
to the assessment of elderly patients with low
educational levels. Nevertheless, Baiyewu et al.
study (2005) did not explore the relationship
between the test scores and specific measures of
functional performance. Their study focused solely
on criterion-related validity and did not explore
aspects related to construct validity – the cognitive
processes being assessed by the test – and ecological
validity – the relationship of the test with everyday
functioning.
The present study aims to evaluate the use of
the Stick Design Test in elderly Brazilian patients
with low formal education. We intend to investigate
its psychometric properties related to construct,
criterion-related and ecological validities, as well
as to assess the influence of formal education on
test performance. Based on the report of Baiyewu
et al. (2005), we hypothesize that the diagnosis
of dementia will be met with a moderate level of
accuracy.
Stick design validity for older adults’ assessment 3
Methods
Participants
In the present study, 128 elderly persons from the
city of Belo Horizonte, Brazil, were assessed in an
outpatient public health unit by a multidisciplinary
team composed of gerontologists, neuropsycho-
logists, occupational therapists, and others. The
patients were divided into two groups: healthy
controls and patients with mild AD, diagnosed
according to DSM-IV (Diagnostic and Statistical
Manual of Mental Disorders, 4th Edition) (APA,
2000), and NINDS-AIREN (McKhan et al.,
2011) criteria. Participants with significant sensorial
limitations, positive psychotic symptoms or without
a close caregiver at the interview were not invited for
participation. Diagnosis involved an unstructured
clinical interview performed by the geriatrician and
objective cognitive assessment, including the Mini-
Mental State Examination (Folstein et al., 1975),
the Mattis Dementia Rating Scale (Mattis, 1988)
and subtests from the CERAD Neuropsychological
Battery (Morris et al., 1989). All of the
neuropsychological assessment was performed by
clinical neuropsychologists specialized in elderly
populations; age and education-corrected scores
and cutoffs were used. Functional status was
assessed in the clinical interview. The Clinical
Dementia Rating was used to stage AD patients;
only participants with no dementia (CDR
= 0,
control group) or mild dementia (CDR
= 1, AD
group) were invited for participation.
Neuropsychological assessment
The Stick Design Test proposed by Baiyewu
et al. (2005) was used in this study. The
test consists of four different models to be
constructed using four wooden matches. The
models vary and are scored based on general
configuration, orientation of the whole figure
and orientation of the wooden matches’ global
aspects. The stimuli vary on different spatial
aspects: open/closed, geometrical/non-geometrical,
canonical/non-canonical position, and right/oblique
angles. The reader should refer to the Baiyewu et al.
(2005) paper for the original stimuli. Three other
neuropsychological measures were used. The Mini-
Mental State Examination (Folstein et al., 1975)
served as a general measure of cognitive functioning.
For testing the Stick Design Test’s construct
validity, we adopted another visuospatial measure,
the Clock Drawing Test (Shulman, 2000) –
scored from 0 (worst) to 5 (best) – as a measure of
convergent validity. The Digit Span Forward Test
(de Paula et al., 2010) – corrected by the product of
the total trials and the maximum span – was used
for the assessment of divergent validity due to its
being a measure of verbal working memory.
Statistical procedures
Due to the non-normal distribution of the data,
non-parametric tests were adopted. For assessing
the Stick Design Test’s construct validity, a factor
analysis procedure was performed with the test
variables, aiming to characterize the test’s latent
structure. Principal axis factoring was used for
factor extraction and, if more than one component
were found, an orthogonal rotation (viz., Varimax)
was selected. Bartlett’s test for sphericity and the
Kaiser–Meyer–Olkim test of sample adequacy were
used for assessing the viability of the procedure.
Convergent validity and divergent validity were
assessed through correlations between the neuro-
psychological measures. Because the Stick Design
Test is supposed to assess visuospatial abilities,
we computed Spearman rank-order correlations
between its total score and the Clock Drawing
Test, a classical neuropsychological measure related
to visuospatial processing. Moderate to high
correlations among these measures were expected to
suggest that a similar construct was being assessed.
The same procedure was applied between the Stick
Design Test and the Digit Span Forward Test –
a measure of the phonological loop of working
memory. Here, weak or non-significant correlations
between these measures were expected to suggest
that different constructs were being measured. The
magnitudes of the correlation coefficients were
compared through Fisher r-to-Z transformations.
The same procedures were adopted to assess the
influence of formal education on the performance
of the two visuospatial ability measures.
Criterion-related validity was assessed through
a receiver operating characteristic (ROC) curve
analysis, which evaluated the Stick Design Test’s
capacity for distinguishing healthy controls from
AD patients, while using the Clock Drawing
Test as a parameter. To assess which test
performed better for diagnostic purposes, we
compared the area under the curve of the
two measures, using the MEDCALC software
(MedCalc for Windows, version 12.4.0.0–64-
bit, MedCalc Software, Mariakerke, Belgium). A
secondary analysis tested a potential synergistic
effect of the Stick Design Test with the
Mini-Mental State Examination for the correct
classification of the participants. This procedure
was performed through logistic regression, with
one model (Model 1) containing only the Mini-
Mental State Examination and the other (Model 2)
adding the Stick Design Test. These procedures
were established because the Mini-Mental State
4 J. J. de Paula et al.
Examination is usually adopted in the screening or
diagnosis of dementia, and the addition of other
tests in the patient’s assessment should not be
redundant. Explained variance was estimated by
the Negelkerke method. Once again, the same
procedure was applied to the Clock Drawing Test
on a third model (Model 3) as a comparison
parameter.
Finally, ecological validity, conceptualized as the
relationship between neuropsychological tests and
the everyday functioning of the patient (Chaytor
and Schmitter-Edgecombe, 2003), was assessed.
This property was evaluated through Spearman
rank-order correlations between the Stick Design
Test and a Activities of Daily Living (ADLs)
inventory based on the scales proposed by Katz
et al. (1963) and Lawton and Brody (1969);
designed to assess self-care ADLs and domestic
and complex components of instrumental ADLs
(de Paula et al., in press A). We expected higher
correlations with Instrumental ADLs because
these more complex activities are usually more
strongly associated with cognitive functioning in this
population (de Paula and Malloy-Diniz, 2013). The
statistical procedures, unless stated differently, were
performed using SPSS 19.0. (IBM Corp, 2010)
Statistical significance was established at 0.05.
Results
The sociodemographic description of the parti-
cipants and neuropsychological tests performance
are shown in Table 1. No significant differences
regarding age (U
= 2729.00, p = 0.573), education
(U
= 2522.00, p = 0.177) or proportion of men
and women (
χ
2
= 2.27, p = 0.152) were found.
The groups differed on all cognitive tests (p
<
0.001), with effect sizes ranging from small to large.
Differences in functional status were significant for
Instrumental Domestic and Complex ADLs (p
<
0.001) but not for Personal Care ADLs (U =
2556.00, p = 0.061).
Considering the factor structure of the Stick
Design Test, the significance of Bartlett’s test of
sphericity (
χ
2
= 173.02, p < 0.001) and the results
of the Kaiser–Meyer–Olkim test of sample adequacy
(0.779) indicated that sufficient sample size existed
for the procedure. Only one factor was extracted
(eigenvalue: 2.60), accounting for 64% of the test
variance. The factor loadings for the four test stimuli
were 0.737, 0.823, 0.726, and 0.633. These factor
loads can be considered to be large.
Correlations between the neuropsychological
measures, age, education, and ADLs are shown
in Table 2. The Stick Design Test was moderately
correlated with the Clock Drawing Test (ϱ
= 0.544,
Table 1. Participants’ description and groups’ comparison on neuropsychological and functional measures
CONTROLS (N = 62) MILD AD (N = 93) GROUP COMPARISONS
MEDIAN ( SE) Pc. 5 Pc. 25 Pc. 75 Pc.95 MEDIAN ( SE) Pc. 5 Pc. 25 Pc. 75 Pc.95 MW-U Zr
.......................................................................................................................................................................................................................................................................................................................................................................................................................................
Age 75 (1.00) 61 68 80 87 75 (0.69) 61 70 79 84 2729.00 −0.563 –
Education 4 (0.56) 0 2 8 14 4 (0.36) 0 3 5 11 2522.00 −1.35 –
Mini-Mental State Examination 28 (0.38) 22 27 29 30 21 (0.41) 14 18 23 27 518.50
∗
−8.66 −0.77
Stick Design Test 12 (0.04) 11 12 12 12 11 (0.25) 6 9 12 12 1407.00
∗
−6.28 −0.56
Clock Drawing Test 5 (0.15) 2 4 5 5 2 (0.17) 0 1 3 5 924.50
∗
−7.28 −0.64
Digit Span Forward 35 (2.44) 20 24 54 77 24 (1.53) 12 20 35 54 1946.00
∗
−3.42 −0.30
Personal Care ADL 10 (0.03) 10 10 10 10 10 (0.09) 8 10 10 10 2556.00 −1.87 –
Instrumental-Domestic ADL 8 (0.02) 8 8 8 8 7 (0.28) 1 4 8 8 826.50
∗
−7.67 −0.68
Instrumental-Complex ADL 8 (0.05) 8 8 8 8 5 (0.30) 0 2 7 8 535.00
∗
−8.59 −0.76
∗
p < 0.001.
AD = Alzheimer’s disease; SE = standard error; Pc = percentile; ADL = activities of daily living; SE = standard error of the mean; Pc = percentile; MW U = Mann–Whitney U test; Z =
difference between standard scores, r = effect size for group comparisons.
Stick design validity for older adults’ assessment 5
Table 2. Correlations between sociodemographic, neuropsychological, and functional measures
1 2 3456789
............................................................................................................................................................................................................................................................................................................................
1 Age 1.000
2 Education − 0.036 1.000
3 Mini-Mental State Examination − 0.158 0.366
∗∗
1.000
4 Clock Drawing Test 0.187
∗
0.503
∗∗
0.654
∗∗
1.000
5 Stick Design Test 0.011 0.211
∗∗
0.587
∗∗
0.544
∗∗
1.000
6 Digit Span Forward − 0.120 0.118 0.327
∗∗
0.255
∗∗
0.245
∗∗
1.000
7Self-CareADL − 0.073 − 0.151 0.168
∗
0.059 0.105 0.071 1,000
8DomesticADL − 0.200
∗
0.051 0.585
∗∗
0.396
∗∗
0.387
∗∗
0.317
∗∗
0.240
∗∗
1.000
9 Complex ADL − 0.141 0.032 0.638
∗∗
0.479
∗∗
0.482
∗∗
0.220
∗∗
0.271
∗∗
0.820
∗∗
1.000
∗
p < 0.05,
∗∗
p < 0.01. ADL: activities of daily living.
p < 0.001), whereas it weakly correlated with
the Digit Span Forward Test (ϱ
= 0.255, p =
0.002). When these two correlation coefficients
were compared, a significant difference was found
(Z
= 2.84, p = 0.004). These results suggest the
existence of a stronger association between the
Stick Design Test another measure of visuospatial
processing (convergent validity) than with a
measure of verbal working memory (divergent
validity), thereby providing evidence of its construct
validity. When testing for the correlations between
formal education and the two visuoconstructional
praxis tests, significant associations were found for
both the Clock Drawing Test (ϱ
= 0.503, p <
0.001) and the Stick Design Test (ϱ = 0.211,
p
= 0.009). However, this factor was smaller in
magnitude in the case of the second test (Z
=
2.68, p = 0.007), suggesting that the Stick Design
Test was significantly less dependent on formal
education than was the Clock Drawing Test for this
population.
The assessment of criterion-related validity using
ROC curve analysis is shown in Figure one. The
area under the curve of the Stick Design Test
(0.756, SE
= 0.03, p < 0.001) can be considered
moderate in magnitude. For the Clock Drawing
Test, the area under the curve was also moderate
in magnitude (0.840, SE
= 0.03, p < 0.001). When
the areas under the curve of these two independent
analyses were compared, a significant difference was
found (Z
= –1.98, p = 0.048), suggesting greater
accuracy when the Clock Drawing Test was used
by itself. The ROC analysis for these two measures
is shown in the Figure 1.
With respect to the logistic regression models,
Model 1, including the Mini-Mental State
Examination, was significant (
χ
2
= 91.87, p
< 0.001) and correctly classified 82% of the
participants. The addition of the Stick Design Test
(Model 2) was significant (p
< 0.001) and improved
the overall prediction rate to 84%. The use of
the Mini-Mental State Examination and the Clock
Drawing Test (Model 3) also showed significant
results (
χ
2
= 100.91, p < 0.001) and, similar to the
previous analysis, classified 84% of the participants
correctly, indicating no differences in the accuracies
of Models 2 and 3. A description of the models is
givenonTable3.
For estimating the ecological validity of the Stick
Design Test, correlations were computed between
the test and the systematic reports of functionality
by the participants’ caregivers (Table 2). Moderate
correlations were found between the test and
aspects of domestic and complex ADL.
Discussion
The development of instruments that could be
comparable across educational levels or involving
a little-studied minority is, in fact, a challenge in
the context of cross-cultural studies of dementia.
Analyzing the influence of formal education on
neuropsychological assessment is important not
only in clinical diagnosis and prognosis but also in
the explanations of cognitive processes that underlie
specific abilities and their neurobiological correlates
(Ardila et al., 2010). These findings support the idea
that the acquisition of reading and writing skills may
radically change cognitive strategies (Rosselli and
Ardila, 2003). Moreover, educational level most
likely explains some of the inconsistencies among
cognitive screening studies (Jacqmin-Gadda et al.,
2000).
Visuospatial tasks involving drawing are
markedly difficult for older adults with little or
no formal education (Baiyewu et al., 2005; Ardila
et al., 2010; de Paula et al., 2010). An alternative
visuospatial task, such as arranging sticks in a
particular pattern, is feasible, as demonstrated in
a previous study (Baiyewu et al., 2005) and by
our present results. We assessed the influence of
formal education on test performance and found
that the Stick Design Test is less influenced by
6 J. J. de Paula et al.
Figure 1. ROC curve analysis for the diagnosis of mild Alzheimer’s disease.
Table 3. Logistic Regression Models for participant’s classification
MODEL χ
2
R
2
p CONTROLS
MILD
AD OVERALL VARIABLES BETA WALD
EXP
.
(
B ) SIG
...........................................................................................................................................................................................................................................................................................................................
1 91.87 60% <0.001 77% 86% 82% Mini-Mental State
Examination
−0.490 45.69 0.612 0.001
2 99.84 64% <0.001 79% 87% 84% Mini-Mental State
Examination
−0.432 32.67 0.650 0.001
Stick Design Test −0.831 4.47 0.436 0.035
3 100.91 65% <0.001 79% 88% 84% Mini-Mental State
Examination
−0.412 28.79 0.662 0.001
Clock Drawing Test −0.519 8.44 0.595 0.002
Model differences: 1 < 2, 1 < 3, 2 = 3.
AD = Alzheimer’s disease.
this variable than the Clock Drawing Test – a
test that, according to a Brazilian review
(Vasconcelos, 2007), is preferentially used for
the assessment of visuospatial abilities. Our
analyses show that the Stick Design Test is a valid
instrument for assessing visuospatial abilities in
older adults with low education levels, as suggested
by lines of evidence supporting its construct
validity (viz., a single-factor structure, with
convergent and divergent correlations with other
neuropsychological measures), criterion-related
validity (viz., a significant and moderate-magnitude
area under the curve, with significant additive
effects in combination with the MMSE) and
ecological validity (viz., significant correlations
with domestic and complex instrumental ADLs).
Together, these results show the adequacy of
an instrument that is less influenced by formal
education in the assessment of visuospatial
abilities.
As suggested by Dansilio and Charamelo (2005),
the loss of perspective is the most frequent error
Stick design validity for older adults’ assessment 7
committed by illiterates when copying figures,
which includes difficulties in arranging oblique
parallels and some steps in the tracing of 3D
drawings. This may explain discrepant results
when literate and illiterate subjects’ performance
in drawing tests is compared (Hong et al., 2011),
because these tasks contain these features. On
the other hand, in samples for which subjects
have relatively high educational levels, it becomes
difficult to identify impaired patients across
selective dementia level groups using traditional
visuoconstructional tasks (Cecato et al., 2012). In
this regard, it is assumed that different educational
groups need approaches specific for assessing this
construct.
Our study found significant differences between
the diagnostic powers of the Stick Design Test and
the Clock Drawing Test. When we consider the
instruments alone in the context of our sample, the
Clock Drawing Test showed better criterion-related
validity, as measured by the area under the curve in a
ROC analysis, although the differences between the
curves were discrete. This might be an artifact of
our study’s sample, which – despite predominantly
including subjects with little formal education –
did not show as high a proportion of illiterates
as did the original study (Baiyewu et al., 2005).
However, when the two instruments were used
along with the Mini-Mental State Examination, no
significant differences were found with respect to
diagnostic accuracy (i.e. both instruments correctly
classified 84% of the participants). In this sense,
when used as a screening method conjointly with the
MMSE, the Stick Design Test is as accurate as the
Clock Drawing Test in determining a participant’s
classification. Because the Stick Design Test is
usually better accepted for patients in the context
of a comprehensive evaluation, it is as effective as
the Clock Drawing Test for diagnostic purposes
and offers the advantage of eliciting less negative
emotional reaction and lowering anxiety levels
during a patient’s assessment.
Despite the usual high sensitivity and specificity
(i.e. approximately 85% each) found for the Clock
Drawing Test in first-world elderly patients, where
educational background usually exceeds eleven
years (Shulman, 2000), Lourenço et al. (2008)
found that the Clock Drawing Test, despite showing
acceptable reliability, does not appear to be valid
for detecting cognitive impairment in low-educated
older adults. Those authors argued that educational
level did not appear to interfere with its stability but
instead strongly limited its capacity to correctly clas-
sify clinical (viz., AD) and control groups. Similar
results have been reported by other Brazilian studies
using drawing-based visuospatial ability measures,
with systematically low or moderate power for the
detection of cognitive impairment and dementia
(Bertolucci et al., 2001; Porto et al., 2003).
Nevertheless, when adopted as a single measure
of cognitive functioning, the Clock Drawing Test
was slightly superior to the Stick Design Test in
the classification of AD. A previous report from
our group (de Paula et al., in press B) investigated
which cognitive abilities underlie the Clock Drawing
Test in older adults with low formal education.
That study suggested that executive functions,
semantic processing, global cognitive status and
visuospatial abilities were significant predictors of
test performance. Executive functions and global
cognitive status, especially, were the factors most
related to performance in the Clock Drawing Test.
In this sense, the Stick Design Test may be a more
specific measure of visuospatial abilities, whereas
the Clock Drawing Test might be better suited as
a general measure of cognitive status or a global
(non-specific) measure of executive functioning in
this population. This hypothesis is corroborated by
the changes in the Wald
χ
2
values of Models 2
and 3 in the logistic regression analysis. The Clock
Drawing Test version adopted for our studies uses a
pre-drawn circle, reducing its visuospatial demands.
The shared variance between the two measures
(30%) suggests that most of the specific variance
may be due to other cognitive factors.
Additional ways that our study contributes to the
research are evident in a comparison to the original
paper on the Stick Design Test (viz., Baiyewu
et al., 2005). Our study focused specifically on
the test’s psychometric properties and feasibility for
diagnosing AD patients. We found criterion-related
validity for the Stick Design Test similar to that
found by Baiyewu and colleagues, reflecting stable
properties across different populations. However,
we also analyzed both construct validity (concerning
the cognitive processes assessed by the test) and
ecological validity (i.e. the relationship between the
test and everyday functioning). As presented above,
our results reinforce the notion that the Stick Design
Test is a good measure of visuospatial processing
and offers good associations with the daily activities
of life. Future studies should investigate the
reliability of the Stick Design Test both in terms
of internal consistency and stability over time.
Some important limitations of the present study
should be addressed. Our sample size is relatively
small and may not be representative of older
Brazilian adults, so our results might not be fully
generalizable to other populations. The version of
the Clock Drawing Test adopted in this research,
although useful for diagnostic purposes (Shulman,
2000), may not be a well-suited measure of
visuospatial abilities due to the involvement of other
cognitive processes. Moreover, the Stick Design
8 J. J. de Paula et al.
Test has a strong ceiling effect (viz., the cutoff is
1 point below the total score), which may have re-
duced the test variance in our sample and influenced
the statistical procedures adopted in our study. Last,
the Stick Design Test might be an interesting tool
for the assessment of impairment in visuospatial
abilities and patients’ diagnoses when used
alongside other neuropsychological tests. However,
its standalone accuracy for diagnosis is limited.
Our results indicate that the Stick Design Test
has adequate psychometric properties and serves as
a potential tool for the assessment of visuospatial
abilities in older adults with low formal education.
The data obtained suggest that the test is sensitive
to cognitive impairment and that this sensitivity is
maintained across different cultures. Furthermore,
its usage in conjunction with other screening tests
such as the MMSE should enhance the assessment
of cognition in elderly people. Future studies
should replicate the current findings and aim to
test the measure’s applicability to the diagnosis
of other clinical conditions and the assessment of
visuospatial abilities.
Conflicts of interest
None.
Description of authors’ roles
J. J. de Paula designed the study, analyzed the data,
and wrote the paper. Costa MV, Bocardi MB, and
Cortezzi M helped with data collection, manuscript
writing, English review, and approved the final
version of the manuscript. Moraes EN supervised
the geriatric assessment of the participants and
approved the final version of the manuscript.
Malloy-Diniz LF helped to design the study,
reviewed the statistical analysis and the manuscript,
approving its final version.
Acknowledgments
Grant INCT-MM (FAPEMIG: CBB-APQ-00075-
09/CNPq 573646/2008–2) and FAPEMIG (APQ-
01972/12-10, APQ-02755-10).
References
American Psychiatric Association. (1994). Diagnostic and
Statistical Manual of Mental Disorders, 4th edn. Washington,
DC: American Psychiatric Association.
Ardila, A. et al. (2010). Illiteracy: the neuropsychology of
cognition without reading. Archives of Clinical
Neuropsychology, 25, 689–712.
Baiyewu, O. et al. (2005). The Stick Design test: a new
measure of visuoconstructional ability. Journal of the
International Neuropsychological Society, 11, 598–605.
Bertolucci, P. H. F., Okamoto, I. H., Brucki, S. M. D.,
Siviero, M. O., Neto, J. T. and Ramos, L. R. (2001).
Applicability of the CERAD neuropsychological battery to
Brazilian elderly. Arquivos de Neuro-Psiquiatria, 59,
532–536.
Cecato, J. F., Fiorese, B., Montiel, J. M., Bartholomeu,
D. and Martinelli, J. E. (2012). Clock Drawing Test in
elderly individuals with different education levels:
correlation with Clinical Dementia Rating. American
Journal of Alzheimer’s Disease & Other Dementias, 27,
620–624.
Chaytor, N. and Schmitter-Edgecombe, M. (2003). The
ecological validity of the neuropsychological tests: a review
of the literature on everyday cognitive skills.
Neuropsychology Review, 13, 181–197.
Dansilio, S. and Charamelo, A. (2005). Constructional
functions and figure copying in illiterates or low-schooled
Hispanics. Archives of Clinical Neuropsychology, 20,
1105–1112.
de Paula, J. J. and Malloy-Diniz, L. F. (2013). Executive
functions as predictors of functional performance in mild
Alzheimer’s dementia and mild cognitive impairment
elderly. Estudos de Psicologia, 18, 117–124.
de Paula, J. J. et al. (2010). Psychometric properties of a
brief neuropsychological protocol for the use in geriatric
populations. Revista de Psiquiatria Clínica, 27, 246–250.
de Paula, J. J. et al. (in press A). Development, validity, and
reliability of the IFI-LIN: a multidimensional measure of
activities of daily living for older people. Revista Brasileira de
Psiquiatria.
de Paula, J. J., Miranda, D. M., Moraes, E. N. and
Malloy-Diniz, L. F. (in press B). Mapping the
clockworks: what does the Clock Drawing Test assess in
normal and pathological aging? Arquivos de
Neuro-Psiquiatria.
Di Santo, S. G., Prinelli, F., Adorni, F., Catagirone, C.
and Musicco, M. (2013). A meta-analysis of the efficacy
of Donepezil, Rivastigmine, Galantamine, and Memantine
in relation to severity of Alzheimer’s disease. Journal of the
Alzheimer’s Disease. Epub ahead of print.
Folstein, M. F., Fosltein, S. E. and McHugh, P. R.
(1975). “Mini-Mental State”. A practical method for
grading the cognitive state of patients for the clinician.
Journal of Psychiatry Research, 12, 189–198.
Guérin, F., Ska, B. and Belleville, S. (1999). Cognitive
processing of drawing abilities. Brain and Cognition, 40,
464–478.
Guruje, O. et al. (1995). The CERAD neuropsychological
test battery: norms from a Yoruba speaking Nigerian
sample. West African Journal of Medicine
, 14, 29–33.
Hong, U. J. et al. (2011). Effects of illiteracy and education
on the visuoconstructional ability of non-demented elderly
patients. Journal of the International Neuropsychological
Society, 17, 934–939.
IBM Corp. (2010). IBM SPSS Statistics for Windows, Version
19.0. Armonk, NY. IBM Corp.
Stick design validity for older adults’ assessment 9
Jacqmin-Gadda, H., Fabrigoule, C., Commenges, D.,
Letenneur, L. and Dartigues, J. F. (2000). A cognitive
screening battery for dementia in the elderly. Journal of
Clinical Epidemiology, 53, 980–987.
Katz, S., Ford, A. B., Moskowitz, R. W., Jackson, B. A.
and Jaffe, M. W. (1963). Studies Of illness in the aged.
Journal of the American Medical Association, 185, 914–
919.
Lawton, M. P. and Brody, E. M. (1969). Assessment of
older people: self-maintaining and instrumental activities of
daily living. Gerontologist, 9, 179–186.
Lourenço, R. A., Ribeiro-Filho, S. T., Moreira, F. H.,
Paradela,E.M.P.andMiranda,A.S.(2008). The
Clock Drawing Test: performance among elderly with low
educational level. Revista Brasileira de Psiquiatria, 30,
309–315.
Mattis, S. (1988). Dementia Rating Scale. Odessa, FL:
Psychological Assessment Resources Inc.
McKhann, G. M. et al. (2011). The diagnosis of dementia
due to Alzheimer’s disease: recommendations from the
National Institute on Aging-Alzheimer’s Association
workgroups on diagnostic guidelines for Alzheimer’s
disease. Alzheimers & Dementia, 7, 263–269.
Morris, J. C. et al. (1989). The Consortium to Establish a
Registry for Alzheimer’s Disease (CERAD). Part I. Clinical
and neuropsychological assessment of Alzheimer’s disease.
Neurology, 39, 1159–1165.
Porto, C. S., Fichman, H. C., Caramelli, P., Bahia,
V. S. and Nitrini, R. (2003). Brazilian version of the
Mattis Dementia Rating Scale. Arquivos de Neuro-
Psiquiatria, 61, 339–345.
Prince, M., Bryce, R., Albanese, E., Wimo, A., Ribeiro,
W.andFerri,C.P.(2013). The global prevalence of
dementia: a systematic review and metaanalysis. Alzheimer’s
&Dementia, 9, 63–75.
Prvulovic, D. et al. (2002). Functional imaging of
visuospatial processing in Alzheimer’s disease. Neuroimage,
17, 1403–1414.
Rey, A. (1941). L’exam en psychologique dans les cases d’
encephalopathy traumatique. Archives de Psychologie, 28,
286–340.
Rosselli, M. and Ardila, A. (2003). The impact of culture
and education on non-verbal neuropsychological
measurements: a critical review. Brain and Cognition, 52,
326–333.
Salmon, D. P. and Bondi, M. W. (2009).
Neuropsychological assessment of dementia. Annual
Reviews of Psychology, 60, 257–282.
Shulman, K. I. (2000). Clock-drawing: is it the ideal
cognitive screening test? International Journal of Geriatric
Psychiatry, 15, 548–561.
Tippett, W. J. and Black, S. E. (2008). Regional cerebral
blood flow correlates of visuospatial tasks in Alzheimer’s
disease. Journal of the International Neuropsychological
Society, 14, 1034–1045.
Unverzagt, F. W.
et al. (1999). Clinical utility of CERAD
neuropsychological battery in elderly Jamaicans. Journal of
the International Neuropsychological Society, 5, 255–259.
Vasconcelos,L.G.,Brucki,S.M.D.andBueno,
O. F. A. (2007). Cognitive and functional dementia
Assessment tools: review of Brazilian literature. Dementia &
Neuropsychologia, 1, 18–23.
Yamaguchi, H., Maki, Y. and Yamagami, T. (2010).
Overview of non-pharmacological intervention for
dementia and principles of brain-activating rehabilitation.
Psychogeriatrics, 10, 206–213.
