ArticlePDF Available

Kettle Test--A Brief Measure of Cognitive Functional Performance: Reliability and Validity in Stroke Rehabilitation

Authors:

Abstract and Figures

We examined the reliability and validity of the Kettle Test, a brief performance measure based on a complex everyday task designed to tap into basic and higher level cognitive processes. Participants included 21 people attending stroke rehabilitation and 4 occupational therapists for the reliability analysis, 36 people at discharge from stroke rehabilitation, and 36 age-matched healthy control participants for the validity analyses. Instruments included a battery of conventional cognitive measures and functional outcomes. Interrater reliability was found to be high. Stroke survivors at discharge from rehabilitation were found to require significantly more assistance on the Kettle Test than control participants (p < .000); their scores on the Kettle Test were significantly and moderately correlated with the conventional cognitive and functional outcome measures. The results support the reliability and validity of the Kettle Test as a top-down measure of cognition-in-function in people at discharge from stroke rehabilitation.
Content may be subject to copyright.
592 September/October 2009, Volume 63, Number 5
Kettle Test—A Brief Measure of Cognitive Functional
Performance: Reliability and Validity in Stroke Rehabilitation
KEY WORDS
• cognition
• reproducibility of results
• stroke
• task performance and analysis
Adina Hartman-Maeir, PhD, OT, is Lecturer,
School of Occupational Therapy, Hadassah and Hebrew
University, Mount Scopus, POB 24026, Jerusalem 91240
Israel; amaeir@mscc.huji.ac.il
Hagit Harel, MSc, OT, is Clinician, Rehabilitation
Department, Sheba Medical Center, Tel Aviv, Israel.
Noomi Katz, PhD, OTR, is Director, Research Institute
for Health and Medical Professions, Ono Academic
College, Israel, and Professor Emeritus, School of
Occupational Therapy, Hadassah and Hebrew University,
Jerusalem, Israel.
OBJECTIVES. We examined the reliability and validity of the Kettle Test, a brief performance measure based
on a complex everyday task designed to tap into basic and higher level cognitive processes.
METHOD. Participants included 21 people attending stroke rehabilitation and 4 occupational therapists for
the reliability analysis, 36 people at discharge from stroke rehabilitation, and 36 age-matched healthy control
participants for the validity analyses. Instruments included a battery of conventional cognitive measures and
functional outcomes.
RESULTS. Interrater reliability was found to be high. Stroke survivors at discharge from rehabilitation were
found to require significantly more assistance on the Kettle Test than control participants (p < .000); their scores
on the Kettle Test were significantly and moderately correlated with the conventional cognitive and functional
outcome measures.
CONCLUSIONS. The results support the reliability and validity of the Kettle Test as a top-down measure of
cognition-in-function in people at discharge from stroke rehabilitation.
Hartman-Maeir, A., Harel, H., & Katz, N. (2009). Kettle Test—A brief measure of cognitive functional performance: Reliability
and validity in stroke rehabilitation. American Journal of Occupational Therapy, 64, 592–599.
Adina Hartman-Maeir, Hagit Harel, Noomi Katz
People with acquired brain injury are at high risk for cognitive impairments that
have a detrimental impact on occupational performance (Cicerone et al., 2005;
Donnovan et al., 2008; Hershkovitz & Brill, 2007; Zinn et al., 2004). Occupational
therapy has developed a unique body of knowledge regarding the relationship
between cognition and occupation involving occupation-based, client-centered
intervention (Gillen, 2009; Katz, 2005). Conventional “tabletop” measures of
cognition are valuable diagnostically but have limited ecological validity and do
not fully address the functional implications of cognitive deficits. Moreover, the
interdisciplinary rehabilitation community has emphasized the need for ecologi-
cally valid measures of cognition in function (Burgess et al., 2006). This current
emphasis is an important milestone in the conceptualization of measurement in
health-related professions. Instruments that follow these directives are essential and
are integral to the occupational therapy philosophy and practice (American
Occupational Therapy Association, 2008). Occupational therapy is in a unique
position to fulfill a central role in this development and is establishing its expertise
in the development of top-down assessments (Hartman-Maeir, Katz, & Baum,
2009; Law, Baum, & Dunn, 2005).
Evaluation of outcomes in stroke rehabilitation, beyond basic activities of daily
living (BADLs), has become extremely important as more people survive stroke and
return to their homes in the community. One primary concern at discharge from
rehabilitation is predicting the ability to function independently in the community
or the degree of assistance required to do so. Instrumental activities of daily living
The American Journal of Occupational Therapy 593
(IADLs) are more complex than BADLs and thus require
basic and higher-level cognitive functions such as executive
functioning (Cahn-Weiner et al., 2007; Coster, Haley, Jette,
Tao, & Seibens, 2007; MacNeill & Lichtenberg, 1997).
Thus, to address this important outcome during stroke reha-
bilitation, occupational therapists have a need for top-down,
performance-based assessments that will incorporate the cog-
nitive complexity involved in IADLs. A small number of
performance-based measures in occupational therapy fulfill
these requirements and incorporate cognitive challenges in
functional IADL contexts; these measures include the
Cognitive Performance Test (CPT; Burns, 2006), the
Assessment of Motor and Process Skill (AMPS; Fisher,
2006a, 2006b), and the Executive Function Performance
Test (EFPT; Baum, Morrison, Hahn, & Edwards, 2003;
Baum et al., 2008). These instruments are valuable top-
down measures that address cognitive and process skills in
IADLs. The CPT and EFPT each have four or five IADL
tasks that provide a wide range of activities in this domain.
The scores on the CPT measure the six cognitive levels
according to the Cognitive Disabilities Model (Allen,
Earhart, & Blue, 1992; Levy & Burns, 2005), and the scores
of the EFPT measure executive functioning components.
The AMPS is an extensively researched, reliable, and valid
measure of motor and process skills in IADLs; however, it
requires intensive and expensive training, which is not avail-
able for all occupational therapists.
The focus of this study is on the development of the
Kettle Test (Hartman-Maeir, Armon, & Katz, 2005), a brief
performance measure that is based on a complex everyday
task and is designed to tap into basic and higher-level cogni-
tive processes. The aim of the test is to evaluate the ability
for independent community living of people with identified
or suspected cognitive disabilities.
Development of the Kettle Test
The Kettle Test (Hartman-Maeir et al., 2005) was developed
to provide a brief performance-based assessment of an IADL
task that can be conducted in a clinical setting or at home.
The test was designed to tap into a broad range of cognitive
skills within a functional context to assist in the clinical deci-
sion-making process regarding the need for assistance in daily
living skills of adults with suspected cognitive disabilities.
The basic task of preparing a hot beverage was chosen
because of its functional significance, broad cultural rele-
vance, and feasibility (in terms of tools and time require-
ments) in multiple settings (e.g., clinic, home). The task
complexity was elevated to enable the assessment of basic
and higher-level cognitive–functional skills that are required
for IADLs, with the following elaborations:
1. The essential task of preparing oneself a hot beverage
was expanded to include preparing an additional cup of
hot beverage for the therapist; the additional cup differs
in two ingredients from that requested by the client,
providing additional load on working memory.
2. The electric kettle is emptied and disassembled (lid and
electric cable are disconnected from the body of the
kettle) to challenge problem-solving skills and safety
judgment regarding the use of electricity.
3. Additional kitchen utensils and ingredients are placed
in the immediate task environment (on a tray) as dis-
tracters to increase attention demands.
This design creates a brief task using familiar objects and
settings that are presented in a manner that targets cognitive
skills underlying complex ADLs. The observation is struc-
tured so that the rater is required to score the performance
on 13 discrete steps of the task (e.g., turning on the faucet,
filling the kettle with 2 cups of water). Clear guidelines for
cueing are provided, and the rater scores each step according
to the degree of cueing that was necessary to complete the
step (0 through 4). Total scores range from 0 to 52 (higher
scores indicate more assistance) and can be transformed to
clinically meaningful categories of independence on the task
(independent, mild assistance, considerable assistance).
Initial research to examine convergent and ecological
validity was conducted on the Kettle Test in a sample of 41
elderly clients (mean age = 75.2) referred to a geriatric
assessment clinic because of suspected cognitive disabilities
(Hartman-Maeir, Katz, & Armon, 2004). Small to moder-
ate significant correlations were found between Kettle Test
scores with conventional measures of cognition (r = .56 with
Mini-Mental Status Evaluation [MMSE; Folstein &
Folstein, 1975]; r = .59 with Clock Drawing Test [CDT;
Freedman et al., 1994]; and r = .32 with the visual attention
Star Cancellation subtest of the Behavioral Inattention Test
[BIT; Wilson, Cockburn, & Halligan, 1987), and moderate
correlations were found with caregiver ratings of ADLs (r =
.53) and IADLs (r = .58). This initial study provided sup-
port for the validity of the instrument. Therefore, the pur-
pose of this study was to further examine the reliability and
validity of the Kettle Test in a geriatric stroke population
faced with similar concerns regarding the impact of cogni-
tive deficits on daily living abilities at discharge from
rehabilitation.
The study was divided into two stages: (1) The first
consisted of the study of interrater reliability of the Kettle
Test between two sets of raters in two rehabilitation set-
tings, and (2) the second consisted of the study of different
aspects of validity in a sample of people at discharge from
stroke rehabilitation and healthy elderly control partici-
pants. The objective of the first stage was to examine the
594 September/October 2009, Volume 63, Number 5
interrater reliability of the Kettle Test scores as administered
by certified occupational therapists to stroke patients in
rehabilitation. The objectives of the second stage were to
examine
The construct validity of the Kettle Test, in terms of its
ability to differentiate between scores of people after
stroke at discharge from rehabilitation and scores of age-
matched healthy control participants who live indepen-
dently in the community;
The convergent validity of the Kettle Test, calculating
relationships between Kettle Test scores with conven-
tional measures of cognition; and
The ecological validity of the Kettle Test, examining the
relationship of safety level and BADL status at discharge
and IADL status at home, 1 month after discharge
Method
Stage 1: Interrater Reliability Study
Participants were recruited from two centrally located geri-
atric rehabilitation hospitals in Jerusalem and Tel Aviv.
Inclusion criteria required participants to be admitted to a
rehabilitation hospital within 1 month after stroke, be ages
60 or older, be alert and without receptive aphasia, and live
independently in the community before stroke. Twenty-one
patients consecutively admitted to each rehabilitation hos-
pital who met inclusion criteria participated in the reliability
study (Hospital 1: n = 10; 4 men, 6 women; mean age =
79.3, standard deviation [SD] = 5.8. Hospital 2: n = 11; 6
men, 5 women; mean age = 77.82, SD = 5.1). The study
was approved by the Human Rights Helsinki Committee
of each hospital, and all participants signed informed con-
sent. Four certified and experienced occupational therapists
(two in each hospital) rated patient performance on the
Kettle Test. The test was administered by one of the raters
while the other observed the assessment. Independent rat-
ings were then conducted immediately after the assessment.
Interrater reliability (Spearman correlation coefficient) was
found to be high for the Kettle Test total scores for both
sets of raters (Hospital 1: r =.851, p = .001; Hospital 2: r =
.916, p = .000).
Stage 2: Validity Study
Participants. Thirty-six stroke patients (18 with right
hemisphere stroke and 18 with left hemisphere stroke) were
recruited from a geriatric rehabilitation hospital before their
discharge (mean length of stay = 63.1 days, SD = 29.2).
Inclusion criteria were the same as delineated in the reliabil-
ity study. In addition, 36 healthy control participants were
recruited from a convenience sample of healthy elderly vol-
unteers. The inclusion criteria for the control participants
were age 60 or older, living independently in the commu-
nity, and scoring >23 on the MMSE; see Table 1 for demo-
graphic data). The study was approved by the Human Rights
Helsinki Committee of the hospital, and all participants
signed informed consent.
Instruments. A short battery of conventional standard-
ized measures of cognition recommended in geriatric stroke
rehabilitation (Adunsky, Fleissig, Levenkrohn, Arad, & Noy,
2002) was used in this study. The battery included the fol-
lowing measures:
The MMSE is a universal measure of cognitive status that
assesses a broad range of basic cognitive abilities including
orientation, attention, recall, working memory, spatial
abilities, and language. Scores range from 0 to 30, and
scores >23 are considered the cutoff for cognitive
impairment.
The CDT is widely used in cognitive screening for many
neurological conditions (Freedman et al., 1994). In the
stroke population, several scoring systems of a free-style
drawing of a clock with hands at 10 past 11 have been
shown to have construct validity and correlate with mul-
tiple cognitive domains, such as abstract thinking, execu-
tive functioning, and visuospatial construction (Suhr,
Grace, Allen, Nadler, & McKenna, 1998). In this study,
we used the scoring system of Rouleau, Salmon, Butters,
Kennedy, and McGuire (1992), with a score range from
0 to 10, which was found to be reliable and valid in stroke
rehabilitation (Suhr et al., 1998).
The Star Cancellation subtest of the BIT was used to
measure visual attention, as recommended for use in
stroke rehabilitation practice (Edwards et al., 2006).
The Cognitive scale of the FIM™ (CognFIM; Granger,
1998) was used as a measure of cognitive–functional sta-
tus as expressed in daily activities (based on general obser-
vation) in five areas of cognition (expressive and receptive
language, problem solving, social interaction, and mem-
ory). Each area is rated on a 7-point scale; total score
ranges from 5 to 35. Reliability and validity have been
extensively established in the stroke population (Heruti,
Lusky, & Dankner, 2002).
The functional outcome battery included the areas of
BADLs, IADLs, and safety—outcomes necessary for
independent community living.
BADLs were measured with the Motor scale of the FIM
(Granger, 1998) because it is a universal, reliable, and
valid measure of rehabilitation outcome. The FIM Motor
scale includes 13 items of basic self-care rated on a 7-point
scale (total scale score range = 13–91).
IADLs were measured with the IADL scale (Lawton &
Brody, 1969; Lawton, Moss, Fulcomer, & Kleban, 1982),
The American Journal of Occupational Therapy 595
a widely used measure for independence in extended areas
of ADLs, including telephone use, shopping, food prepara-
tion, housekeeping, laundry, transportation, taking medica-
tion, and financial management (total score range = 0–23).
Safety was measured with the Safety Rating scale, which
is part of the Routine Task Inventory (RTI–E), a struc-
tured observation used to rate cognitive levels in daily
routine activities (Allen et al., 1992; Katz, 2006). The 4-
point scale ranges from 3 (unable to recognize the need for
safety precautions) to 6 (anticipates hazards and plans safety
procedures). The reliability and validity of the RTI–E has
been studied in populations with cognitive disabilities
(Katz, 2006).
The Fugl-Meyer Motor Assessment (FMA; Fugl-Meyer,
Jääsko, Leyman, Olsson, & Steglind, 1975; Rabadi &
Rabadi, 2006) Upper-Extremity scale was used to mea-
sure motor status to evaluate the possible confounding
relationship between motor function after stroke with
performance on the Kettle Test.
Procedure
The assessment battery was administered within the last
week before discharge from the rehabilitation hospital. The
Kettle Test was administered by Hagit Harel. The other
assessments were administered by experienced occupational
therapists and rehabilitation professionals as part of the rou-
tine discharge assessment battery of the rehabilitation depart-
ment. In addition, to examine IADL status 1 month after
discharge, study group follow up included a telephone inter-
view administered to caregivers. The follow-up assessment
was completed on 29 participants, because 7 participants
could not be reached. The Kettle Test scores of the 7 partici-
pants that were lost to follow-up were not significantly
different from those of the other participants (t[34] = .892,
p = .403)
Data Analysis
Data were analyzed with SPSS (Version 15.0; SPSS, Inc.,
Chicago). Descriptive statistics and analysis of covariance
(ANCOVA) were used to examine the group effect on the
Kettle Test while controlling for years of education, which
were found to differ significantly between groups. Pearson
correlation analyses were used to examine the relationships
among the cognitive and functional measures and motor and
demographic variables. No significant differences were found
between participants with right- and left-hemisphere stroke
on the Kettle Test (t[34] = 0.178, p = .860); therefore, the
analyses were conducted on the entire stroke group.
Results
Before studying validity, we examined the relationships of
the Kettle Test with demographic and motor variables. The
correlations between the Kettle Test scores and age, years of
education, and FMA were all low and nonsignificant (rs =
.04, –.14, and –.19, respectively). Regarding gender, no sig-
nificant differences were found between men and women on
the Kettle Test (t [34] = .435, p = .67).
Table 1. Demographic Variables and Performance on the Kettle Test by Group
Stroke Control
Demographics n (%) n (%) Results and Significance
Gender
Male 22 (61) 7 (19) χ2(1) = 12.99
p = .000
Female 14 (39) 29 (81)
Total 36 (100) 36 (100)
Range M (SD) Range M (SD)
Age (years)
Education (years)
60–89
0–19
74.81 (7.32)
10.44 (5.28)
60–84
0–19
72.67 (6.59)
10.44 (5.28) t(70) = 1.30, ns
t(70) = –2.31
p = .02413.10 (4.05)
Kettle Test scores
Total scorea1–29 9.34 (5.79) 0–3 0.42 (0.91) F(1,60)b = 63.53
p = .000
Assistance level n (%) n (%)
Independent 5 (17) 35 (97) χ2(3) = 43.53 and 3.53
t = .000
Mild assistance 13 (45) 1 (3)
Considerable assistance 11 (38) 0 (0)
Note. m = mean; SD = standard deviation; ns = not significant.
aHigher scores represent more assistance needed.
bAnalysis of covariance, controlling for years of education.
596 September/October 2009, Volume 63, Number 5
Construct Validity
As seen in Table 1, the Kettle Test scores of the stroke group
were significantly higher than those of the control group,
showing the need for more assistance on the test. ANCOVA,
controlling for years of education (because a significant dif-
ference was found between groups on this variable), demon-
strated a large significant group effect on the test (F[1, 60]
= 63.53, p = .000). The study group showed a wide range of
performance on the Kettle Test (1–29), whereas the control
group showed a narrow range (0–3), and only 1 control
participant required mild assistance, demonstrating a floor
effect on the test for this group.
Convergent Validity
The mean scores of the study group on the conventional
battery of cognitive measures are presented in Table 2.
Results show a wide range of performance on all measures.
The Kettle Test, which targets cognitive abilities in a func-
tional context, was found to moderately significantly corre-
late with these conventional cognitive measures (Table 2).
The correlations of the Kettle Test with the four cognitive
measures ranged from .478 to .659 (all significant at p < .01),
the highest being with the CognFIM, which is a measure
based on observation in ADLs.
Ecological Validity
The mean scores of the stroke group on the functional out-
comes were as follows: 76.09 (SD =12.19) on the FIM Motor
scale, 4.94 (SD = 0.75) on the Safety Rating scale, and 10.70
(SD = 5.07) on the IADL scale at follow-up. The correlations
of the Kettle Test and the other cognitive measures with
these functional outcomes are presented in Table 3. The
Kettle Test scores were significantly correlated with all three
outcome measures, FIM Motor (r = –.759), Safety (r =
–.571), and IADL 1 month after discharge at home (r =
–.505). The correlations between the other cognitive mea-
sures and functional outcomes were lower, and no single
measure correlated significantly with all three outcomes.
Discussion
The findings of this study provide initial support for the
reliability and validity of the Kettle Test in stroke rehabilita-
tion. We demonstrated that the Kettle Test—a structured
observation of performance in a complex task—can be scored
consistently among professional occupational therapists. In
addition, participants after stroke required significantly more
assistance than healthy control participants, who scored
almost perfectly on the test. Moreover, the test performance
of participants after stroke was not related to their motor
status or educational background, but it was moderately
correlated with their conventional cognitive test scores.
Finally, Kettle Test scores were found to be significantly
correlated with functional outcomes at the time of discharge
from the rehabilitation hospital and at home.
The results concerning interrater reliability were encour-
aging, considering the difficulty of reliably assessing complex
functional tasks. This positive finding can be attributed to
the structured scoring scale of the observation (divided into
discrete steps) and the expertise of the raters, who were occu-
pational therapists trained in observations of functional per-
formance. Further reliability analysis is in progress using
videotape recordings that will enable comparisons among
multiple raters. Another aspect of reliability pertaining to the
stability of performance (test–retest reliability) was not
deemed relevant for the test. Because the test incorporates
an element of novel everyday problem solving, we hypothe-
sized that immediate learning would occur from the expo-
sure to the task and examiner cueing. Thus, similar to other
tests that incorporate novel problem solving, performance is
not expected to be identical in a subsequent immediate retest
(Wilson, Alderman, Burgess, Emslie, & Evans, 1996).
The construct validity of the test was highly supported
by the significant differences that were found between the
stroke and control groups. This group effect could not be
attributed to the difference in education because the effect
Table 2. Conventional Cognitive Measures: Descriptive Statistics
and Pearson Correlation Coefficients With Kettle Test Scores
Cognitive Measures
Minimum–
Maximum M (SD )
r With
Kettle Test
Mini-Mental Status Evaluation 15–30 26.11 (3.96) –.478*
Clock Drawing Test 2–10 8.16 (1.85) –.566*
BIT Star Cancellation subtest 44–54 52.94 (2.33) –.578*
FIM Cognitive scale 15–35 30.38 (4.44) –.659*
Note. M = mean; SD = standard deviation; BIT = Behavioral Inattention Test.
*p < .01.
Table 3. Pearson Correlation Coefficients of Functional Outcomes
With the Kettle Test and Conventional Cognitive Measures
BADLs
(FIM Motor
Scale)
Safety Level
(RTI Safety
Rating Scale)a
IADLs
(IADLs Scale)
Kettle Test –.759** –.571** –.505**
Mini-Mental Status Evaluation .261 .122 .401*
Clock Drawing Test .365* .112 .182
Star Cancellation subtest of BIT .462** .102 .237
FIM Cognitive scale .435** .446** .287
Note. BADLs = basic activities of daily living; RTI = Routine Task Inventory;
BIT = Behavioral Inattention Test; IADLs = instrumental activities of daily living.
aSpearman correlation analysis was performed on this variable because of its
ordinal 4-point scale.
*p < .05. **p < .01.
The American Journal of Occupational Therapy 597
remained when controlling for years of education. Moreover,
education was not found to be significantly related to per-
formance on the Kettle Test, and no difference was found
between the performance of men and women on the test.
Therefore, the differences between the groups support the
construct validity of the test to differentiate between known
groups with identified differences in cognitive functional
status. We expected that healthy adults who are living inde-
pendently in the community would succeed on the Kettle
Test; this expectation was indeed confirmed by the floor
effect among the control group, in which only 1 participant
needed mild assistance on the test. The findings suggest that
the test can detect cognitive–functional deficits in adults
with neurological dysfunction; however, the test is not sensi-
tive to possible variance in cognitive–functional performance
of normative healthy adults. Further research is required on
larger, more representative samples of different diagnostic
groups with identified cognitive disabilities.
The significant moderate correlations that were found
with conventional established measures of cognition support
the convergent validity of the Kettle Test, demonstrating the
expected strength of association (r = .50) between measures
with overlap of construct (Gregory, 2000). The correlations
with the “table-top” tests (MMSE, CDT, and Star Cancella-
tion) suggest that common underlying cognitive abilities,
including attention, memory, praxis, and executive functions,
are being tapped by these measures. However, the moderate
degree of association supports the contention that the Kettle
Test is also tapping into additional aspects of performance
not addressed by these conventional cognitive measures.
These findings are similar to the initial findings regarding
the Kettle Test in adults with suspected dementia (Hartman-
Maeir et al., 2004) and to those of other studies of top-down
assessments such as the AMPS and the EFPT, where moder-
ate associations were found with table-top measures of cogni-
tion (Baum et al., 2008; Fisher, 2006a). The significant cor-
relation that was found between the Kettle Test scores and the
FIM supports the validity of the test as a measure of cognition
in function. The FIM Cognitive scale provides a valuable
measure of functional cognition in stroke rehabilitation; how-
ever, it relies on the day-to-day observations of clinicians in
the rehabilitation setting and has a high focus on BADLs. This
setting does not necessarily provide an opportunity to observe
the client in the more complex tasks that are essential for
independent living in the community after discharge. The
moderate association that was found may shed light on the
possible unique contribution of a short IADL performance
test, beyond a general observation in daily life.
The ecological validity of the Kettle Test was substan-
tially supported by the significant correlations with all func-
tional outcomes (ADLs, safety, and IADLs). Conversely, the
pattern of correlations between these outcomes and the con-
ventional cognitive measures showed fewer and smaller
effects, and no single measure correlated significantly with
all outcomes. The outcome measures that were chosen for
this study represent functional areas that are pertinent issues
for discharge planning after stroke and for independent com-
munity living. The findings clearly support the advantage of
the Kettle Test in its relation to these outcomes.
From our clinical experience, the ecological and face
validity of the Kettle Test for discharge planning has been
shown to be valuable for clients and their caregivers. On
discharge from hospital, the need for assistance in ADLs and
safety status is of major concern in planning for community
living. Questions such as “Can I return home alone?” or
“What type of assistance will I need to live independently in
the community?” are of vital importance. For example, study
participant A. R. is a 73-year-old married man who was liv-
ing independently in the community before his stroke. He
did not have any residual motor deficits at this stage of reha-
bilitation and was independent in BADLs, yet his conven-
tional cognitive test scores revealed some mild deficits in
measures of clock drawing and visual attention. A. R.’s per-
formance on the Kettle Test demonstrated a significant need
for assistance on multiple steps: He was baffled by the empty
kettle, had difficulty connecting the electrical cord, and only
prepared one cup of beverage (instead of two), using cold
water. The face validity of this assessment for the client and
his wife was useful in accepting the current need for assis-
tance on going home.
Conclusions, Limitations, and
Future Recommendations
The Kettle Test is a brief, cognitive–functional, top-down
measure that assesses actual performance on a familiar IADL
task with built-in complexity. The results of this study support
the reliability and validity of this measure in stroke rehabilita-
tion. The Kettle Test can be used in diverse settings; it is short,
easy to learn and administer, and provides meaningful infor-
mation pertaining to independent community living.
The study was limited by a relatively small sample size
that did not enable multiple regression analysis for examin-
ing the unique prediction of the Kettle Test to the explained
variance of functional outcomes beyond other conventional
measures commonly used in geriatric stroke rehabilitation.
In addition, the data regarding IADLs were collected by a
telephone interview with caregivers, which may not have
fully captured this area of functioning. Further studies should
include home visits to confirm and replicate the current
findings in the geriatric stroke population and in other diag-
nostic groups with cognitive disabilities. s
598 September/October 2009, Volume 63, Number 5
References
Adunsky, A., Fleissig, Y., Levenkrohn, S., Arad, M., & Noy, S.
(2002). Clock drawing task, Mini-Mental State Examination
and cognitive–functional independence measure: Relation to
functional outcome of stroke patients. Archives of Gerontology
and Geriatrics, 35, 153–160
Allen, C. K., Earhart, C. A., & Blue, T. (1992). Occupational
therapy treatment goals for the physically and cognitively dis-
abled. Bethesda, MD: American Occupational Therapy
Association.
American Occupational Therapy Association. (2008). Occupational
therapy practice framework: Domain and process (2nd ed.).
American Journal of Occupational Therapy, 62(6), 625–683.
Baum, C. M., Connor, L. T., Morrison, T., Hahn, M., Dromerick,
A. W., & Edwards, D. F. (2008). Reliability, validity, and
clinical utility of the Executive Function Performance Test:
A measure of executive function in a sample of people with
stroke. American Journal of Occupational Therapy, 62(4),
446–455.
Baum, C., Morrison, T., Hahn, M., & Edwards, D. (2003).
Executive Function Performance Test: Test protocol booklet. St.
Louis, MO: Program in Occupational Therapy, Washington
University School of Medicine.
Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates,
L. M.-A., Dawson, D. R., et al. (2006). The case for the
development and use of “ecologically valid” measures of
executive function in experimental and clinical neuropsy-
chology. Journal of the International Neuropsychological Society,
12, 194–209.
Burns, T. (2006). Cognitive Performance Test (CPT). Pequannock,
NJ: Maddak.
Cahn-Weiner, D., Tomaszewski Farias, S. T., Julian, L., Harver,
D. J., Kramer, J. H., Reed, B. R., et al. (2007). Cognitive and
neuroimaging predictors of instrumental activities of daily
living. Journal of the International Neuropsychological Society,
13, 747–757.
Cicerone K. D., Dahlberg, C., Malec, J. F., Langenbahn, D. M.,
Felicetti, T., Kneipp, S., et al. (2005). Evidence-based cog-
nitive rehabilitation: Updated review of the literature from
1998–2002. Archives of Physical Medicine and Rehabilitation,
86, 1681–1692.
Coster, W., Haley, S. N., Jette, A., Tao, W., & Seibens, H.
(2007). Predictors of basic and instrumental activities of
daily living performance in persons receiving rehabilitation
services. Archives of Physical Medicine and Rehabilitation, 88,
928–935.
Donnovan, N. J., Kendall, D. L., Heaton, S. C., Kwon, S., Velozo,
C. A., & Duncan, P. W. (2008). Conceptualizing functional
cognition in stroke. Neurorehabilitation and Neural Repair,
22, 122–135.
Edwards, D. F., Hahn, M. G., Baum, C. M., Perlmutter, M. S.,
Sheedy, C., & Dromerick, A. W. (2006). Screening patients
with stroke for rehabilitation needs: Validation of the post-
stroke rehabilitation guidelines. Neurorehabilitation and
Neural Repair, 20, 42–48.
Fisher, A. G. (2006a). Assessment of motor and process skills. Vol. 1:
Development, standardization, and administration manual (6th
ed.). Fort Collins, CO: Three Star Press.
Fisher, A. G. (2006b). Assessment of motor and process skills. Vol. 2:
User manual (6th ed.). Fort Collins, CO: Three Star Press.
Folstein, M. F., & Folstein, S. E. (1975). Mini Mental State:
A practical method for grading the cognitive state of
patients for the clinician. Journal of Psychiatric Research,
12, 189–198.
Freedman, M., Leach, L., Kaplan, E., Winocur, G., Shulman, K.,
& Delis, D. C. (1994). Clock drawing: A neuropsychological
analysis. New York: Oxford University Press.
Fugl-Meyer, A. R., Jääsko, L., Leyman, I., Olsson, S., & Steglind,
S. (1975). The post-stroke hemiplegic patient: A method for
evaluation of physical performance. Scandinavian Journal of
Rehabilitation Medicine, 7, 13–31.
Gillen, G. (2009). Managing executive function impairments to
optimize function. In Cognitive and perceptual rehabilita-
tion: Optimizing function (pp. 245–283). New York: Mosby/
Elsevier.
Granger, C. V. (1998). The emerging science of functional assess-
ment: Our tool for outcomes analysis. Archives of Physical
Medicine and Rehabilitation, 79, 235–240.
Gregory, R. J. (2000). Psychological testing: History, principles, and
applications (3rd ed.). Needham Heights, MA: Allyn & Bacon.
Hartman-Maeir, A., Armon, N., & Katz, N. (2005). Kettle
Test protocol. Jerusalem: School of Occupational Therapy,
Hadassah and Hebrew University of Jerusalem.
Hartman-Maeir, A., Katz, N., & Armon, N. (2004, July). Validity
of a cognitive–functional observation (the “Kettle Test”) in an
elderly sample with suspected dementia. Paper presented at the
Israeli Society for Occupational Therapy Annual Conference,
Haifa.
Hartman-Maeir, A., Katz, N., & Baum, C. (2009). Cognitive
Functional Evaluation (CFE) process for individuals with sus-
pected cognitive disabilities. Occupational Therapy in Health
Care, 23, 1–23.
Hershkovitz, A., & Brill, S. (2007). The association between
patients’ cognitive status and rehabilitation outcome in a
geriatric day hospital. Disability and Rehabilitation, 29,
333–337.
Heruti, R. J., Lusky, A., & Dankner, R. (2002) Rehabilitation
outcome of elderly patients after a first stroke: Effect of cogni-
tive status at admission on the functional outcome. Archives of
Physical Medicine and Rehabilitation, 83, 742–749.
Katz, N. (2005). Cognition and occupation across the life span:
Models for intervention in occupational therapy. Bethesda, MD:
AOTA Press.
Katz, N. (2006). Routine Task Inventory–RTI–E manual. Retrieved
March 11, 2009, from www.allen-cognitive-network.org/
images/stories/pdf_files/rtimanual2006.pdf
Law, M., Baum, C., & Dunn, W. (2005). Measuring occupational
performance: Supporting best practice in occupational therapy
(2nd ed.). Thorofare, NJ: Slack.
Lawton, M. P., & Brody, E. M. (1969). Assessment of older
people: Self-maintaining and instrumental activities of daily
living. Gerontologist, 9, 179–186.
Lawton, M. P., Moss, M., Fulcomer, M., & Kleban, M. (1982). A
research- and service-oriented multilevel assessment. Journal
of Gerontology, 37, 91–99.
Levy, L. L., & Burns, T. (2005). Cognitive disabilities reconsid-
ered: Rehabilitation of older adults with dementia. In N. Katz
The American Journal of Occupational Therapy 599
(Ed.), Cognition and occupation across the life span: Models for
intervention in occupational therapy (pp. 347–388). Bethesda,
MD: AOTA Press.
MacNeill, S. E., & Lichtenberg, P.A. (1997). Home alone: The
role of cognition in return to independent living. Archives of
Physical Medicine and Rehabilitation, 78, 755–758.
Rabadi, M. H., & Rabadi, F. M. (2006). Comparison of the
Action Research Arm Test and the Fugl-Meyer assess-
ments as measures of upper extremity motor weakness after
stroke. Archives of Physical Medicine and Rehabilitation, 87,
962–966.
Rouleau, I., Salmon, D. P., Butters, N., Kennedy, C., & McGuire,
K. (1992). Quantitative and qualitative analyses of clock
drawings in Alzheimer’s and Huntington’s disease. Brain and
Cognition, 18, 70–87.
Suhr, J., Grace, J., Allen, J., Nadler, J., & McKenna, M. (1998).
Quantitative and qualitative performance of stroke versus nor-
mal elderly on six clock drawing systems. Archives of Clinical
Neuropsychology, 13, 495–502.
Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans,
J. J. (1996). Behavioral Assessment for Dysexecutive Syndrome.
St. Edmunds, England: Thames Valley Test Company.
Wilson, B., Cockburn, J., & Halligan, P. (1987). Behavioral
Inattention Test: Manual. Suffolk, England: Thames Valley
Test Company.
Zinn, S., Dudley, T. K., Bosworth, H. B., Hoenig, H. M., Duncan,
P. W., & Horner R. D. (2004). The effect of poststroke cog-
nitive impairment on rehabilitation process and functional
outcome. Archives of Physical Medicine and Rehabilitation,
85, 1084–1090.
... Participants expressed knowledge about the Functional Independence Measure and Functional Assessment Measure (FIM, FAM) [37] (a standardised performance-based assessment of function), but only four of the participants reported using this tool regularly. The Kettle Test [38] (a performance-based assessment of functional cognition based on hot drink preparation) was mentioned by some participants, but only two participants reported that they integrated this assessment into their practice. Five therapists mentioned the Assessment of Motor and Process Skills (AMPS) [39] (a well cited standardised performance-based assessment of function in occupational therapy practice) and only two participants were trained in its actual use. ...
Article
Purpose: To investigate how functional cognition of people post-stroke is evaluated and reported by occupational therapists in Ireland. Functional cognition refers to the use and integration of cognitive skills for daily function. Methods: This study used a qualitative design. Six focus groups and one individual interview were conducted with 20 occupational therapists purposively sampled for variation across different clinical grades. Data were analysed according to the Braun and Clark thematic analysis framework. Results: Participants felt that assessment of functional cognition was an integral feature of occupational therapy assessment in stroke care but acknowledged that terminology used by occupational therapists for functional cognition was inconsistent. Non-standardised observational assessment was routinely used by participants. Challenges were reported with respect to written documentation of non-standardised observations. Participants reported that use of standardised cognitive assessments required considered clinical reasoning before administration. Standardised performance-based assessments were not widely implemented by participants. Conclusion: Occupational therapists in Ireland reported a multi-component assessment process to evaluate functional cognition post-stroke. Establishing practice guidelines for the assessment of functional cognition may be of benefit to occupational therapists working in stroke care. Further research is needed to quantify procedures in this assessment process to account for variation in practice.
... To assess usability of the device, a modified Kettle test has been designed with senior Occupational Therapy colleagues in our institution (see Additional File 3). The Kettle test is a brief, objective assessment of functional skills [44] which has been designed specifically to ascertain the participants' ability to independently use the NEMOS© tVNS device. At the end of the third session whereby the participants will have been introduced to the NEMOS© tVNS device on three occasions, worn it for two sessions (one active at the cymba conchae and one sham at the earlobe), they will then be presented with the device and asked to set up and use the device as independently as possible. ...
Article
Full-text available
Background Over 55 million adults are living with dementia globally, which is projected to reach 157 million by 2050. Mild cognitive impairment (MCI), a syndrome of memory impairment with intact activities of daily living, may precede dementia by several years. Around 5–15% of individuals with MCI convert to dementia annually. Novel treatments which delay progression of MCI to dementia are urgently needed. Transcutaneous vagal nerve stimulation (tVNS) is a non-invasive neuromodulation technique that targets the vagus nerve. Importantly, tVNS has been shown to improve cognition in healthy volunteers, but has not been extensively examined as a potential therapeutic approach in MCI. VINCI-AD will examine the safety and feasibility of tVNS in older adults with MCI. Design VINCI-AD is an investigator-led, single-site, single-blind, sham-controlled crossover pilot study which aims to assess the safety and feasibility of tVNS in 40 participants with amnestic MCI. All participants will attend for three consecutive study visits during which they will be randomised to receive no stimulation (baseline), active tVNS stimulation (stimulation at cymba conchae of left ear) or sham tVNS stimulation (at earlobe). Safety will be primarily assessed by ascertainment of adverse events. Further safety assessment will examine the impact of acute tVNS on subjective (orthostatic symptoms), peripheral (finometry-based blood pressure) and central (assessed via Near Infrared Spectroscopy [NIRS]) haemodynamic responses to active stand. Feasibility will be determined using a custom-designed occupational assessment of device usability. Exploratory secondary analysis in VINCI-AD will examine the potential impact of acute tVNS on associative memory, spatial memory and inhibitory control to inform sample size estimates for future trials of tVNS in older adults with MCI. Discussion VINCI-AD will report on the safety (adverse events/haemodynamic responses to active stand) and feasibility of tVNS as a potential therapeutic option in MCI. Detailed reporting of study eligibility and completion rates will be reported. Exploratory analysis will examine the potential cognitive benefits of acute tVNS on cognitive function in MCI to report potential effect sizes that may inform future clinical trials in this cohort. Trial registration https://clinicaltrials.gov/ct2/show/NCT05514756. Trial Registration Number NCT05514756 (24th August 2022 for this protocol, version 1.0.)
... Since Shallice and Burgess (1991) introduced ecologically valid performance-based assessments, occupational therapists have both incorporated the MET into their practice on the basis of scientific evidence (Dawson et al., 2009;Morrison et al., 2013) and developed other performance-based measures to assess the interaction of the person performing an activity in an environment ( Arnad ottir & Fisher, 2008;Baum et al., 2008;Baum & Edwards, 1993;Fisher, 1995;Hartman-Maeir, Harel, & Katz, 2009;Holm & Rogers, 2008). Results from performance assessments enable therapists to help their clients as they face the challenges of setting goals and planning and implementing tasks that are necessary and important as they return to their daily lives. ...
Article
Full-text available
Importance: Functional cognition is emerging as a professional priority for occupational therapy practice. It is important to understand how it relates to other established cognitive constructs, so that occupational therapists can demonstrate their unique contributions. Objective: To examine whether functional cognition is a construct that is distinct from crystallized and fluid cognitive abilities. Design: Secondary analysis of data collected from a cross-sectional study. Setting: Community. Participants: Adults with spinal cord injury, traumatic brain injury, or stroke (N = 493). Outcomes and measures: The National Institutes of Health Toolbox Cognition Battery and the Executive Function Performance Test. Results: We used exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to investigate the factor structure of cognition. EFA identified three factors representing crystallized, fluid, and functional cognition. CFA revealed a second-order model in which the three cognitive constructs contribute hierarchically to a general cognitive factor. Conclusions and relevance: This study provides important and timely evidence for establishing functional cognition as a unique construct that is distinct from executive function as well as from fluid and crystallized cognition. Functional cognition is central to performance in daily activities, and its use will ensure that occupational therapy services support continued recovery and community reintegration. What This Article Adds: This study supports occupational therapy professionals in establishing the profession's role in evaluating and treating deficits of functional cognition to support patients' return to desired occupations in the family, workplace, and community.
... 9 Examples include the Executive Function (EF) Performance Test 10 and the Kettle test. 11 However, there is, currently, no gold standard performance-based assessment 12 and the number of functional domains that can be evaluated are limited. 13 Questionnaire-based methods are also common in assessing complex everyday activities and functional cognition. ...
Article
Full-text available
The Daily Living Questionnaire (DLQ) constitutes one of a number of functional cognitive measures, commonly employed in a range of medical and rehabilitation settings. One of the drawbacks of the DLQ is its length which poses an obstacle to conducting efficient and widespread screening of the public and which incurs inaccuracies due to the length and fatigue of the subjects. Objective This study aims to use Machine Learning (ML) to modify and abridge the DLQ without compromising its fidelity and accuracy. Method Participants were interviewed in two separate research studies conducted in the United States of America and Israel, and one unified file was created for ML analysis. An ML-based Computerized Adaptive Testing (ML-CAT) algorithm was applied to the DLQ database to create an adaptive testing instrument—with a shortened test form adapted to individual test scores. Results The ML-CAT approach was shown to reduce the number of tests required on average by 25% per individual when predicting each of the seven DLQ output scores independently and reduce by over 50% when predicting all seven scores concurrently using a single model. These results maintained an accuracy of 95% (5% error) across subject scores. The study pinpoints which DLQ items are more informative in predicting DLQ scores. Conclusions Applying the ML-CAT model can thus serve to modify, refine and even abridge the current DLQ, thereby enabling wider community screening while also enhancing clinical and research utility.
... Although functional cognition requires looking beyond discrete skills, we contend that initially employing a sound, generalized proxy for network dysfunction can increase access to more specific, comprehensive, and ecologically valid types of intervention currently missing from MiS care. Neurorehabilitation research and practice are moving toward performance-based assessment tools, such as the Executive Function Performance Test (Baum et al., 2008), the Menu Task (Al-Heizan et al., 2020), and the Kettle Test (Hartman-Maeir et al., 2009). Yet, traditional instruments such as the Trail Making Test (Reitan, 1958), Symbol-Digit Modalities Test (SDMT) (Smith, 1973), and Stroop Color-Word Test (SCWT) (Golden and Freshwater, 2002) remain part of the gold standard for validating newer instruments. ...
Article
Full-text available
Contemporary stroke assessment protocols have a limited ability to detect vascular cognitive impairment (VCI), especially among those with subtle deficits. This lesser-involved categorization, termed mild stroke (MiS), can manifest compromised processing speed that negatively impacts cognition. From a neurorehabilitation perspective, research spanning neuroimaging, neuroinformatics, and cognitive neuroscience supports that processing speed is a valuable proxy for complex neurocognitive operations, insofar as inefficient neural network computation significantly affects daily task performance. This impact is particularly evident when high cognitive loads compromise network efficiency by challenging task speed, complexity, and duration. Screening for VCI using processing speed metrics can be more sensitive and specific. Further, they can inform rehabilitation approaches that enhance patient recovery, clarify the construct of MiS, support clinician-researcher symbiosis, and further clarify the occupational therapy role in targeting functional cognition. To this end, we review relationships between insult-derived connectome alterations and VCI, and discuss novel clinical approaches for identifying disruptions of neural networks and white matter connectivity. Furthermore, we will frame knowledge translation efforts to leverage insights from cutting-edge structural and functional connectomics research. Lastly, we highlight how occupational therapists can provide expertise as knowledge brokers acting within their established scope of practice to drive substantive clinical innovation.
Article
Background: Various performance-based instruments exist to assess mental function after stroke and users have to select one for research or clinical practice. Objectives: To evaluate the measurement properties of performance-based instruments to assess (any aspect of) mental function during activity and participation in persons after stroke. Material and methods: We searched in five electronic databases. COSMIN methodology was used to conduct the review. The strength of evidence was assessed using a modified GRADE approach. Results: Fifty articles were included reporting on 20 instruments assessing (1) multiple mental functions including ≥ four subdomains (2) attention, memory and executive functions, or single subdomains (3) executive functions, (4) perception, and (5) mental function of language. Highest quality evidence for sufficient results was found for some measurement properties in seven instruments. These instruments included: FIM + FAM, MPAI-4 and EFPT, MET, CBS/KF-NAP, BIT and the Scenario Test. Conclusions: Further studies of high methodological quality are needed that evaluate the measurement properties of instruments to allow clinicians and researchers to select the most suitable performance-based measures for purpose. Significance: Results may be used to select the most suitable performance-based instrument to measure mental function during activity and participation in persons with stroke. Trial registration number: PROSPERO CRD42018086744.
Article
Full-text available
Introduction and aims Digital biomarkers can provide a cost-effective, objective and robust measure for neurological disease progression, changes in care needs and the effect of interventions. Motor function, physiology and behaviour can provide informative measures of neurological conditions and neurodegenerative decline. New digital technologies present an opportunity to provide remote, high-frequency monitoring of patients from within their homes. The purpose of the living lab study is to develop novel digital biomarkers of functional impairment in those living with neurodegenerative disease (NDD) and neurological conditions. Methods and analysis The Living Lab study is a cross-sectional observational study of cognition and behaviour in people living with NDDs and other, non-degenerative neurological conditions. Patients (n≥25 for each patient group) with dementia, Parkinson’s disease, amyotrophic lateral sclerosis, mild cognitive impairment, traumatic brain injury and stroke along with controls (n≥60) will be pragmatically recruited. Patients will carry out activities of daily living and functional assessments within the Living Lab. The Living Lab is an apartment-laboratory containing a functional kitchen, bathroom, bed and living area to provide a controlled environment to develop novel digital biomarkers. The Living Lab provides an important intermediary stage between the conventional laboratory and the home. Multiple passive environmental sensors, internet-enabled medical devices, wearables and electroencephalography (EEG) will be used to characterise functional impairments of NDDs and non-NDD conditions. We will also relate these digital technology measures to clinical and cognitive outcomes. Ethics and dissemination Ethical approvals have been granted by the Imperial College Research Ethics Committee (reference number: 21IC6992). Results from the study will be disseminated at conferences and within peer-reviewed journals.
Article
The purpose of this paper is to conceptualize the evaluation process for individuals with suspected cognitive disabilities. The Cognitive Functional Evaluation (CFE) process yields a comprehensive profile of the clients' cognitive strengths and weaknesses in occupational performance. The components of the CFE are outlined in six stages as a decision tree with examples of standardized instruments from which to choose the assessments for each client evaluated: (1) interview and background information; (2) cognitive screening and baseline status tests; (3) general measures of cognition and executive functions in occupation; (4) cognitive tests for specific domains; (5) measures of specific cognitive domains in occupations; and (6) environmental assessment. The first three stages are required to ascertain basic cognitive abilities underlying occupational performance. Tests for each stage can be chosen from the ones listed according to the client characteristics and the theory utilized, there is no need to use all of them. Once this data is available a further decision is made whether a more in-depth assessment is needed (stages (4) and (5)). The environmental component is evaluated in all instances with at least one of the assessments. The CFE process for individuals with suspected cognitive disabilities is recommended to be used by occupational therapists as a common ground for evaluation, documentation, and communicating information.
Article
This book is designed to provide the reader with knowledge about the characteristics, objectives, and wide-ranging effects of the consequential enterprise, psychological testing. In addition to a breadth of coverage of traditional topics, this book also provides detailed presentations on neuropsychological and geriatric assessment, the early uses and abuse of testing, assessment of learning disabilities, testing in special settings, race differences in IQ, and cheating on national group achievement tests. The author also describes and critiques the latest versions of the most widely used tests, examines the subtleties of the testing process, and explores the value-laden issues surrounding the wisdom of testing. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The purpose of this study was to examine the validity of the Assessment of Motor and Process Skills (AMPS) when used to evaluate Swedish clients. AMPS is an observational assessment of the extent to which motor and process skill deficits impact on domestic or instrumental daily living task performance (IADL). The validity of the AMPS scales was evaluated in terms of (a) unidimensionality of the items and tasks that comprise each scale, (b) person response validity, and (c) the ability of the scales to differentiate among clients of varying levels of functional ability. The results of a multi-faceted Rasch analysis revealed overall scale and individual response validity of both AMPS scales. The AMPS motor and process scales also differentiated significantly between clients who were judged to be independent, to require minimal assistance, or moderate to maximum assistance in order to live in the community. The results of this study support the validity of using the AMPS evaluation technique in Sweden.