Minimal Neuropsychological Assessment of MS Patients: A Consensus Approach
Cognitive impairment is common in multiple sclerosis (MS), yet patients seen in MS clinics and neurologic practices are not routinely assessed neuropsychologically. In part, poor utilization of NP services may be attributed to a lack of consensus among neuropsychologists regarding the optimal approach for evaluating MS patients. An expert panel composed of neuropsychologists and psychologists from the United States, Canada, United Kingdom, and Australia was convened by the Consortium of MS Centers (CMSC) in April, 2001. Our objectives were to: (a) propose a minimal neuropsychological (NP) examination for clinical monitoring of MS patients and research, and (b) identify strategies for improving NP assessment of MS patients in the future. The panel reviewed pertinent literature on MS-related cognitive dysfunction, considered psychometric factors relevant to NP assessment, defined the purpose and optimal characteristics of a minimal NP examination in MS, and rated the psychometric and practical properties of 36 candidate NP measures based on available literature. A 90-minute NP battery, the Minimal Assessment of Cognitive Function in MS (MACFIMS), emerged from this discussion. The MACFIMS is composed of seven neuropsychological tests, covering five cognitive domains commonly impaired in MS (processing speed/working memory, learning and memory, executive function, visual-spatial processing, and word retrieval). It is supplemented by a measure of estimated premorbid cognitive ability. Recommendations for assessing other factors that may potentially confound interpretation of NP data (e.g., visual/sensory/motor impairment, fatigue, and depression) are offered, as well as strategies for improving NP assessment of MS patients in the future.
The Clinical Neuropsychologist 1385-4046/02/1603-381$16.00
2002, Vol. 16, No. 3, pp. 381–397 # Swets & Zeitlinger
Minimal Neuropsychological Assessment
of MS Patients: A Consensus Approach
Ralph H.B. Benedict
, Jill S. Fischer
, Cate J. Archibald
, Peter A. Arnett
, William W. Beatty
, Gordon J. Chelune
, John D. Fisk
, Dawn W. Langdon
, Lauren Caruso
, Nicholas G. LaRocca
, Lindsey Vowels
, Amy Weinstein
, John DeLuca
Stephen M. Rao
, and Frederick Munschauer
Department of Neurology, State University of New York (SUNY) at Buffalo, Buffalo, NY, USA,
Institute, Buffalo, NY, USA,
Independent Consulting Practice, Chicago, IL, USA,
Foothills Medical Center, Calgary,
Psychology Department, Penn State University, University Park, PA, USA,
Oklahoma University Health Sciences Center, Oklahoma City, OK, USA,
Medical College of Wisconsin, Milwaukee, WI, USA,
Mellen Center for MS Treatment and Research, Cleveland,
Department of Psychology, Queen Elizabeth Health Sciences Center, Halifax, Nova Scotia, Canada,
Department of Psychology, University of London, Surrey, UK,
Department of Neurology, New York University
Medical School, New York, NY, USA,
Department of Psychology, Yeshiva University, Bronx, NY, USA,
National MS Society, New York, NY, USA,
MS Society of Victoria, Victoria, Australia,
Department of Neurology,
University of Rochester Medical Center, Rochester, NY, USA, and
Neuropsychology and Neuroscience Laboratory,
Kessler Medical Rehabilitation Research and Education Corporation, West Orange, NJ, USA
Cognitive impairment is common in multiple sclerosis (MS), yet patients seen in MS clinics and neurologic
practices are not routinely assessed neuropsychologically. In part, poor utilization of NP services may be
attributed to a lack of consensus among neuropsychologists regarding the optimal approach for evaluating
MS patients. An expert panel composed of neuropsychologists and psychologists from the United States,
Canada, United Kingdom, and Australia was convened by the Consortium of MS Centers (CMSC) in April,
2001. Our objectives were to: (a) propose a minimal neuropsychological (NP) examination for clinical
monitoring of MS patients and research, and (b) identify strategies for improving NP assessment of MS
patients in the future. The panel reviewed pertinent literature on MS-related cognitive dysfunction,
considered psychometric factors relevant to NP assessment, deﬁned the purpose and optimal characteristics
of a minimal NP examination in MS, and rated the psychometric and practical properties of 36 candidate NP
measures based on available literature. A 90-minute NP battery, the Minimal Assessment of Cognitive
Function in MS (MACFIMS), emerged from this discussion. The MACFIMS is composed of seven
neuropsychological tests, covering ﬁve cognitive domains commonly impaired in MS (processing speed/
working memory, learning and memory, executive function, visual-spatial processing, and word retrieval). It
is supplemented by a measure of estimated premorbid cognitive ability. Recommendations for assessing
other factors that may potentially confound interpretation of NP data (e.g., visual/sensory/motor impairment,
fatigue, and depression) are offered, as well as strategies for improving NP assessment of MS patients in the
Cognitive dysfunction is common in MS, occur-
ring in nearly half of all patients (Heaton, 1985;
Rao, Leo, Bernardin, & Unverzagt, 1991). The
adverse impact of NP impairment is well
documented in studies of employment (Beatty,
Blanco, Wilbanks, & Paul, 1995; Rao, Leo,
Address correspondence to: Ralph H.B. Benedict, PhD, Department of Neurology, Buffalo General Hospital, Suite
D-6, 100 High Street, Buffalo, NY 14203, USA. Tel.: þ 1-716-859-1403. Fax: þ 1-716-859-1419. E-mail:
Accepted for publication: September 9, 2002.
Ellington, et al., 1991; Rao, Leo, Haughton,
Aubin-Faubert, & Bernardin, 1989), social and
avocational activity (Rao, Leo, Ellington, et al.,
1991) and instrumental activities of daily living
(Amato et al., 1995; Higginson, Arnett, & Voss,
2000; Rao, Leo, Bernardin, et al., 1991). NP
impairment may also hinder a patient’s capacity
to beneﬁt from rehabilitative therapies (Langdon
& Thompson, 1999).
Unfortunately, MS patients’ cognitive deﬁcits
are often difﬁcult to detect during an interview or
neurological examination, no matter how skilled
the clinician (Fischer, 1989; Fischer et al., 1994;
Peyser, Edwards, Poser, & Filskov, 1980). MS-
related cogni tive deﬁcits are often focal rather
than global, may be subtle in their presentation,
and can vary from patient to patient (Fischer,
2001). Striking abnormalities of language and
visual agnosias are rare (Fischer, 2001). Proces-
sing speed/working memory and learning/
memory are the most commonly compromised
functions, but deﬁcits in executive functions and
visual-spatial abilities also occur reasonably
often, either in isolation or in combination with
other deﬁcits (Fischer, 2001; Rao et al., 1991). NP
test performance is only weakly correl ated with
disease duration and physical disability (Beatty,
Goodkin, Hertsgaard, & Monson, 1990; Rao, Leo,
Bernardin, et al., 1991). Although correlations
between NP performance and T2 lesion burden
on magnetic resonance imaging (MRI) are stron-
ger (Rao et al., 1989) they are still too weak to
allow prediction in an individual case.
Despite the acknowledged difﬁculty of detect-
ing cognitive impairment in an interview or neu-
rologic examination, objective NP testing is not
routinely employed in MS clinics. A variety of
factors may account for this. NP assessment is not
required to diagnose MS (McDonald et al., 2001;
Poser et al., 1983) and cogni tive deﬁcits that
might trigger a NP referral may not be evident
during a follow-up ofﬁce visit. NP testing may not
be readily available in some settings, and even if
available, may be costly and time consuming.
With the advent of disease-modifying med-
ications for MS and emphasi s on early interven-
tion and treatment, detection of cognitive
impairment at its earliest stage becomes particu-
larly important. Routine NP testing of MS
patients is feasible, but only if it can be performed
in a cost-effective manner.
NP assessment lies on a continuum ranging
from brief screening measures used to identify
patients in need of further testing, to comprehen-
sive evaluation of a patient’s deﬁcits and strengths
which is indicated when an individual is facing
complex decisions regarding employment, driv-
ing, or capacity for ﬁnancial decision-making.
Routine NP testing would require an intermediate
approach, a minimal NP examination that would
assess the principal features of MS-related
cognitive dysfunction. Several brief batteries
for assessment of MS patients have been proposed
(Franklin, Heaton, Nelson, Filley, & Seibert,
1988; Beatty et al., 1995; Basso, Beason- Hazen,
Lynn J., & Bornstein, 1996). As is evident in
Table 1, these limited NP batteries are too narrow
for routine clinical assessment due to their omis-
sion of key cognitive domains in the interest
of brevity. In contrast, the collaborative NP
research battery proposed by Peyser, Rao,
LaRocca, and Kaplan (1990) is inefﬁcient for
routine clinical NP assessment in that it in-
corporates measures of cognitive functions that
are typically spared by MS (see Table 2).
Furthermore, the Peyser battery includes non-
standardized or abbreviated measures that would
be difﬁcult to apply clinically on a widespread
Recognizing that approaches to NP assessment
often differ across examiners and settings, the
Consortium of MS Centers convened an expert
panel to recommend a minimal set of NP tests for
routine clinical assessment of MS patients. Meth-
ods used by previous consensus groups that
focused on a core collaborative NP research
battery (Peyser et al., 1990), gaps in the empirical
literature for clinical NP practice in MS (Fischer
et al., 1994), and attempts to develop a functional
composite outcome measure for MS clinical trials
(Fischer, Rudick, Cutter, & Reingold, 1999a;
Rudick, Antel, Confavreux, et al., 1996), served
as models for this effort. The panel’s primary
objectives were to: (a) identify a parsimonious set
of currently available, psychometrically sound
NP measures for assessing MS patients; and (b)
develop strategies for improving the proposed
battery in the future.
382 RALPH H.B. BENEDICT ET AL.
Table 2. Peyser et al. (1990) Core Battery of Neuropsychological Tests.
Cognitive domain Test(s)
Global dementia screen Mini-Mental State Examination (MMSE)
Processing speed/working memory Auditory As, Trails A; Paced Auditory Serial Addition Test (PASAT);
Symbol-Digit Modalities test (SDMT); Modiﬁed Stroop Test
Learning and memory California Verbal Learning Test (CVLT); Wechsler Memory Scale –
Revised (WMS–R) Logical Memory; 7/24 Spatial Recall Test
Reasoning and executive functions Wisconsin Card Sorting Test (WCST); Raven’s Standard Progressive
Matrices; WAIS–R Comprehension
Visual-spatial abilities Abbreviated Hooper Visual Organization Test (HVOT); Modiﬁed
WAIS–R Block Design
Language A bbreviated Token Test; Abbreviated Boston Naming Test;
Controlled Oral Word Association Tests
Note. Auditory As, Trails A and the Raven’s Standard Progressive Matrices were considered optional tests.
Table 1. NP Screening Batteries Proposed for MS.
Screening battery Reference Tests included
Franklin et al. (1988) Symbol-Digit Modalities Test (SDMT); Trails A,
B; Numerical Attention Test; Prose learning
(story from Wechsler Memory Scale); Visual
learning and copy (part of Rey–Osterreith
Comple x Figure); Visual Naming, Aural
Comprehension of Words and Phrases, and
Speech Articulation Rating from Multilingual
Aphasia Examination; Western Aphasia Bat-
tery Commands with Auditory Sequencing;
Phonemic ﬂuency (written and oral)
Rao et al. (1991) Paced Auditory Serial Addition Test (PASAT);
Buschke Verbal Selective Reminding Test (12
items 12 trials, with delay); 7/24 Spatial
Recall Test; Controlled Oral Word Association
Screening Examination for
Beatty et al. (1995) Verbal learning (10 items 3 trials); Shipley
Institute of Living Scale (SILS); SDMT (oral
Basso Screening Battery Basso et al. (1996) Seashore Rhythm Test; Wechsler Memory
Scale – Revised Logical Memory; Controlled
Oral Word Association Test; Halstead–Reitan
Graphesthesia and Stereognosis
A slightly modiﬁed version of this battery, termed the Brief Repeatable Neuropsychological Battery, has also been
proposed, which includes the Selective Reminding Test (12 items 6 trials); the 10/36 Spatial Recall Test; PASAT;
SDMT; and phonemic ﬂuency (3 items).
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 383
PROCESS OF DERIVING A MINIMAL
Phase 1: Panel Selection
and Conference Preparation
The expert panel was co-chaired by Drs. Benedict
and Fischer. Neuropsychologists and clinical
psychologists who had published extensively on
neuropsychological aspects of MS, and experi-
enced clinicians providing neuropsychological
and/or psychological services in major MS
clinics, were identiﬁed. Of the 19 individuals
invited to the April, 2001 conference in New
Orleans, 14 agreed to partic ipate, 2 could not
attend but agreed to offer comments on the results
of the meeting, and 3 declined due to conﬂicting
commitments. Panel members represented four
countries: the United States, Canada, Australia,
and the United Kingdom. In preparation for the
conference, participants read a core literature on
NP deﬁcits in MS and psycho metric considera-
tions in NP assessment and reviewed lists of NP
measures used in published MS studies (copies of
the lists of articles and measures are available
from the senior author).
Phase 2a: Deﬁning the Features
of an Optimal NP Examination in MS
At the conference, participants ﬁrst agreed that an
optimal minimal NP examination in MS should
Be sensitive enough to detect mild cognitive
impairment and be able to discriminate patients
with subtle deﬁcits from heal thy control s.
Monitor changes in cognitive function over
time, both natural changes related to disease or
normal aging and treatment-related changes
(i.e., be responsive to change).
Facilitate clinical decision-making, including
treatment planning (with treatment broadly
deﬁned to include medication, rehabilitation,
Be applicable across a wide range of settings
(i.e., be ‘‘translatable’’ to other languages and
cultures, minimize the use of ‘‘high tech’’
equipment if possible, and minimize motor
demands on the patient).
Be parsimonious (i.e., administered in 1–2 hr).
After presentations on MS-related cognitive
impairment, longitudinal studies of cognitive
dysfunction, and the contribution of non-cogni-
tive factors (e.g., depression, fatigue), ﬁve
cognitive domains were deemed essential to
include in the battery: (a) processing speed/
working memory; (b) learning and memory; (c)
visual-spatial processing; (d) executive function;
and (e) language/other verbal abilities.
Phase 2b: Establishing Criteria
for Evaluating Candidate Measures
The panel then considered psychometric issues in
NP assessment, particularly those pertinent to
repeated assessment (e.g., use of alternate forms).
Following presentations on test-retest reliability
and validity, participants discussed psychometric
and practical criteria against which to evaluate
candidate NP measures, ultimately arriving at a
consensus on the criteria summarized in Table 3.
Other desirable features of candidate measures
identiﬁed by conference participa nts included
construct validity (i.e., the extent to which a test
measures what it purports to measure, as evi-
denced by its pattern of correlations with similar
and dissimilar measures), efﬁciency (i.e., the num-
ber of distinct, meaningful scores that it yields),
and ability to identify and separate underlying
Phase 2c: Nominating and Evaluating
the Merits of Speciﬁc Candidate Measures
Participants then nominated 36 measures to be
considered for inclusion in the minimal NP
examination (10 processing speed/working mem-
ory, 10 learning/memory, 7 executive function, 6
visual-spatial ability, and 3 language) (the list of
candidate measures is available from the ﬁrst
author). Referring to published data in journal
articles, test manuals, and standard NP references
(e.g., Lezak, 1995; Spreen & Strauss, 1998),
participants then evaluated how well each candi-
date measure fulﬁlled each of the criteria
summarized in Table 3 using a 3-point scale
(adequate , inadequate [ÿ1], or equivocal/
unclear ). In rare cases when published data
were not available, unpublished data from work in
progress by panel members were considered.
384 RALPH H.B. BENEDICT ET AL.
Phase 3: Derivation of Minimal NP Exa m
After the conference, the co-chairs prepared
detailed tables containing scores for each candi-
date measure on each of the psychometric and
practical criteria identiﬁed in Phase 2b (Table 3).
Panel members then rank-ordered the ﬁve cogni-
tive domains identiﬁed in Phase 2a in terms of
their relative importance in MS assessment, and
within each domain, rank-ordered the candidate
measures based on consensus criteria. Descriptive
statistics (sum, median, mode, minimum, and
maximum) were used to summarize the distribu-
tions of panelists’ rankings. These statistics were
presented to the expert panel for review and
comment. A recommended test had to: (a) be
ranked ﬁrst or second within its cognitive domain;
(b) have a median ranking of 1 or 2; (c) have a
modal ranking of 1 or 2; and (d) not be ranked as
‘‘unacceptable’’ by more than two panelists. This
algorithm yielded clear choices in four of the ﬁve
cognitive domains. However, no visual-spatial
tests fulﬁlled this standard, and the rankings of the
top two tests were very similar. Consequently,
panel members were asked which of these two
tests they preferred at which point a strong
Phase 4: Postconference Work Groups
Three work groups reviewed the pertinent litera-
ture and offered recommendations regarding: (a)
methods for assessing factors that may potentially
confound interpretation of MS patients’ NP
performance (e.g., depression; fatigue; and visual,
sensory, or motor impairments) (P.A., J.D.F.,
D.L.); (b) promising experimental methods for
assessing processing speed/working memory
(C.A., J.D.F.); and (c) alternative approaches to
assessing verbal ﬂuency that would be com-
parable across languages and cultures (W.B.,
D.L.). These work groups circulated summaries
of their ﬁndings and recommenda tions to the
entire panel for consideration, and consensus
recommendations were incorporated into this
Table 3. Consensus Criteria for Evaluating Adequacy of Candidate Measures for the Minimal NP Battery.
1. Standardized stimulus materials
Standardized test stimuli and instructions for test administration
should be readily available.
2. Normative data Published normative data on the test performance of a large sample of
healthy adults in the age range frequently encountered in MS (20–
55) should be available.
3. Adequate range The test should be free of ceiling and ﬂoor effects.
4. Reliability Test scores should have moderate to high test-retest reliability. Data
pertaining to internal consistency and inter-rater reliability may
also be available, but are less critical.
5. Criterion validity The test should be able to discriminate MS patients (particularly
mildly impaired patients) from healthy controls with reasonable, if
not high, sensitivity and speciﬁcity.
6. Alternate forms The test should have alternate forms of equivalent difﬁculty. This
standard may not be equally applicable to all cognitive domains.
7. Practical The test should be brief and easily administered in a wide variety of
settings with minimal equipment. Test performance should not be
strongly confounded by basic motor and sensory functions unless
there is no other way to assess the function of interest.
Note. The adequacy of the 36 candidate measures was separately assessed on each criterion, using a 3-point scale
(adequate , inadequate [ÿ1], or equivocal/inconclusive data ).
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 385
EXPERT PANEL RECOMMENDATIONS
Proposed Minimal NP Examination
There was nearly unanimous agreement about the
rankings of cognitive domains, as processing
speed/working memory and learning/memory
were ranked ﬁrst or second by nearly all panelists:
processing speed/working memory (rank sum ¼
17, range ¼ 1–3), learning and memory (rank
sum ¼ 25, range ¼ 1–3), executive function
(rank sum ¼ 37, range ¼ 2–4); visual-spatial
ability (rank sum ¼ 54, range ¼ 4–5); language
and other abilities (rank sum ¼ 62, range ¼ 3–5).
These rankings parallel data on the prevalence of
cognitive impairment in MS patients across
domains (Rao, Leo, Bernardin, et al., 1991). The
proposed minimal NP examination included two
tests in each of the domains that the expert panel
ranked as most important to assess (processing
speed/working memory and learning/memory),
and one test in each of the other domains. Table 4
presents this minimal NP examination for MS
Table 4. Tests in the Minimal Assessment of Cognitive Function in MS (MACFIMS).
1 or 2
Processing speed/Working memory
Paced Auditory Serial Addition Test 1 86 Rao Version, using a 3.0 and 2.0 inter-
stimulus interval. Normative data based
on 100 healthy controls in peer-reviewed
publication (Rao, Leo, Ellington, et al.,
Symbol Digit Modalities Test 2 71 Oral administration only. Normative data
based on 100 healthy controls in peer-
reviewed publication (Rao, Leo,
Ellington, et al. 1991).
Learning and memory
California Verbal Learning Test–II 1 86 There are two equivalent alternate forms.
Brief Visuospatial Memory Test – Revised 2 71 There are six equivalent alternate forms.
D-KEFS Sorting Test 1 79 The test is commercially available as of
August, 2001. There are two adminis-
tration formats. To conserve time, panel
recommends administration of free sort
condition only, for which separate norms
Visual perception/Spatial processing
Judgment of Line Orientation Test 1 93 The two forms (V, H) described in the
Benton et al. (1994) manual are actually
the same test items administered in a
different order. Although odd/even item
splits of the JLO have been evaluated, the
panel recommends the full length JLO in
the interest of ensuring test reliability
Controlled Oral Word Association Test 1 86 Two alternate forms, letters C, F, L and P,
R, W. Suggested normative data pro-
vided by Benton and Hamsher (1989).
386 RALPH H.B. BENEDICT ET AL.
patients, referred to as the Minimal Assessment of
Cognitive Function in Multiple Sclerosis (MAC-
FIMS). Each test is brieﬂy described below within
a general discussion of its respective cognitive
Deﬁcits in information processing speed, and
in the capacity to tempor arily store and simulta-
neously manipulate verbal or visuospatial infor-
mation in working memory, are well documented
in MS patients (Archibald & Fisk, 2000; Beatty,
Goodkin, Monson, & Beatty, 1989; Camp et al.,
1999; D’Esposito et al., 1996; DeLuca, Johnson,
& Natelson, 1993; Demaree, DeLuca, Gaudino, &
Diamond, 1999; Fisk & Archibald, 2001;
Franklin et al., 1988; Litvan, Grafman, Vendrell,
& Martinez, 1988a; Litvan, Grafman, Vendrell, &
Martinez, 1988b; Kujala, 1994; Rao, 1989).
Available clinical measures unfor tunately do not
distinguish the relative contributions of proces-
sing speed and components of working memory.
According to the expert panel’s rankings, the
Paced Auditory Serial Addition Test (PASAT) is
the best of the currently available clinical mea-
sures at fulﬁlling the psychometric and practical
criteria established for selecting measures for the
minimal NP exam for MS.
First developed by Gronwall to assess patients
recovering from concussion (Gronwall, 1977), the
PASAT requires patients to monitor a series of 61
audiotaped digits while adding each consecutive
digit to the one immediately preceding it. The
PASAT requires both rapid information proces-
sing and simultaneous allocation of attention to
two tasks, as well as reasonably intact calculation
ability. In its origina l format, the PASAT was
administered at four inter-stimulus intervals
(2.4, 2.0, 1.6, and 1.2 s). The number of inter-
stimulus intervals and presentation rates were
subsequently modiﬁed by Rao and colleagues
for use with MS patients (3.0 and 2.0 s) (Rao,
Leo, Bernardin et al., 1991). This version was
selected to be a component of the MS Functional
Composite (MSFC), a clinical outcome measure
composed of quantitative measures of leg, arm/
hand, and cognitive function (Cutter et al., 1999;
Fischer, Rudick et al., 1999; Rud ick et al., 1996).
The standard dependent variable on the PASAT is
the total number of correct responses at each
presentation rate (or a variant of this such as
processing rate, which aggregates performance
across presentation rates). The expert panel
strongly recommended work on a more detailed
analysis of PASAT performanc e that accounts for
strategies used to perform well (e.g., dyad scor-
ing) (Fisk & Archibald, 2001). Two equivalent
alternate forms of the Rao version of the PASAT
The expert panel also ranked a second measure
of processing speed/working memory highly, the
Symbol Digit Modalities Test (SDMT; Smith,
1982). The SDMT presents a series of nine
symbols, each of which is paired with a single
digit in a key at the top of an 8.5 11 in. sheet.
The remainder of the page has a pseudo-rando-
mized sequence of these symbols and the subject
must respond with the digit associated with each
of these as quickly as possible. Visual scanning
and, to a lesser extent, secondary memory are
involved in that patients must either rapidly locate
the correct pairing on the key or recall these
symbol-digit pairings. The SDMT was originally
designed to permit either written or oral
responses, but the panel recommended the oral
administration with MS patients to minimize
confounds due to upper extremity weakness or
incoordination. The primary dependent variable
on the SDMT is the total number of items correct
in 90 s. Numerous alternate forms were developed
by a National MS Society Cognitive Function
Study Group (National Multiple Sclerosis Society
Cognitive Functions Study Group, 1990) although
a recent study suggests that these forms may not
be equivalent (Boringa et al., 2 001).
Poor recent memory is perhaps the most com-
mon NP complaint of MS patients, so it is not
surprising that deﬁcits on tests requiring recall of
recently acquired information are often obser ved
(Beatty, Goodkin, Monson, Beatty, & Hertsgaard,
1988; Fischer, 1988; Grant, McDonald, Trimble,
Smith, & Reed, 1984; Minden, 1990; Rao, 1989;
Rao, Hammeke, McQuillen, Khatri, & Lloyd,
1984). The Califor nia Verbal Learning Test,
which has recently undergone revision (CVLT-
II) (Delis, Kramer, Kaplan, & Ober, 2000), was
the measure of learning and memory ranked most
highly by nearly all members of the expert panel.
Learning of a 16-word list (List A) is ﬁrst exam-
ined over the course of ﬁve trials, then recall of a
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 387
new 16-word list (List B) is assessed on an
interference trial, after which recall of List A is
reassessed. Delayed recall and recognition (yes/
no and forced choice) of List A are assessed again
after a 25-min delay interval. The total number of
words recalled on Trials 1–5 and on Delayed Free
Recall are likely to be most sensitive to MS-
associated cognitive impairment (Delis, Kramer,
Kaplan, & Ober, 1987; Scarrabelotti & Carroll,
1998). The CVLT-II provides a rich array of
additional variables that permit the examiner to
examine cognitive processes that may affect over-
all performance (e.g., semantic clustering, sus-
ceptibility to interference, and learning curve),
although scores on these variables may not be as
stable statistically as the summary scores. One
equivalent alternate form of the CVLT-II is
The Brief Visuospatial Memory Test – Revised
(BVMT–R; Benedict, 1997; Benedict, Schretlen,
Groninger, Dobraski, & Shpritz, 1996) is an
efﬁcient measure of spatial learning and memory
that was also ranked highly by the majority of the
panelists. Learning of a matrix of six simple
abstract designs (presented for 10 s) is examined
over three trials, and then delayed recall and
recognition (yes/no) are assessed after 25 min.
The BVMT–R is sensitive to cerebral disease in
general (Benedict, 1997) and has also been used
with MS patients (Benedict, Priore, Miller,
Munschauer, & Jacobs, 2001). Some manual
dexterity is required to produce the designs, but
subjects are allowed as much time as necessary to
complete them and motor problems evident on a
copy trial are taken into account when scoring
design accuracy. The BVMT–R offers six alter-
nate forms, the equivalence of which is well
The term ‘‘executive functions’’ refers to a
collection of related abilities comprising abstract
reasoning, conceptual ﬂexibility, and the planning
and organization of behavior. Although less well
studied than deﬁcits in processing speed/working
memory or learning/memory, deﬁcits in executive
functions occur with considerable frequency in
MS (Beatty, 1995; Beatty & Monson, 1994;
Beatty & Monson, 1996; Foong et al., 1999;
Foong et al., 1997; Rao, Leo, Bernardin et al.,
1991b). Moreover, impairments in executive
functions can affect performance on tests of
other cognitive abilities (Beatty & Monson,
1996; Foong et al., 1997; Troyer, Graves, &
Cullum, 1994). The D–KEFS Sorting Test (DST;
Delis, Kaplan, & Kramer, 2001), a measure of
conceptual reasoning that permits the differentia-
tion of concept formation from conceptual ﬂex-
ibility, emerged as the test of executive function
most strongly favored by the expert panel. Stan-
dardized administration of the DST calls for both
free sorting and structured (cued) sorting condi-
tions, but the free sorting condition can be admi-
nistered alone to reduce total testing time.
Numerous dependent variables can be examined
(e.g., number of sorts attempted, number of cor-
rect verbal concepts, number of correct perceptual
concepts, number of perseverations, and types of
rule violations). The DST has two alternate forms,
an important quality for a measure of novel
problem-solving that will be used to monitor
changes in function over time.
Deﬁcits in visual-spatial abilities are nearly as
common as executive dysfunction in MS (Beatty
et al., 1989; Rao, Leo, Bernardin et al., 1991;
Ryan, Clark, Klonoff, Li, & Paty, 1996; Van den
Burg, Van Zomeron, Minderhoud, Prange, &
Meijer, 1987; Vleugels et al., 2000). The expert
panel selected Benton’s Judgement of Line Orien-
tation Test (JLO; Benton, Sivan, Hamsher,
Varney, & Spreen, 1994) as the best available
measure of visual-spatial ability for the minimal
NP examination. Brief versions of the JLO can be
formed by administering only odd or even items
(Woodard et al., 1996; Woodard et al., 1998) but
the expert panel recommended the full-length
JLO for the minimal NP exam. The JLO requires
subjects to identify the angle deﬁned by two
stimulus lines from among those deﬁned by a
visual array of lines covering 180 degrees. The
dependent variable on the JLO is the total number
of correct responses.
MS patients may also present with subtle
language problems, the most common of which
is deﬁcient word retrieval (Beatty et al., 1989;
Rao, Leo, Bernardin et al., 1991b). The Con-
trolled Oral Word Association Test (COWAT;
Benton & Hamsher, 1989) a measure of phonemic
ﬂuency, was selected for the minimal NP exam
based on its strong psychometric properties and
388 RALPH H.B. BENEDICT ET AL.
sensitivity to impairment in MS. On this task,
subjects have 1 min to generate as many words as
they can that begin with each of three different
stimulus lett ers, which are presented individually.
Thus, the COWAT cannot be considered a ‘‘pure’’
language measure test because performance is
strongly affected by the efﬁciency and speed of
searching one’s lexicon. The principal dependent
variable is the total number of words generated
across all three trials. Strategies used to perform
the COWAT (e.g., clustering, switching) can also
be examined (Troyer, 2000; Troster et al., 1998),
although the statistical reliability of these scores
is not well established. Two equivalent alternate
forms are available. Norms for other languages
and cultures and the equivalence of these forms in
other languages remain to be established.
Assessing Potential Confounds to NP
Interpretation of the NP test performance of MS
patients can be confounded by a number of
factors, including variation in premorbid intellec-
tual ability, depression, fatigue, sensory/motor
impairment, and medications. The panel strongly
recommended that: (a) a culturally appropriate
measure of premorbid ability be administered the
ﬁrst time a patient is assessed; (b) a measure of
depression be administered routinely in conjunc-
tion with the minimal NP exam; and (c) measures
of visual/sensory/motor defects and fatigue be
employed when indicated.
The inﬂuence of premorbid cogni tive ability
on NP test results is well established and clin-
icians routinely compare current test performance
with an estimated baseline capacity (Lezak,
1995). Educational level and occupation are
often used as surrogates for premorbid abilities,
either informally, or formally through the appli-
cation of regression formulas (Barona, Reynolds,
& Chastain, 1984). However, educational and
occupational attainment can be inﬂuenced by
cultural expectations, particularly among older
adults. Several tests used as surrogates for pre-
morbid ability were suggested by the panel, with
the acknowledgement that none is likely to be
applicable across all countries and cultures. Most
panel members favored the North American
Adult Reading Test (NAART; Blair & Spreen,
1989) or the National Adult Reading Test in the
UK and Australia (Nelson & Willison, 1991) to
assess premorbid function. Selected verbal subt-
ests from the WAIS-III (Wechsler, 1997; Informa-
tion, Vocabulary, or the Verbal Comprehension
Index) or the Reading subt est from the Wide
Range Achievement Test, third edition (WRAT-
III; Wilkinson, 1993) wer e also proposed by some
members. These measures can be used to set
general expectations for patient performance,
bearing in mind that test scores are subject to
regression to the mean (making subjects appear
more ‘‘average’’ than they were premorbidly) and
that premorbid ability may be underestimated in
MS patients with global impairment (Friend &
Depression is common in MS and can
adversely affect NP performance, particularly on
capacity-demanding tasks (Arnett, Higginson,
Voss, Wright, et al., 1999; Arnett, Higginson,
Voss, Bender, et al., 1999). Mood disturbance
can also affect the accuracy of patients’ percep-
tions of their performance (Fischer, 1989). The
expert panel recommended the 42-item Chicago
Multiscale Depression Inventory (CMDI;
Nyenhuis et al., 1995), a self-report measure of
depression, for use in conjunction with the
MACFIMS. Although it may be less familiar than
some measures, the CMDI has separate subscales
for the Vegetative, Mood, and Evaluative compo-
nents of depre ssion, allowing the clinician to
determine whether the total score may be ‘‘con-
taminated’’ by MS symptoms that overlap with
those of depression (Mohr & Goodkin, 1999). The
seven-item Beck Depression Inventory – Fast
Screen (BDI–FS; Beck, Steer, & Brown, 2000),
designed to screen medical patients for depres-
sion, is also promising but needs to be validated in
an MS sample before it can be recommended.
Brief standardized measures of sensory and
motor funct ions aid clinicians in interpreting NP
test ﬁndings. Hand-held eye charts (e.g.,
Rosenbaum Pocket Vision Screener) can be
used to determine whether a patient’s visual
acuity is adequate to perform tests in the
MACFIMS. Small print characters presented dur-
ing NP testing are similar to the characters pre-
sented at the 20/50–70 threshold when the
Rosenbaum card is positioned 14 in. from the
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 389
corrected eye. By design, most tests in the
MACFIMS require an oral rather than a manual
motor response. Scoring on the BVMT–R, on
which the patient must manually rende r a matrix
of visual desi gns from memory, can be informally
corrected for mild tremor by using the patient’s
copied designs as a reference (Bened ict, 1997).
The panel recommends the use of quantitative
tests of upper extremity motor speed and coordi-
nation such as the 9-Hole Peg Test (Smith þ
Nephew, Notts, UK), which has been incorporated
into the MS Functional Composite measure
(Cutter et al., 1999; Fischer, Rudick, et al., 1999;
Rudick et al., 1996). Finally, performance on tests
requiring rapid oral responses can be confounded
by dysarthria in some patients. The Maximum
Repetition Rate of Syllables and Multisyllabic
Combinations (Kent, Kent, & Rosenbek, 1987)
is a brief series of tasks requiring the rapid repeti-
tion of syllables (e.g., ‘‘pa,’’ ‘‘ta,’’ and ‘‘ka’’) in
separate 10-s trials, with the last trial involving
three syllables repeated in sequence (e.g., ‘‘pa, ta,
ka..pa, ta, ka’’). This test was recommended to
evaluate the degree to which a primary articulatory
speed output problem is contributing to deﬁcient
performance on tests requiring rapid oral
responses (i.e., PASAT, SDMT, and COWAT).
Early MS research suggested that testing-
induced fatigue had little effect on NP test per-
formance (Van den Burg et al., 1987). However,
recent investigation suggests that fatigue can
compromise performance on measures placing
signiﬁcant demands on processing speed and
working memory (Johnson, Lange, DeLuca,
Korn, & Natelson, 1997; Krupp, 2000). Conse-
quently, measures such as the PASAT and the
SDMT should be separated by measures of other
cognitive functions during examination (see pro-
posed order of administration in Table 5). Clin-
icians who suspect that fatigue may substantially
affect a patient’s test performance are encouraged
to administer the Fatigue Impact Scale (FIS;
Krupp, 2000). Scores of 75 or greater on the
original (40-item) FIS are associated with signiﬁ-
cant functional limitations (Fisk, Pontefract,
Ritvo, Archibald, & Murray, 1994). Both the
original version of the FIS, and a 21-item version
developed for the MS Quality of Life Inventory
(MSQLI) by eliminating redundant items
(Fischer, LaRocca, et al., 1999), contain a reliable
Cognitive Fatigue subscale. However, as this
subscale is highly correlated with other FIS sub-
scales, it should not be interpreted as a ‘‘pure’’
measure of ‘‘cognitive fatigue.’’
Table 5. Suggested Order of Administration for
Minimal NP Exam.
CMDI ( þ FIS if appropriate) Variable
Record review/clinical interview
measures as appropriate
BVMT–R Learning Trials 7
BVMT–R delayed recall
SDMT (oral administration) 5
CVLT-II learning trials and
DST (free sorting condition) 25
CVLT-II long-delay recall
(If initial assessment)
Measure of premorbid abilities
(NAART, NART, WRAT-3
Reading, or selected WAIS-III
Note. CMDI ¼ Chicago Multiscale Depression Inven-
tory; FIS ¼ Fatigue Impact Scale; 9-HPT ¼
9-Hole Peg Test; MRRSMC ¼ Maximum Repe-
tition Rate of Syllables and Multisyllabic Com-
binations; COWAT ¼ Controlled Oral Word
Association Test; BVMT–R ¼ Brief Visuospatial
Memory Test – Revised; PASAT ¼ Paced Audi-
tory Serial Addition Test; JLO ¼ Judgment of
Line Orientation; SDMT ¼ Symbol-Digit Mod-
alities Test; CVLT-II ¼ California Verbal Learn-
ing Test – Second Edition; DST ¼ D–KEFS
Sorting Test; NAART ¼ North American Adult
Reading Test; NART ¼ National Adult Reading
Test; WRAT-3 ¼ Wide Range Achievement
Test – 3rd edition; WAIS-III ¼ Wechsler Adult
Intelligence Scale III.
390 RALPH H.B. BENEDICT ET AL.
The Minimal Assessment of Cognitive Function
in MS (MACFIMS) is a clinically oriented NP
examination of cognitive functions in MS. To be
sure, deﬁcits in these domains are not unique to
MS, but rather, can be present in patients with
many other conditions known to affect the ce-
rebral white matter or cortical-subcortical cir-
cuits. This battery represents the consensus of an
expert panel of 16 neuropsychologists and clinical
psychologists with extensive experience in MS
research and/or clinical care. The designation of
these proposed tests as the ‘Minimal Assessment
of Cognitive Function in MS’ should by no means
imply that these are the only tests appropriate for
assessing MS patients. Rather, these speciﬁc tests
were selected because, in the view of the expert
panel, they come closest to meeting psychometric
and practical criteria established a priori for a
minimal NP examination in MS. The MACFIMS
is intended to be used both to detect cognitive
impairment in MS patients and to monitor
changes in cognitive function over time.
This battery can be administered by a tra ined
examiner in approximately 90 min, although addi-
tional time is required for record review and
clinical interview, administration of the self-
report measure of depression, and formal assess-
ment of other factors that may potentially
confound NP test interpretation. While practical
for the routine monitoring of MS patients in
clinical settings and clinical trials, the MACFIMS
is not intended for differential diagnosis of MS
and other neurologic conditions. These tests can
be supplemented by additional NP measures when
a more detailed assessment of cognitive functions
is desired, or when speciﬁc clinical questions
require more comprehensive assessment (e.g.,
differential diagnosis, educational and occupa-
tional planning, and disability determination).
As in any NP exam, appropriate clinical applica-
tion and interpretation requires adequate training
in clinical neuropsychology and familiarity with
the quality of cognitive impairment associated
with the suspected underlying disease.
The MACFIMS differs from the NP battery of
Peyser et al. (Peyser et al., 1990) in that it targets
only those functions commonly disrupted in MS,
includes only clinical measures for which pub-
lished norms are available, and employs tests
selected using explicit psychometric and practical
criteria. The MACFIMS is distinct also from the
Brief Repeatable Battery proposed by Rao and
colleagues (Rao, Leo, Bernardin, et al., 1991) and
other brief NP batteries (Basso, et al., 1996; Beatty
et al., 1995) in its broader coverage of cognitive
functions. Guided by a conceptual consensus-
driven approach to the assessment of speciﬁc
cognitive functions, the expert panel consulted
the literature to identify measures with demon-
strated reliability and validity. This approach was
deemed the most empirically sound approach
possible, given the absence of multicenter NP
datasets that would permit empir ical comparisons
of the psycho metric properties of one test with
another in the same population. In the future, it
will be important to collect data on these measures
at multiple sites to obtain uniform norms (for both
single and repeated assessments) and to conﬁrm
the validity of the various tests over time.
Practice effects (i.e., improvement in task
performance due to prior exposure to task stimuli
and/or procedures when no real change in under-
lying abilities has taken place) are ubiquitous in
NP assessment. Practice effects can complicate
interpretation of NP performance when tests are
readministered to monitor changes in cognitive
function over time (Bever, Grattan, Panitch, &
Johnson, 1995). In group studies, a control group
can be used to estimate the mag nitude of practice
effects, as well as other factors unrelated to the
treatment of interest (i.e., sources of systematic
error, in statistical terms) (Fischer, 1999; Fischer
et al., 2000). However, the clinician assessing an
individual patient typically does not have such a
reference group for estimating practice effects.
Alternate forms have often been recommended as
a method of minimizing the effects of practice on
test performance. Indeed, the use of alternate
forms can mitigate practice effects to some extent
and can help preserve construct validity on some
measures (Benedict & Zgaljardic, 1998). Unfor-
tunately, the equivalence of alternate forms is
often assumed, but it may not be conﬁrmed
when explicitly investigated (Boringa et al.,
2001). Even when the equivalence of alternate
forms is well established, test/retest reliability
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 391
may be attenuated from one test session to
another, which increases the error term in a
statistical analysis and potentially makes it more
difﬁcult to detect a reliable change in test perfor-
mance. Furthermore, it is impossible to eliminate
practice effects entirely when prior exposure to
the task procedures themselves (not just the sti-
muli) facilitates subsequent task performance. We
propose that more research is needed before the
use of alternate forms of NP measures can be
unconditionally recommende d when assessment
of change is paramount. Normative studies using
the MACFIMS (both cross-sectional and long-
itudinal) are planned.
As NP assessment evolves, the MACFIMS
must be ﬂexible enough to incorporate im-
provements in its component measures and to
allow new tests to be substituted if they are
psychometrically superior. Processing speed/
working memory is the cognitive domain in
which improvements are most sorely needed.
The calculation component of the PASAT may
potentially confound interpretation of test perfor-
mance in patients with premorbid calculation
difﬁculties. Some patients may ‘‘chunk’’ stimulus
items as they perform the task, especially during
the faster paced trials, which alters the PASAT’s
working memory demands (Fisk & Archibald,
2001). Furthermor e, clinical experience with the
PASAT suggests that it is perceived as challen-
ging by most patients and as unpleasant by some.
The SDMT is better received, but SDMT perfor-
mance can also be contaminated by difﬁculties
with visual acuity or visual scanning. The ideal
measure of processing speed/working memory
would be sensitive to subtle defects and posses
the psychometric strengths of the PASAT and the
Three promising experimental paradigms have
been used to assess working memory in MS:
(a) Baddeley and Hitch’s dual task paradigm
(Baddeley & Hitch, 1974) applied to MS patients
by D’Esposito and colleagues (D’Esposito et al.,
1996); (b) the Keeping Track Task (Salthouse,
1992; Salthouse, Babcock, & Shaw, 1991) which
assesses two related components of working
memory; and (c) the n-back procedure, as applied
in MS (Wishart et al., 2001). Dual task ex-
periments typically require patients to perform
primary and secondary tasks ﬁrst alone, and then
together, to evaluate decrements in performance
associated with allocation of attention to two
tasks. While this methodology may be sensitive
to cognitive dysfunction in MS, it has not been
well standardized. The Salthouse procedure is
sensitive to subtle deﬁcits in MS patients that
are not evident on standard clinical NP measures
(Archibald & Fisk, 2000). However, like the
PASAT, it is confounded by calculation ability.
The n-back procedure may be more promising to
develop as a clinical measure. In this task, sub-
jects are asked simply to report whether the
current stimulus is the same as that presented
‘‘n’’ (i.e., 1, 2, or 3) items previously. Like the
PASAT, the n-back procedure requires temporary
storage, rehearsal, and updating of information
held in working memory. Unlike the PASAT or
the SDMT, however, other cognitive operations
such as calculation and visual scanning are not
involved. Stimuli can potentially be presented
either aurally or visually, and response alterna-
tives can be simpliﬁed to a dichotomous yes/no or
go/no-go response. Computerized versions of the
n-back paradigm have been employed widely in
functional neuroimaging studies (Botvinick,
Nystrom, Fissell, Carter, & Cohen, 1999; Braver
et al., 1997; Cohen et al., 1997) demonstrating
modality-speciﬁc lateralized activation of anterior
cortical regions (Smith, Jonides, & Koeppe,
1996). In a recent pilot study of MS patients,
frontal cortex activity during an n-back procedure
was related to total lesion volume (Wishart et al.,
2001). To be widely applicable clinically, how-
ever, the n-back paradigm must have standardized
administration procedures and stimuli (preferably
in a ‘‘low tech’’ version), its psychometric proper-
ties must be documented, its sensitivity to MS-
associated impairment must be compared with
that of established measures (i.e., PASAT and
SDMT), and normative data must be collected.
Assessment of verbal ﬂuency also needs to be
developed further in order to achieve the expert
panel’s goal of ‘‘translatability’’ of the MAC-
FIMS. Although phonemic ﬂuency tests, which
require subjects to generate words rapidly based
on their lexical attributes (e.g., COWAT) are
sensitive to MS-related cognitive impairment
(Pozzilli et al., 1991; Rao, Leo, Bernardin, et al.,
392 RALPH H.B. BENEDICT ET AL.
1991) and are balanced for word frequency in
English, they are not necessarily equivalent across
languages. One solution might be to develop a
new phonemic ﬂuency test using letters that yield
high word output across different languages and
countries. A different approach is to shift to a
semantic ﬂuency measure with categories that
have a large number of exemplars in many or all
languages, thereby avoiding ﬂoor effects. In an
investigation of the COWAT and two semantic
ﬂuency tasks (animals and parts of body) in a U.S.
sample of 203 MS patients and 87 healthy con-
trols (Beatty, 2001), receiver operating charac-
teristic (ROC) curve analyses indicated that
sensitivity and speciﬁcity values for discriminat-
ing patients from controls were quite similar for
all three ﬂuency measures, particularly when
globally impaired patients were excluded from
the analysis. This ﬁnding, if replicated in other
countries, could lead to normative data collection
on ﬂuency measures that are truly equivalent
across languages and cultures, at least in mildly
to moderately impaired patients. To be sure, even
tests of functions other than verbal abilities may
have different psycho metric properties when
applied to patients with different languages and
cultures. Demonstration of the ‘translatability’ of
the MACFIMS must await collection of norma-
tive data from patients differing in language and
In conclusion, cognitive impairment in MS is
common, functionally disabling, and difﬁcult to
detect without formal NP assessment. In this
paper, we propose a practical, rationally derived,
clinically oriented minimal NP battery that is
based upon expert consensus regarding the
cognitive functi ons that are most important to
assess in MS. It is ﬂexible enough to accommo-
date improvements in its component measures
and to allow substitution of new tests if they are
psychometrically superior to current tests. The
MACFIMS can be applied routinely with MS
patients to detect deﬁcits in core cognitive
domains, particularly subtle deﬁcits in patients
who may beneﬁt from early treatment, and to
monitor changes in cognitive function over time.
Thus, it ﬁlls a crucial gap between brief NP
screening measures used to identify patients in
need of further testing and comprehensive NP
evaluations that are often impractical for routine
This project was supported by the Consortium of
Multiple Sclerosis Centers (CMSC) through an unrest-
ricted educational grant from Biogen, Inc. The authors
thank June Halper, MSN, ANP, FAAN, Tina Trott, and
Kelly Sallinger for their assistance in organizing the
panel and meeting.
Amato, M.P., Ponziani, G., Pracucci, G., Bracco, L.,
et al. (1995). Cognitive impairment in early-onset
multiple sclerosis: Pattern, predictors, and impact
on everyday life in a 4-year follow-up. Archives of
Neurology, 52, 168–172.
Archibald, C.J., & Fisk, J.D. (2000). Information
processing efﬁciency in patients with multiple
sclerosis. Journal of Clinical and Experimental
Neuropsychology, 22, 686–701.
Arnett, P.A., Higginson, C.I., Voss, W.D., Bender, W.I.,
Wurst, J.M., & Tippin, J.M. (1999). Depression in
multiple–sclerosis: Relationship to working mem-
ory capacity. Neuropsychology, 13, 546–556.
Arnett, P.A., Higginson, C.I., Voss, W.D., Wright, B.,
Bender, W.I., & Wurst, J.M. (1999). Depressed
mood in multiple sclerosis: Relationship to capa-
city-demanding memory and attentional function-
ing. Neuropsychology, 13, 434–446.
Baddeley, A., & Hitch, G. (1974). Working memory.
In G.A. Bower (Ed.), Recent advances in learning
and motivation (pp. 47–90). New York: Academic
Barona, A., Reynolds, C.R., & Chastain, R. (1984). A
demographically based index of pre-morbid intelli-
gence for the WAIS–R. Journal of Consulting and
Clinical Psychology, 52, 885–887.
Basso, M.R., Beason-Hazen, S., Lynn, J., Rammohan,
K., & Bornstein, R.A. (1996). Screening for
cognitive dysfunction in multiple sclerosis. Archives
of Neurology, 53, 980–984.
Beatty, W.W. (1995). Verbal abstraction deﬁcit in
multiple sclerosis. Neuropsychology, 9, 198–205.
Beatty, W.W. (2001). Verbal ﬂuency in multiple
sclerosis: Which measure is best? Multiple Sclero-
Beatty, W.W., Blanco, C.R., Wilbanks, S.L., & Paul,
R.H. (1995). Demographic, clinical, and cognitive
characteristics of multiple sclerosis patients who
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 393
continue to work. Journal of Neurologic Rehabilita-
tion, 9, 167–173.
Beatty, W.W., Goodkin, D.E., Hertsgaard, D., &
Monson, N. (1990). Clinical and demographic
predictors of cognitive performance in multiple
sclerosis: Do diagnostic type, disease duration, and
disability matter? Archives of Neurology, 47,
Beatty, W.W., Goodkin, D.E., Monson, N., & Beatty,
P.A. (1989). Cognitive disturbances in patients with
relapsing remitting multiple sclerosis. Archives of
Neurology, 46, 1113–1119.
Beatty, W.W., Goodkin, D.E., Monson, N., Beatty, P.A.,
& Hertsgaard, D. (1988). Anterograde and retro-
grade amnesia in patients with chronic progressive
multiple sclerosis. Archives of Neurology, 45,
Beatty, W.W., & Monson, N. (1994). Picture and motor
sequencing in multiple sclerosis. Journal of Clinical
and Experimental Neuropsychology, 16, 165–172.
Beatty, W.W., & Monson, N. (1996). Problem solving
by patients with multiple sclerosis: Comparison of
performance on the Wisconsin and California Card
Sorting Test. Journal of the International Neurop-
sychological Society, 2 , 134–140.
Beatty, W.W., Paul, R.H., Wilbanks, S.L., Hames, K.A.,
Blanco, C.R., & Goodkin, D.E. (1995). Identifying
multiple sclerosis patients with mild or global
cognitive impairment using the Screening Exam-
ination for Cognitive Impairment (SEFCI). Neurol-
ogy, 45, 718–723.
Beck, A.T., Steer, R.A., & Brown, G.K. (2000). BDI-
Fast Screen for Medical Patients: Manual. San
Antonio, TX: Psychological Corporation.
Benedict, R.H.B. (1997). Brief Visuospatial Memory
Test – Revised: Professional manual. Odessa,
Florida: Psychological Assessment Resources.
Benedict, R.H.B., Priore, R.L., Miller, C., Munschauer,
F.E., & Jacobs, L.D. (2001). Personality disorder in
MS correlates with cognitive impairment. Journal of
Neuropsychiatry and Clinical Neurosciences, 13,
Benedict, R.H.B., Schretlen, D., Groninger, L.,
Dobraski, M., & Shpritz, B. (1996). Revision of
the Brief Visuospatial Memory Test: Studies of
normal performance, reliability, and validity. Psy-
chological Assessment, 8, 145–153.
Benedict, R.H.B., & Zgaljardic, D.J. (1998). Practice
effects during repeated administrations of memory
tests with and without alternate forms. Journal of
Clinical and Experimental Neuropsychology, 20,
Benton, A.L., & Hamsher, K. (1989). Multilingual
Aphasia Examination. Iowa City: AJA Associates.
Benton, A.L., Sivan, A.B., Hamsher, K., Varney,
N.R., & Spreen, O. (1994). Contributions to
neuropsychological assessment (2nd ed.). New York:
Oxford University Press.
Bever, C., Grattan, L., Panitch, H.S., & Johnson, K.P.
(1995). The Brief Repeatable Battery of Neuropsy-
chological Tests for multiple sclerosis: A prelimin-
ary serial study. Multiple Sclerosis, 1, 165–169.
Blair, J.R., & Spreen, O. (1989). Predicting premorbid
IQ: A revision of the National Adult Reading Test.
Clinical Neuropsychologist, 3, 129–136.
Boringa, J.B., Lazeron, R.H.C., Reuling, I.E.W., Ader,
H.J., Pfennings, L., Lindeboom, J., de Sonneville,
L.M., Kalkers, N.F., & Polman, C.H. (2001). The
Brief Repeatable Battery of Neuropsychological
Tests: Normative values allow application in multi-
ple sclerosis clinical practice. Multiple Sclerosis, 7,
Botvinick, M., Nystrom, L.E., Fissell, K., Carter, C.S.,
& Cohen, J.D. (1999). Conﬂict monitoring versus
selection-for-action in anterior cingulate cortex.
Nature, 402, 179–181.
Braver, T.S., Cohen, J.D., Nystrom, L.E., Jonides, J.,
Smith, E.E., & Noll, D.C. (1997). A parametric
study of prefrontal cortex involvement in human
working memory. Neuroimage, 5, 49–62.
Camp, S.J., Stevenson, V.L., Thompson, A.J., Miller,
D.H., Borras, C., Auriacombe, S., Brochet, B.,
Falautano, M., Filippi, M., Herisse-Dulo, L.,
Montalban, X., Parrcira, E., Polman, C.H., De Sa,
J., & Langdon, D.W. (1999). Cognitive function in
primary progressive and transitional progressive
multiple sclerosis: A controlled study with MRI
correlates. Brain, 122, 1341–1348.
Cohen, J.D., Perlstein, W.M., Braver, T.S., Nystrom,
L.E., Noll, D.C., Jonides, J., & Smith, E.E. (1997).
Temporal dynamics of brain activation during a
working memory task [see comments]. Nature, 386,
Cutter, G.R., Baier, M.L., Rudick, R.A., Cookfair, D.L.,
Fischer, J.S., Petkau, J., Syndulko, K., Weinshenker,
B.G., Antel, J.P., Confavreux, C., Ellison, G.W.,
Lublin, F., Miller, A.E., Rao, S.M., Reingold, S.,
Thompson, A., & Wiloughby, E. (1999). Develop-
ment of a multiple sclerosis functional composite as
a clinical trial outcome measure. Brain, 122,
Delis, D.C., Kaplan, E., & Kramer, J.H. (2001).
California Sorting Test. Psychological Corporation.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A.
(1987). California Verbal Learning Test: Adult
version. San Antonio: Psychological Corporation.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A.
(2000). California Verbal Learning Test manual:
Second edition, Adult version. San Antonio, TX:
DeLuca, J., Johnson, S.K., & Natelson, B.H. (1993).
Information processing efﬁciency in chronic fatigue
394 RALPH H.B. BENEDICT ET AL.
syndrome and multiple sclerosis. Archives of
Neurology, 50, 301–304.
Demaree, H.A., DeLuca, J., Gaudino, E.A., &
Diamond, B.J. (1999). Speed of information proces-
sing as a key deﬁcit in multiple sclerosis: Implica-
tions for rehabilitation. Journal of Neurology,
Neurosurgery, and Psychiatry, 67, 661–663.
D’Esposito, M., Onishi, K., Thompson, H., Robinson,
K., Armstrong, C., & Grossman, M. (1996). Work-
ing memory impairments in multiple sclerosis:
Evidence from a dual-task paradigm. Neuropsychol-
ogy, 10, 51–56.
Fischer, J.S. (1988). Using the Wechsler Memory
Scale – Revised to detect and characterize memory
deﬁcits in multiple sclerosis. Clinical Neuropsy-
chologist, 2, 149–172.
Fischer, J.S. (1989). Objective memory testing in
multiple sclerosis. In K. Jensen, L. Knudsen, E.
Stenager, & I. Grant (Eds.), Mental disorders and
cognitive deﬁcits in multiple sclerosis (pp. 39–49).
London: John Libbey.
Fischer, J.S. (1999). Assessment of neuropsychological
function. In R.A. Rudick & D.E. Goodkin (Eds.),
Multiple sclerosis therapeutics (pp. 31–47).
London: Martin Dunitz.
Fischer, J.S. (2001). Cognitive impairment in multi-
ple sclerosis. In S.D. Cook (Ed.), Handbook of
multiple sclerosis (pp. 233–255). New York: Marcel
Fischer, J.S., Foley, F.W., Aikens, J.E., Ericson, G.D.,
Rao, S.M., & Shindell, S. (1994). What do we really
know about cognitive dysfunction, affective dis-
orders, and stress in multiple slcerosis? A
practitioner’s guide. Journal of Neurologic Rehabi-
litation, 8, 151–164.
Fischer, J.S., LaRocca, N.G., Miller, D.M., Ritvo, P.G.,
Andrews, H., & Paty, D. (1999). Recent develop-
ments in the assessment of quality of life in multiple
sclerosis (MS). Multiple Sclerosis, 5, 251–259.
Fischer, J.S., Priore, R.L., Jacobs, L.D., Cookfair, D.L.,
Rudick, R.A., Herndon, R.M., Richert, J.R., Salazar,
A.M., Goodkin, D.E., Granger, C.V., Simon, J.H.,
Grafman, J.H., Lezak, M.D., O’Reilly Hovey, K.M.,
Perkins, K.K., Barilla-Clark, D., Schacter, M.,
Shucard, D.W., Davidson, A.L., Wende, K.E.,
Bourdette, D.N., & Kooijmans-Coutinho, M.F.
(2000). Neuropsychological effects of interferon
beta-1a in relapsing multiple sclerosis. Multiple
Sclerosis Collaborative Research Group . Annals of
Neurology, 48, 885–892.
Fischer, J.S., Rudick, R.A., Cutter, G.R., & Reingold,
S.C. (1999). The Multiple Sclerosis Functional
Composite Measure (MSFC): An integrated
approach to MS clinical outcome assessment.
National MS Society Clinical Outcomes Assess-
ment Task Force. Multiple Sclerosis, 5, 244–250.
Fisk, J.D., & Archibald, C.J. (2001). Limitations of the
Paced Auditory Serial Addition Test as a measure of
working memory in patients with multiple sclerosis.
Journal of the International Neuropsychological
Society, 7, 363–372.
Fisk, J.D., Pontefract, A., Ritvo, P.G., Archibald, C.J.,
& Murray, R.J. (1994). The impact of fatigue on
patients with multiple sclerosis. Canadian Journal
of Neurological Science, 21, 9–14.
Foong, J., Rozewicz, L., Davie, C.A., Thompson, A.J.,
Miller, D.H., & Ron, M.A. (1999). Correlates of
executive function in multiple sclerosis: The use
of magnetic resonance spectroscopy as an index of
focal pathology. Journal of Neuropsychiatry and
Clinical Neurosciences, 11, 45–50.
Foong, J., Rozewicz, L., Quaghebeur, G., Davie, C.A.,
Kartsounis, L.D., Thompson, A.J., Miller, D.H., &
Ron, M.A. (1997). Executive function in multiple
sclerosis. The role of frontal lobe pathology. Brain,
Franklin, G.M., Heaton, R.K., Nelson, L.M., Filley,
C.M., & Seibert, C. (1988). Correlation of neuro-
psychological and MRI ﬁndings in chronic/
progressive multiple sclerosis. Neurology, 38,
Friend, K.B., & Grattan, L. (2000). Use of the North
American Adult Reading Test to estimate premorbid
intellectual function in patients with multiple
sclerosis. Journal of Clinical and Experimental
Neuropsychology, 20, 846–851.
Grant, I., McDonald, W.I., Trimble, M.R., Smith, E., &
Reed, R. (1984). Deﬁcient learning and memory in
early and middle phases of multiple sclerosis.
Journal of Neurology, Neurosurgery, and Psychia-
try, 47, 250–255.
Gronwall, D.M.A. (1977). Paced auditory serial addi-
tion task: A measure of recovery from concussion.
Perceptual and Motor Skills, 44, 367–373.
Heaton, R.K. (1985). Neuropsychological ﬁndings in
relapsing-remitting and chronic-progressive multi-
ple sclerosis. Journal of Consulting and Clinical
Psychology, 53, 103–110.
Higginson, C.I., Arnett, P.A., & Voss, W.D. (2000). The
ecological validity of clinical tests of memory and
attention in multiple sclerosis. Archives of Clinical
Neuropsychology, 15, 185–204.
Johnson, S.K., Lange, G., DeLuca, J., Korn, L.R., &
Natelson, B.H. (1997). The effects of fatigue on
neuropsychological performance in patients with
chronic fatigue syndrome, multiple sclerosis,
and depression. Applied Neuropsychology, 4,
Kent, R.D., Kent, J.F., & Rosenbek, J.C. (1987).
Maximum performance tests of speech production.
Journal of Speech and Hearing Disorders, 52,
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 395
Krupp, L.B. (2000). Fatigue and declines in cognitive
functioning in multiple sclerosis. Neurology, 55,
Kujala, P. (1994). Automatic and controlled informa-
tion processing in multiple sclerosis. Brain, 117,
Langdon, D.W., & Thompson, A.J. (1999). Multiple
sclerosis: A preliminary study of selected variables
affecting rehabilitation outcome. Multiple Sclerosis,
Lezak, M. (1995). Neuropsychological assessment (2nd
ed.). New York: Oxford University Press.
Litvan, I., Grafman, J., Vendrell, P., & Martinez, J.
(1988a). Multiple memory deﬁcits in patients with
multiple sclerosis: Exploring the working memory
system. Archives of Neurology, 45, 607–610.
Litvan, I., Grafman, J., Vendrell, P., & Martinez, J.
(1988b). Slowed information processing in multiple
sclerosis. Archives of Neurology, 45, 281–285.
McDonald, W.I., Compston, A., Edan, G., Goodkin,
D.E., Hartung, H., Lublin, F., McFarland, H.F., Paty,
D.W., Polman, C.H., Reingold, S.C., Sandberg-
Wollheim, M., Sibley, W.A., Thompson, A., van der
Noort, S., Weinshenker, B.Y., & Wolinsky, J.S.
(2001). Recommended diagnostic criteria for multi-
ple sclerosis: Guidelines from the international
panel on the diagnosis of multiple sclerosis. Annals
of Neurology, 50, 121–127.
Minden, S.L. (1990). Memory impairment in multiple
sclerosis. Journal of Clinical and Experimental
Neuropsychology, 12, 566–586.
Mohr, D.C., & Goodkin, D.E. (1999). Treatment of
depression in multiple sclerosis: Review and meta-
analysis. Clinical Psychology: Science and Prac-
tice, 6, 1–9.
National Multiple Sclerosis Society Cognitive Func-
tions Study Group. (1990). A manual for the Brief
Repeatable Battery of Neuropsychological Tests in
MS. New York: National Multiple Sclerosis Society.
Nyenhuis, D.L., Rao, S.M., Zajecka, J.M., Luchetta, T.,
Bernardin, L., & Garron, D.C. (1995). Mood
disturbance versus other symptoms of depression
in multiple sclerosis. Journal of the International
Neuropsychological Society, 1, 291–296.
Peyser, J.M., Edwards, K.R., Poser, C.M., & Filskov,
S.B. (1980). Cognitive function in patients with
multiple sclerosis. Archives of Neurology, 37,
Peyser, J.M., Rao, S.M., LaRocca, N.G., & Kaplan, E.
(1990). Guidelines for neuropsychological research
in multiple sclerosis. Archives of Neurology, 47,
Poser, C.M., Paty, D.W., Scheinberg, L., McDonald,
W.I., Davis, F.A., Ebers, G.C., Johnson, K.P., Sibley,
W.A., Silberberg, D.H., & Tourtellotte, W.W.
(1983). New diagnostic criteria for multiple
sclerosis: Guidelines for research protocols. Annals
of Neurology, 13, 655–661.
Pozzilli, C., Passaﬁume, D., Bernardi, S., Pantano, P.,
Incoccia, C., Bastianello, S., Bozzao, L., Lenzi,
G.L., & Fieschi, C. (1991). SPECT, MRI and
cognitive functions in multiple sclerosis. Journal
of Neurology, Neurosurgery, and Psychiatry, 54,
Rao, S.M. (1989). On the nature of memory disturbance
in multiple sclerosis. Journal of Clinical and
Experimental Neuropsychology, 11, 699–712.
Rao, S.M., Hammeke, T.A., McQuillen, M.P., Khatri,
B.O., & Lloyd, D. (1984). Memory disturbance in
chronic progressive multiple sclerosis. Archives of
Neurology, 41, 625–631.
Rao, S.M., Leo, G.J., Bernardin, L., & Unverzagt, F.
(1991). Cognitive dysfunction in multiple sclerosis
I. Frequency, patterns, and prediction. Neurology,
Rao, S.M., Leo, G.J., Ellington, L., Nauertz, T.,
Bernardin, L., & Unveragt, F. (1991). Cognitive
dysfunction in multiple sclerosis II. Impact on
employment and social functioning. Neurology, 41,
Rao, S.M., Leo, G.J., Haughton, V.M., Aubin-Faubert,
P.S., & Bernardin, L. (1989). Correlation of
magnetic resonance imaging with neuropsycholog-
ical testing in multiple sclerosis. Neurology, 39,
Rudick, R., Antel, J., Confavreux, C., et al. (1996).
Clinical outcomes assessment in multiple sclerosis.
Annals of Neurology, 40, 469–479.
Ryan, L., Clark, C.M., Klonoff, H., Li, D., & Paty, D.
(1996). Patterns of cognitive impairment in relap-
sing-remitting multiple sclerosis and their relation-
ship to neuropathology on magnetic resonance
images. Neuropsychology, 10, 176–193.
Salthouse, T.A. (1992). Inﬂuence of processing speed
on adult age differences in working memory. Acta
Psychologica, 79, 155–170.
Salthouse, T.A., Babcock, R.L., & Shaw, R.J. (1991).
Effects of adult age on structural and operational
capacities in working memory. Psychology and
Aging, 6, 118–127.
Scarrabelotti, M., & Carroll, M. (1998). Awareness of
remembering achieved through automatic and con-
scious processes in multiple sclerosis. Brain and
Cognition, 38, 183–201.
Smith, A. (1982). Symbol Digit Modalities Test:
Manual. Los Angeles: Western Psychological
Smith, E.E., Jonides, J., & Koeppe, R.A. (1996).
Dissociating verbal and spatial working memory
using PET [published erratum appears in Cereb
Cortex 1998 Dec;8(8):262]. Cerebral Cortex, 6,
396 RALPH H.B. BENEDICT ET AL.
Spreen, O., & Strauss, E. (1998). A compendium of
neuropsychological tests: Administration, norms,
and commentary (2nd ed.). New York: Oxford
Troster, A.I., Fields, J.A., Testa, J.A., Blanco, C.R.,
Hames, K.A., Salmon, D.P., & Beatty, W.W. (1998).
Cortical and subcortical inﬂuences on clustering and
switching in the performance of verbal ﬂuency
tasks. Neuropsychologia, 36, 295–304.
Troyer, A.K. (2000). Normative data for clustering and
switching on verbal ﬂuency tasks. Journal of
Clinical and Experimental Neuropsychology, 22,
Troyer, A.K., Graves, R.E., & Cullum, C.M. (1994).
Executive functioning as a mediator of the relation-
ship between age and episodic memory in healthy
aging. Aging Neuropsychology and Cognition, 1,
Van den Burg, W., Van Zomeron, A.H., Minderhoud,
J.M., Prange, A.J.A., & Meijer, N.S.A. (1987).
Cognitive impairment in patients with multiple
sclerosis and mild physical disability. Archives of
Neurology, 44, 494–501.
Vleugels, L., Lafosse, C., van Nunen, A., Nachtergaele,
S., Ketelaer, P., Charlier, M., & Vandenbussche, E.
(2000). Visuoperceptual impairment in multiple
sclerosis patients diagnosed with neuropsychologi-
cal tasks. Multiple Sclerosis, 6, 241–254.
Wechsler, D. (1997). Wechsler Adult Intelligence Scale
(3rd ed.). San Antonio: Psychological Corporation.
Wilkinson, G.S. (1993). Wide Range Achievement Test
administration manual: 1993 edition. Wilmington:
Wishart, H., Saykin, A.J., Mamourian, A., Flashman,
L.A., Ford, J.C., Fadul, C., Ryan, K., & Kasper, L.
(2001). Working memory in multiple sclerosis:
Relation of fMRI activation to lesion volume.
NeuroImage, 13, S762–S762.
Woodard, J.L., Benedict, R.H.B., Roberts, V.J.,
Goldstein, F.C., Kinner, K.M., Capruso, D.X., &
Clark, A.N. (1996). The reliability and validity of
alternate short forms for the Benton Judgement of
Line Orientation Test. Journal of Clinical and
Experimental Neuropsychology, 18, 898–904.
Woodard, J.L., Benedict, R.H.B., Salthouse, T.A., Toth,
J.P., Zgaljardic, D.J., & Hancock, H.E. (1998).
Normative data for Odd and Even Parallel Forms of
the Judgement of Line Orientation Test. Journal of
Clinical and Experimental Neuropsychology, 20,
MINIMAL NEUROPSYCHOLOGICAL ASSESSMENT OF MS PATIENTS 397