Sample Size Requirements for Training to a κ Agreement Criterion on Clinical Dementia Ratings

Department of Neurology, Georgetown University School of Medicine, Washington, DC 20057, USA.
Alzheimer disease and associated disorders (Impact Factor: 2.44). 07/2010; 24(3). DOI: 10.1097/WAD.0b013e3181d489c6

ABSTRACT The Clinical Dementia Rating (CDR) is a valid and reliable global measure of dementia severity. Diagnosis and transition across stages hinge on its consistent administration. Reports of CDR ratings reliability have been based on 1 or 2 test cases at each severity level; agreement (kappa) statistics based on so few rated cases have large error, and confidence intervals are incorrect. Simulations varied the number of test cases, and their distribution across CDR stage; to derive the sample size yielding a 95% confidence that estimated is at least 0.60. We found that testing raters on 5 or more patients per CDR level (total N=25) will yield the desired confidence in estimated kappa, and if the test involves greater representation of CDR stages that are harder to evaluate, at least 42 ratings are needed. Testing newly trained raters with at least 5 patients per CDR stage will provide valid estimation of rater consistency, given the point estimate for kappa is roughly 0.80; fewer test cases increases the standard error and unequal distribution of test cases across CDR stages will lower kappa and increase error.

Download full-text


Available from: Futoshi Yumoto, Apr 14, 2014
1 Follower
12 Reads
  • Source
    • "Investigators and research assistants may have limited opportunity to administer the examinations under supervised circumstances. Reliability studies of the Clinical Dementia Rating show that at least five administrations of the tool at each severity level are required to achieve interrater reliability [38]. Trial investigators are not trained to this level of reliability, and data collected exhibit large standard deviations and confidence intervals. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Alzheimer's disease (AD) therapies are increasingly being tested in global clinical trials. A search of revealed that of 269 currently active trials, 28% are currently being conducted in the United States; the majority of trials and the majority of trial sites are ex-US. The US has the largest number of trial sites of any single country; cumulatively, nearly half of all sites are outside the US. The US conducts more trials in all phases of drug development but has a greater proportion of phase 3 trials. The increasing importance of global participants in clinical trials emphasizes the importance of considering the ethnic and international factors that may influence trial outcome. The International Conference on Harmonization guidelines divide ethnic factors that may affect drug development into intrinsic and extrinsic influences. These include language, cultural factors, educational levels, the general level of health and standard of care, as well as nutrition and diet. Ethnic influences on pharmacokinetics are known for some metabolic pathways. The biology of AD may also differ among the world's populations. The frequency of the apolipoprotein e4 allele, a major risk factor for AD, differs internationally. Genetic variations might also affect inflammatory, excitotoxic, and oxidative components of AD. Diagnostic standards and experience vary from country to country. Levels of practitioner training and experience, diagnostic approaches to AD, and attitudes regarding aging and AD may differ. Experience and sophistication with regard to clinical trial conduct also vary within and between countries. Experience with conducting the necessary examinations, as well as the linguistic and cultural validity of instrument translations, may affect trial outcomes. Operational and regulatory aspects of clinical trials vary and provide important barriers to seamless conduct of multiregional clinical trials. Collection and testing of biological samples, continuous provision of drug substance, and protection of the integrity of supply lines may be difficult in some international circumstances. Attention to these potential influences on clinical trials will determine the success of global drug development programs and the utility of global trials for developing new AD therapeutics.
    Alzheimer's Research and Therapy 08/2011; 3(4):24. DOI:10.1186/alzrt86 · 3.98 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective To determine the relationship between functional status (independent activities of daily living) and judgment/problem solving and the extent to which select demographic characteristics such as dementia subtype and cognitive measures may moderate that relationship in older adult individuals with dementia.Methods The National Alzheimer's Coordinating Center Universal Data Set was accessed for a study sample of 3,855 individuals diagnosed with dementia. Primary variables included functional status, judgment/problem solving, and cognition.ResultsFunctional status was related to judgment/problem solving (r = 0.66; p < 0.0005). Functional status and cognition jointly predicted 56% of the variance in judgment/problem solving (R2 = 0.56, p < 0.0005). As cognition decreases, the prediction of poorer judgment/problem solving by functional status became stronger.Conclusions Among individuals with a diagnosis of dementia, declining functional status as well as declining cognition should raise concerns about judgment/problem solving.
    International Journal of Geriatric Psychiatry 05/2013; 28(5). DOI:10.1002/gps.3854 · 2.87 Impact Factor