DSM-5 Field Trials in the United States and Canada, Part II: Test-Retest Reliability of Selected Categorical Diagnoses

University of Pittsburgh, Pittsburgh, Pennsylvania, United States
American Journal of Psychiatry (Impact Factor: 12.3). 10/2012; 170(1). DOI: 10.1176/appi.ajp.2012.12070999
Source: PubMed


The DSM-5 Field Trials were designed to obtain precise (standard error,0.1) estimates of the intraclass kappa asa measure of the degree to which two clinicians could independently agree on the presence or absence of selected DSM-5 diagnoses when the same patient was interviewed on separate occasions, in clinical settings, and evaluated with usual clinical interview methods.

Eleven academic centers in the United States and Canada were selected,and each was assigned several target diagnoses frequently treated in that setting.Consecutive patients visiting a site during the study were screened and stratified on the basis of DSM-IV diagnoses or symptomatic presentations. Patients were randomly assigned to two clinicians for a diagnostic interview; clinicians were blind to any previous diagnosis. All data were entered directly via an Internet-based software system to a secure central server. Detailed research design and statistical methods are presented in an accompanying article.

There were a total of 15 adult and eight child/adolescent diagnoses for which adequate sample sizes were obtained to report adequately precise estimates of the intraclass kappa. Overall, five diagnoses were in the very good range(kappa=0.60–0.79), nine in the good range(kappa=0.40–0.59), six in the questionable range (kappa = 0.20–0.39), and three in the unacceptable range (kappa values,0.20). Eight diagnoses had insufficient sample sizes to generate precise kappa estimates at any site.

Most diagnoses adequately tested had good to very good reliability with these representative clinical populations assessed with usual clinical interview methods. Some diagnoses that were revised to encompass a broader spectrum of symptom expression or had a more dimensional approach tested in the good to very good range.

1 Follower
25 Reads
  • Source
    • "• Will help individuals with these symptoms access suitable treatment adapted to their impairment. • Field trials from DSM-5 provided evidence of SCD, indicating that the lower ASD diagnoses in DSM-IV could be explained by a shift to the SCD diagnostic category [21]. • It is likely that individuals with social communication and/or pragmatic language disability were diagnosed as DSM-IV PDD-NOS. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Symptoms of inattention and hyperactivity, features of attention-deficit/hyperactivity disorder (ADHD), have been frequently documented in children with autism spec‐ trum disorders (ASDs) and often co-occur. Evidence indicates that 20-50% of children with ADHD meet criteria for ASD, and 30-80% of ASD children meet criteria for ADHD. According to the DSM-IV, the essential features of Autistic Disorder (AD) are " the presence of markedly abnormal or impaired development in social interaction and communication and a markedly restricted repertoire of activity and interests ". Differ‐ ential diagnosis of " Pervasive Developmental Disorder " (PDD: Autistic Disorder, Re‐ tt's Disorder, Childhood Disintegrative Disorder, Asperger's Disorder) and " Pervasive Developmental Disorder-Not Otherwise Specified " (PDD-NOS) is often difficult in the preschool child. This is particularly true when assessing verbal and nonverbal com‐ munication since both expressive and cognitive language are not yet established and there are many differences in their acquisition period among these children (before the age of three). As a result, many of these children are diagnosed with PDD-NOS not meeting the criteria for a specific type of PDD; this category includes "atypical au‐ tism" presentations that do not meet the criteria for AD. As a result, the concept of PDD-NOS has become a mixed bag. Often, diagnosis cannot be established before age three, delaying therapeutic interventions. Moreover, differential diagnosis between ADHD and PDD-NOS can be especially difficult, mainly in infant and young chil‐ dren. However, and following the recommendations of the DSM-IV, the ASDs diag‐ nosis has been included among the exclusion criteria for the ADHD. Such exclusion has generated considerable controversy regarding the necessity and benefits of main‐ taining these separations. At present, a new edition of the DSM has been published: DSM-5® (Fifth Edition, 2013). Among the advantages that this new manual provides are: i) further categoriza‐ tion of the persons affected and ii) the possibility of diagnosis before the age of three. The DSM-5 takes into account that limitations in language are not specific to autism. The new diagnostic category " Social Communication Disorder " appears separate from ASD, which does not seek to create a new subcategory. In light of the new DSM-V criteria which allow a dual diagnosis of ASD and ADHD behaviors, in this chapter we will review the clinical overlap of these two conditions, particularly regarding their comorbidities in community pre-schoolers (generally categorized as PDD-NOS). We will also look into possible future research directions necessary to enhance our understanding of the etiology/genetics factors as well as the appropriate sequence of therapeutic interventions and pharmacological treatment (psychostimulant and nonstimulant medications) for the co-occurrence of these disor‐ ders.
    ADHD - New Directions in Diagnosis and Treatment, Edited by Dr. Jill M. Norvilitis, 09/2015: chapter Chapter 6: pages 109-164; InTech- Open Access Publisher., ISBN: 978-953-51-2166-4
    • "would be a cause for concern as well. Although some have argued that reliability levels in this range are adequate for clinical care (Regier et al., 2013), we would argue that reliability in this low range is insufficient to facilitate the advancement of psychological research . It has long been argued that reliability sets an upper limit on validity (Nelson-Gray, 1991; Spitzer & Fleiss, 1974); however, the current test–retest analyses are arguably more strongly linked to diagnostic validity than are results obtained via audio/videorecording or joint-interview methods. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Diagnostic reliability is essential for the science and practice of psychology, in part because reliability is necessary for validity. Recently, the DSM-5 field trials documented lower diagnostic reliability than past field trials and the general research literature, resulting in substantial criticism of the DSM-5 diagnostic criteria. Rather than indicating specific problems with DSM-5, however, the field trials may have revealed long-standing diagnostic issues that have been hidden due to a reliance on audio/video recordings for estimating reliability. We estimated the reliability of DSM-IV diagnoses using both the standard audio-recording method and the test-retest method used in the DSM-5 field trials, in which different clinicians conduct separate interviews. Psychiatric patients (N = 339) were diagnosed using the SCID-I/P; 218 were diagnosed a second time by an independent interviewer. Diagnostic reliability using the audio-recording method (N = 49) was "good" to "excellent" (M κ = .80) and comparable to the DSM-IV field trials estimates. Reliability using the test-retest method (N = 218) was "poor" to "fair" (M κ = .47) and similar to DSM-5 field-trials' estimates. Despite low test-retest diagnostic reliability, self-reported symptoms were highly stable. Moreover, there was no association between change in self-report and change in diagnostic status. These results demonstrate the influence of method on estimates of diagnostic reliability. (PsycINFO Database Record (c) 2015 APA, all rights reserved).
    Journal of Abnormal Psychology 06/2015; DOI:10.1037/abn0000069 · 4.86 Impact Factor
  • Source
    • "People day-to-day perceptions of the global levels of violence are usually disconnected from the actual proportions and real prevalence. But this does not mean to say that we should be at ease with the steady violence decline per se, but we should remain engaged in trying to explain why the decline has occurred and commit ourselves to reduce even further the still unacceptably high current levels of lethal violence among certain population segments, low-income countries and regions, to the lowest possible levels, if not to their total eradication (Pinker 2011 ). This approach would hopefully lead us into a different and more fruitful direction . "
    Trauma and Migration: Cultural Factors in the Diagnosis and Treatment of Traumatised Immigrants, Edited by MeryamSchouler-Ocak, 06/2015: chapter 2: pages 9-31; Springer International Publishing Switzerland.
Show more


25 Reads
Available from