Interobserver Agreement in the Assessment of Clinical Findings in Children With First Unprovoked Seizures

Columbia University College of Physicians and Surgeons, Morgan Stanley Children's Hospital of New York, 622 W 168th St, PH 137, New York, NY 10032, USA.
PEDIATRICS (Impact Factor: 5.47). 05/2011; 127(5):e1266-71. DOI: 10.1542/peds.2010-1752
Source: PubMed


Variables used in prediction rules and clinical guidelines should show acceptable agreement when assessed by different observers. Our objective was to determine the interobserver agreement of patient history and physical examination variables used to assess children undergoing emergency department (ED) evaluation for a first seizure not provoked by a known precipitant such as fever or trauma (ie, an unprovoked seizure).
We conducted a prospective cohort study of children aged 28 days to 18 years evaluated for unprovoked seizures at 6 tertiary care EDs. We excluded patients if previously evaluated for a similar event. Two clinicians independently completed a clinical assessment before neuroimaging. We determined agreement for each clinical variable by using the unweighted κ statistic.
A total of 217 paired observations were analyzed; median patient age was 53.5 months, and 38% were younger than 2 years. Agreement beyond chance was at least moderate (κ ≥ 0.41) for 21 of 31 (68%) variables for which κ could be calculated. κ was ≥0.41 for 7 of 11 (64%) general history variables, all 8 seizure-specific history variables (including seizure focality), and 6 of 12 (50%) physical examination variables. Agreement beyond chance was substantial or better (κ ≥ 0.61) for 2 of 11 (18%) general history variables, for 5 of 8 (63%) seizure-specific history variables, and for 2 of 12 (17%) physical examination variables.
For children with first unprovoked seizures evaluated in the ED, clinicians frequently assess findings from seizure-specific history with substantial agreement beyond chance. Those clinical variables that have been associated with the presence of intracranial abnormalities and show reliability between assessors, such as seizure focality and the presence of any focal neurological finding, may be more useful in the ED assessment of children with first unprovoked seizures.

Download full-text


Available from: James Callahan, Nov 18, 2015
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The "Rule of 7's," a Lyme meningitis clinical prediction rule, classifies children at low risk for Lyme meningitis when each of the following 3 criteria are met: <7 days of headache, <70% cerebrospinal fluid (CSF) mononuclear cells, and absence of seventh or other cranial nerve palsy. The goal of this study was to test the performance of the Rule of 7's in a multicenter cohort of children with CSF pleocytosis. We performed a retrospective cohort study of children evaluated at 1 of 3 emergency departments located in Lyme disease-endemic areas with CSF pleocytosis and Lyme serology obtained. Lyme meningitis was defined using the Centers for Disease Control and Prevention criteria (either positive Lyme serology test result or an erythema migrans [EM] rash). We calculated the performance of the Rule of 7's in our overall study population and in children without physician-documented EM. We identified 423 children, of whom 117 (28% [95% confidence interval (CI): 24%-32%]) had Lyme meningitis, 306 (72% [95% CI: 68%-76%]) had aseptic meningitis, and 0 (95% CI: 0%-1%) had bacterial meningitis. Of the 130 classified as low risk, 5 had Lyme meningitis (sensitivity, 112 of 117 [96% (95% CI: 90%-99%)]; specificity, 125 of 302 [41% (95% CI: 36%-47%)]). In the 390 children without EM, 3 of the 127 low-risk patients had Lyme meningitis (2% [95% CI: 0%-7%]). Patients classified as low risk by using the Rule of 7's were unlikely to have Lyme meningitis and could be managed as outpatients while awaiting results of Lyme serology tests.
    PEDIATRICS 12/2011; 129(1):e46-53. DOI:10.1542/peds.2011-1215 · 5.47 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The Bacterial Meningitis Score, a derived and validated clinical decision rule, identifies children with cerebrospinal fluid (CSF) pleocytosis who are at very low risk of bacterial meningitis. Low-risk features include the following: negative CSF Gram stain, CSF absolute neutrophil count (ANC) <1000 cells/μl, CSF protein <80 mg/dl, peripheral blood ANC <10 000 cells/μl and no seizure at or prior to initial presentation. The study objective of the present work was to calculate the performance of the Bacterial Meningitis Score by performing a meta-analysis of all published validation studies. A meta-analysis of all studies published between 2002 and 2012 was performed, evaluating the performance of the Bacterial Meningitis Score in children with CSF pleocytosis. Study quality was assessed using the assessment of diagnostic accuracy studies instrument and then the test performance of the prediction rule was calculated. From 8 studies, 5312 patients were identified, of whom 4896 (92%) had sufficient clinical data to calculate the Bacterial Meningitis Score. Bacterial meningitis was diagnosed in 1242 children (23% of study patients). The combined sensitivity of the Bacterial Meningitis Score for bacterial meningitis was 99.3% (1224/1233; 95% CI 98.7% to 99.7%), specificity 62.1% (2274/3663; 95% CI 60.5% to 63.7%) negative predictive value 99.7% (2274/2283, 95% CI 99.3% to 99.9%), positive likelihood ratio 2.6 (95% CI 2.5 to 2.7) and negative likelihood ratio 0.01 (95% CI 0.01 to 0.02). The Bacterial Meningitis Score is a highly accurate clinical scoring system that could be used to assist clinical decision making for the management of children with CSF pleocytosis.
    Archives of Disease in Childhood 07/2012; 97(9):799-805. DOI:10.1136/archdischild-2012-301798 · 2.90 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Objectives: The objective was to determine the interobserver agreement of historical and physical examination findings assessed during the emergency department (ED) evaluation of children with blunt abdominal trauma. Methods: This was a planned substudy of a multicenter, prospective cohort study of children younger than 18 years of age evaluated for blunt abdominal trauma. Patients were excluded if injury occurred more than 24 hours prior to evaluation or if computed tomography (CT) imaging was obtained at another hospital prior to transfer to a study site. Two clinicians independently recorded their clinical assessments of a convenience sample of patients onto data collection forms within 60 minutes of each other and prior to CT imaging (if obtained) or knowledge of laboratory results. The authors categorized variables as either subjective symptoms (i.e., patient history) or objective findings (i.e., physical examination). For each variable recorded by the two observers, the agreement beyond that expected by chance was estimated, using the kappa (κ) statistic for categorical variables and weighted κ for ordinal variables. Variables with 95% lower confidence limits (LCLs) κ ≥ 0.4 (moderate agreement or better) were considered to have acceptable agreement. Results: A total of 632 pairs of physician observations were obtained on 23 candidate variables. Acceptable agreement was achieved in 16 (70%) of the 23 variables tested. For six subjective symptoms, κ ranged from 0.48 (complaint of shortness of breath) to 0.90 (mechanism of injury), and only the complaint of shortness of breath had a 95% LCL κ < 0.4. For the 17 objective findings, κ ranged from -0.01 (pelvis instability) to 0.82 (seat belt sign present). The 95% LCL for κ was <0.4 for flank tenderness, abnormal chest auscultation, suspicion of alcohol or drug intoxication, pelvis instability, absence of bowel sounds, and peritoneal irritation. Conclusions: Observers can achieve at least acceptable agreement on the majority of historical and physical examination variables in children with blunt abdominal trauma evaluated in the ED. Those variables are candidates for consideration for development of a clinical prediction rule for intra-abdominal injury in children with blunt trauma.
    Academic Emergency Medicine 05/2013; 20(5). DOI:10.1111/acem.12132 · 2.01 Impact Factor
Show more