Investigation and Treatment of Missing Item Scores in Test and Questionnaire Data

Tilburg University, Tilburg, North Brabant, Netherlands
Multivariate Behavioral Research (Impact Factor: 2.48). 06/2010; 38(4). DOI: 10.1207/s15327906mbr3804_4


This article first discusses a statistical test for investigating whether or not the pattern of missing scores in a respondent-by-item data matrix is random. Since this is an asymptotic test, we investigate whether it is useful in small but realistic sample sizes. Then, we discuss two known simple imputation methods, person mean (PM) and two-way (TW) imputation, and we propose two new imputation methods, response-function (RF) and mean response-function (MRF) imputation. These methods are based on few assumptions about the data structure. An empirical data example with simulated missing item scores shows that the new method RF was superior to the methods PM, TW, and MRF in recovering from incomplete data several statistical properties of the original complete data. Methods TW and RF are useful both when item score missingness is ignorable and nonignorable.

Full-text preview

Available from:
  • Source
    • "As Roth, Switzer and Switzer (1999) pointed out, up until then, there was scarcely any literature on the problem of missing values at the item level. Since then, there have been numerous studies in this fi eld, though not always dealing directly with psychometric aspects (Bernaards & Sijtsma, 2005; Carpita & Manisera, 2011; Cuesta, Fonseca-Pedrero, Vallejo, & Muñiz, 2013; Enders, 2003, 2004; Fernández-Alonso, Suárez-Alvarez, & Muñiz, 2012; Gmel, 2001; McDonald, Thurston, & Nelson, 2000; Shrive, Stuart, Quan, & Ghali, 2006; Sijtsma & van der Ark, 2003; Van Ginkel, van der Ark, & Sijtsma, 2007a, 2007b). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: The problem of missing values at the item level is common in studies using educational and psychological tests. The aim of the present work is to explore how the estimation of reliability is affected by missing values. Method: Using real data, we simulated missing values in accordance with a "missing at random mechanism". Four factors were manipulated with the aim of checking their effect on the estimation of the reliability of the instrument: missing data mechanism, percentage of missing data in the database, sample size, and procedure employed for the treatment of missing values. Results: The results show that the quality of estimations depends on the interaction of various factors. The general tendency is that the estimations are worse when the sample size is small and the percentage of missing values increases. Listwise is the worst procedure for treatment of the missing data in the simulated conditions. Conclusions: It is concluded that with a small percentage of missing values one can obtain estimations that are acceptable from a practical point of view with all the procedures employed, except Listwise.
    Full-text · Article · Nov 2014 · Psicothema
  • Source
    • "The P-PDI is used for children ages 11 years and older, whereas parents reported on children's disability for children younger than 11 years of age. Up to 2 missing values (25%) were imputed using the 2-way imputation method [35]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Pediatric chronic pain, which can result in deleterious effects for the child, bears the risk of aggravation into adulthood. Intensive interdisciplinary pain treatment (IIPT) might be an effective treatment, given the advantage of consulting with multiple professionals on a daily basis. Evidence for the effectiveness of IIPT is scarce. Here, we investigated the efficacy of an IIPT within a randomized controlled trial by comparing an intervention group (IG) (n=52) to a waiting-list control group (WCG) (n=52). We made assessments before (PRE), immediately after treatment (POST), as well as at short-term (POST6MONTHS) and long-term (POST12MONTHS) follow-up. We determined a combined endpoint, "improvement" (pain intensity, disability, school absence) [16,21], and investigated three additional outcome domains (anxiety, depression, catastrophizing). We also investigated changes in economic parameters (healthcare use, parental work absenteeism, subjective financial burden) and their relationship to the child's improvement. Results at POST showed that significantly more children in the IG than in the WCG were assigned to improvement (55% compared to 14%; Fisher p<.001; 95% CI for incidence difference: 0.21%-0.60%). While immediate effects were achieved for disability, school absence, depression and catastrophizing, pain intensity and anxiety did not change until short-term follow-up. More than 60% of the children in both groups were improved long-term. The parents reported significant reductions in all economic parameters. The results from the present study support the efficacy of the IIPT. Future research is warranted to investigate differences in treatment response and to understand the changes in economic parameters in nonimproved children.
    Full-text · Article · Sep 2013 · Pain
  • Source
    • "The total score is computed by summing all items and ranges from 12 to 60 (Cronbach's alpha in this sample ¼ .85). The 2-way imputation method (Sijtsma &amp; van der Ark, 2003) wasused to impute missing values when at least 9 of 12 items were responded to. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Objective: To investigate the long-term effectiveness of a 3-week multimodal inpatient program for children and adolescents with chronic pain. Methods: 167 adolescents were evaluated at pretreatment baseline, 3-, and 12-month follow-up. Long-term effectiveness was investigated for pain-related variables (pain-related disability, school absence, pain intensity) and emotional distress. Results: We found statistically and clinically significant changes in all variables. After 1 year, the majority (56%) showed overall improvement as indexed by decreased pain-related disability or school absence. 22% had an unsuccessful treatment outcome. Those showing only short-term improvements had higher levels of emotional distress at baseline. Conclusions: 1 year after completing a multimodal inpatient program adolescents report less chronic pain, disability, and emotional distress. Clinically significant changes remain stable. Adolescents with high levels of emotional distress at admission may require special attention to maintain positive treatment outcomes. Specialized inpatient therapy is effective for children with chronic pain.
    Full-text · Article · Nov 2012 · Journal of Pediatric Psychology
Show more