Commentary: selecting performance measures by consensus: an appropriate extension of the delphi method?

RAND, 1776 Main Street, P.O. Box 2138, Santa Monica, California 90407-2138, USA.
Psychiatric Services (Impact Factor: 1.99). 01/2006; 56(12):1583. DOI: 10.1176/
Source: PubMed
  • Source
    The Journal of Rehabilitation Research and Development 01/2007; 44(6). · 1.69 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: To identify components of a provisional clinical response index for thyroid eye disease using a modified Delphi technique. The International Thyroid Eye Disease Society conducted a structured, 3-round Delphi exercise establishing consensus for a core set of measures for clinical trials in thyroid eye disease. The steering committee discussed the results in a face-to-face meeting (nominal group technique) and evaluated each criterion with respect to its feasibility, reliability, redundancy, and validity. Redundant measures were consolidated or excluded. Criteria were parsed into 11 domains for the Delphi surveys. Eighty-four respondents participated in the Delphi 1 survey, providing 220 unique items. Ninety-two members (100% of the respondents from Delphi 1 plus 8 new participants) responded in Delphi 2 and rated the same 220 items. Sixty-four members (76% of participants) rated 153 criteria in Delphi 3 (67 criteria were excluded because of redundancy). Criteria with a mean greater than 6 (1 = least appropriate to 9 = most appropriate) were further evaluated by the nominal group technique and provisional core measures were chosen. Using a Delphi exercise, we developed provisional core measures for assessing disease activity and severity in clinical trials of therapies for thyroid eye disease. These measures will be iteratively refined for use in multicenter clinical trials.
    Archives of ophthalmology 09/2009; 127(9):1155-60. · 3.86 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The purpose of study was to identify a list of performance measures for schizophrenia treatment services and to assemble a multistakeholder group to reach consensus on a core list. The study was conducted in two stages: first, a systematic review of the literature was conducted to identify a comprehensive list of measures; second, a consensus-building technique, the Delphi process, was used with participants from six groups of stakeholders: schizophrenia experts, mental health clinicians, mental health administrators, the payer (the Alberta Ministry of Health and Wellness), patients, and family members. Thirty stakeholders participated in three rounds of self-completed questionnaires. The degree of consensus achieved in the Delphi process was defined as the semi-interquartile range for each measure. Ninety-seven measures were identified in the literature review. The Delphi method reduced the list to 36 measures rated as essential. The measures address eight domains of service-level evaluation: acceptability, accessibility, appropriateness, competence, continuity, effectiveness, efficiency, and safety. Despite the diversity in backgrounds of the stakeholder groups, the Delphi technique was effective in moving participants' ratings toward consensus through successive questionnaire rounds. The resulting measures reflected the interests of all stakeholders. Several further steps are required before these measures are implemented and include working toward reliability and validity of specific measures, assessing the feasibility and cost-effectiveness of collecting the data, and finally, undertaking risk adjustment for outcome measures.
    Psychiatric services (Washington, D.C.) 04/2012; 63(6):584-91. · 2.81 Impact Factor


Available from