Statistical considerations in a systematic review of proxy measures of clinical behaviour

Institute of Health and Society, Newcastle University, 21 Claremont Place, Newcastle upon Tyne, NE2 4AA, UK.
Implementation Science (Impact Factor: 3.47). 02/2010; 5:20. DOI: 10.1186/1748-5908-5-20
Source: PubMed

ABSTRACT Studies included in a related systematic review used a variety of statistical methods to summarise clinical behaviour and to compare proxy (or indirect) and direct (observed) methods of measuring it. The objective of the present review was to assess the validity of these statistical methods and make appropriate recommendations.
Electronic bibliographic databases were searched to identify studies meeting specified inclusion criteria. Potentially relevant studies were screened for inclusion independently by two reviewers. This was followed by systematic abstraction and categorization of statistical methods, as well as critical assessment of these methods.
Fifteen reports (of 11 studies) met the inclusion criteria. Thirteen analysed individual clinical actions separately and presented a variety of summary statistics: sensitivity was available in eight reports and specificity in six, but four reports treated different actions interchangeably. Seven reports combined several actions into summary measures of behaviour: five reports compared means on direct and proxy measures using analysis of variance or t-tests; four reported the Pearson correlation; none compared direct and proxy measures over the range of their values. Four reports comparing individual items used appropriate statistical methods, but reports that compared summary scores did not.
We recommend sensitivity and positive predictive value as statistics to assess agreement of direct and proxy measures of individual clinical actions. Summary measures should be reliable, repeatable, capture a single underlying aspect of behaviour, and map that construct onto a valid measurement scale. The relationship between the direct and proxy measures should be evaluated over the entire range of the direct measure and describe not only the mean of the proxy measure for any specific value of the direct measure, but also the range of variability of the proxy measure. The evidence about the relationship between direct and proxy methods of assessing clinical behaviour is weak.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Publication in the field of technical sciences Plagiarism is a problem with far-reaching consequences for the sciences. However, even today’s best software-based systems can only reliably identify copy&paste plagiarism. Disguised plagiarism forms, including paraphrased text, cross-language plagiarism, as well as structural and idea plagiarism often remain undetected. This weakness of current systems results in a large percentage of scientific plagiarism going undetected. Bela Gipp provides an overview of the state-of-the art in plagiarism detection and an analysis of why these approaches fail to detect disguised plagiarism forms. The author proposes Citation-based Plagiarism Detection to address this shortcoming. Unlike character-based approaches, this approach does not rely on text comparisons alone, but analyzes citation patterns within documents to form a language-independent "semantic fingerprint" for similarity assessment. The practicability of Citation-based Plagiarism Detection was proven by its capability to identify so-far non-machine detectable plagiarism in scientific publications. Contents Current state of plagiarism detection approaches and systems Citation-based Plagiarism Detection Target Groups Readers interested in the problem of plagiarism in the sciences Faculty and students from all disciplines, but especially computer science The Author Bela Gipp is a postdoctoral researcher at the University of California, Berkeley. Content Level » Research Keywords » Citation analysis - Citation-based Plagiarism Detection - Detection approaches - Plagiarism detection - Plagiarism forms
    01/2014: pages 354; Springer Vieweg Research., ISBN: 978-3-658-06393-1
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Background This paper aims to assess the sensitivity and specificity of exit interviews as a measure of malaria case management practice as compared to direct observation.Methods The malaria case management of 1654 febrile patients attending 110 health facilities from across Papua New Guinea was directly observed by a trained research officer as part of a repeat cross sectional survey. Patient recall of 5 forms of clinical advice and 5 forms of clinical action were then assessed at service exit and statistical analyses on matched observation/exit interview data conducted.ResultsThe sensitivity of exit interviews with respect to clinical advice ranged from 36.2% to 96.4% and specificity from 53.5% to 98.6%. With respect to clinical actions, sensitivity of the exit interviews ranged from 83.9% to 98.3% and specificity from 70.6% to 98.1%.Conclusion The exit interview appears to be a valid measure of objective malaria case management practices such as the completion of a diagnostic test or the provision of antimalarial medication, but may be a less valid measure of low frequency, subjective practices such as the provision of malaria prevention advice.
    BMC Health Services Research 12/2014; 14(1):628. DOI:10.1186/s12913-014-0628-8 · 1.66 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: To assess validity of a low-intensity measure of fitness (X) in a population of older adults as a proxy measure for the original, high-intensity measure (Y), we used ordinary least square regression with the new, potential proxy measure (X) as the sole explanatory variable for Y. A perfect proxy measure would be unbiased (i.e., result in a regression line with a y-intercept of zero and a slope of one) with no error (variance equal to zero). We evaluated the properties of potential biases of proxy measures. A two degree-of-freedom approach using a contrast matrix in the setting of simple linear ordinary least squares regression was compared to a one degree-of-freedom paired t test alternative approach. We found that substantial improvements in power could be gained through use of the two degree-of-freedom approach in many settings, while scenarios where no linear bias was present there could be modest gains from the paired t test approach. In general, the advantages of the two degree-of-freedom approach outweighed the benefits of the one degree-of-freedom approach. Using the two degree-of-freedom approach, we assessed the data from our motivating example and found that the low-intensity fitness measure was biased, and thus was not a good proxy for the original, high-intensity measure of fitness in older adults.

Full-text (3 Sources)

Available from
May 22, 2014