Teaching critical appraisal skills in health care settings.

Health Care Research Unit South Academic Block, Southampton General Hospital, Tremona Road, Southampton, Hampshire, UK, SO16 6YD.
Cochrane database of systematic reviews (Online) (Impact Factor: 5.94). 02/2001; DOI: 10.1002/14651858.CD001270
Source: PubMed

ABSTRACT Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training.
To assess the effects of teaching critical appraisal skills to health professionals, on the process of care, patient outcomes and knowledge of health professionals.
We searched The Cochrane Library (to Issue 2 2000), MEDLINE (1966 to 1997), EMBASE (1980 to 1997), Eric (1966 to 1997), Cinahl (1982 to 1997), Lisa (1976 to 1997), Sigle (1980 to 1997), Science Citation Index (1981 to 1997), PsycLit (1974 to 1997), the world-wide-web, and reference lists of articles. We also contacted major medical education centres.
Randomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses of educational interventions teaching critical appraisal to health professionals. The outcomes were: process of care; patient mortality, quality of life, and satisfaction; and health professional knowledge/awareness based upon objective, standardised, validated instruments.
Two reviewers independently extracted data and three reviewers independently assessed study quality.
One USA hospital-based randomised trial was included involving 44 doctors. The outcome assessed was critical appraisal knowledge. Process of care, patient health or attitude/awareness outcomes were not assessed. Critical appraisal teaching was reported to have resulted in a 25% improvement (adjusted figure) in critical appraisal knowledge in the intervention group compared to a 6% improvement in the control group, which was statistically significant (p=0.02).
There is evidence that critical appraisal teaching has positive effects on participants' knowledge, but as only one study met the inclusion criteria the validity of drawing general conclusions about the effects of teaching critical appraisal is debatable. There are large gaps in the evidence as to whether teaching critical appraisal impacts on decision-making or patient outcomes. It is also unclear whether the size of benefit seen is large enough to be of practical significance, or whether this varies according to participant background or teaching method. The evidence supporting all outcomes is weakened by the generally poorly designed, executed and reported studies that we found.

  • [Show abstract] [Hide abstract]
    ABSTRACT: The purpose of this paper was to examine the perceptions of students of health sciences on research training programs offered at Saudi universities. Design/methodology/approach – A cross-sectional survey design was adopted to capture the perceptions of health science students about research training programs offered at selected Saudi universities. A criterion-based sampling technique was adopted, and accordingly, 630 students were selected to participate in the study. A pre-tested questionnaire tool titled, “Students Attitude towards Research (SAR) questionnaire ”, was used to capture student responses on a five-point Likert scale with respect to three dimensions, namely, extent of research activities offered in their college, involvement of faculty in research and the infrastructural facilities offered by the college for research. The students’ attitudes with respect to various components of research training programs were analyzed using mean and cumulative percentage of students satisfied with the training. Multivariate analysis of variance (MANOVA) was used to study whether there is any significant difference in attitudes among the students belonging to four health science programs of the seven selected universities. Findings – The results of this study demonstrate that only less than 50 per cent of the students from the selected health science colleges are satisfied with the existing research training programs. There are significant differences in the students’ attitude toward research training programs offered in seven selected Saudi universities. Originality/value – This is the first Saudi Arabia-based study that provides an alarming signal to Educational-policy planners on students’ perceptions and attitudes toward research training programs offered in Saudi higher education institutions.
    Quality Assurance in Education 01/2015; 23(2):196-210.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Background: A discipline which critically looks at the evidence for practice should itself be critically examined. Credible evidence for the effectiveness of training in evidence-based healthcare (EBHC) is essential. We attempted to summarise the current knowledge on evaluating the effectiveness of training in EBHC while identifying the gaps. Methods: A working group of EBHC teachers developed a conceptual framework of key areas of EBHC teaching and practice in need of evidence mapped to appropriate methods and outcomes. A literature search was conducted to review the current state of research in these key areas. Studies of training interventions that evaluated effectiveness by considering learner, patient or health system outcomes in terms of knowledge, skills, attitude, judgement, competence, decision-making, patient satisfaction, quality of life, clinical indicators or cost were included. There was no language restriction. Results: Of 55 articles reviewed, 15 met the inclusion criteria: six systematic reviews, three randomised controlled trials and six before–after studies. We found weak indications that undergraduate training in EBHC improves knowledge but not skills, and that clinically integrated postgraduate teaching improves both knowledge and skills. Two randomised controlled trials reported no impact on attitudes or behaviour. One before–after study found a positive impact on decision-making, while another suggested change in learners' behaviour and improved patient outcome. We found no studies assessing the impact of EBHC training on patient satisfaction, health-related quality of life, cost or population-level indicators of health. Conclusions: Literature evaluating the effectiveness of training in EBHC has focused on short-term acquisition of knowledge and skills. Evaluation designs were methodologically weak, controlled trials appeared inadequately powered and systematic reviews could not provide conclusive evidence owing to weakness of study designs.
    International Journal of Evidence-Based Healthcare 01/2007; 5(4):468-476. DOI:10.1097/01258363-200712000-00008
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Objective Physician participation in Continuing Medical Education programs may be influenced by a number of factors. To evaluate the factors associated with compliance with the Continuing Medical Education requirements at a private hospital, we investigated whether physicians' activity, measured by volumes of admissions and procedures, was associated with obtaining 40 Continuing Medical Education credits (40 hours of activities) in a 12-month cycle. Methods In an exclusive and non-mandatory Continuing Medical Education program, we collected physicians' numbers of hospital admissions and numbers of surgical procedures performed. We also analyzed data on physicians' time since graduation, age, and gender. Results A total of 3,809 credentialed, free-standing, private practice physicians were evaluated. Univariate analysis showed that the Continuing Medical Education requirements were more likely to be achieved by male physicians (odds ratio 1.251; p=0.009) and who had a higher number of hospital admissions (odds ratio 1.022; p<0.001). Multivariate analysis showed that age and number of hospital admissions were associated with achievement of the Continuing Medical Education requirements. Each hospital admission increased the chance of achieving the requirements by 0.4%. Among physicians who performed surgical procedures, multivariate analysis showed that male physicians were 1.3 time more likely to achieve the Continuing Medical Education requirements than female physicians. Each surgical procedure performed increased the chance of achieving the requirements by 1.4%. Conclusion The numbers of admissions and number of surgical procedures performed by physicians at our hospital were associated with the likelihood of meeting the Continuing Medical Education requirements. These findings help to shed new light on our Continuing Medical Education program.
    03/2015; 13(1). DOI:10.1590/S1679-45082015AO3189