No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Physician numeracy: Essential skills for practicing evidence-based medicine
Abstract
Teaching evidence-based medicine (EBM) has become common in family medicine residency programs. EBM teaching usually takes the form of journal clubs or encouraging residents to use EBM summaries of original articles. Both have significant limitations. Residents and other physicians recognize that understanding statistics is important for interpreting results. Unfortunately, they report low self confidence in this area reflected by studies that show poor skills. Physician numeracy refers to a broad range of mathematical skills needed for practicing medicine, including the statistical knowledge necessary to interpret original research papers. Numeracy should be given much more emphasis in residency curricula and can form the foundation of EBM teaching. Objectives for a numeracy curriculum based on a recent research article and a curriculum using an EBM framework are described.
... ANTECEDENTES Rao (2008) propone un currículo para la formación estadística de residentes de medicina, siguiendo los principios de medicina basada en la evidencia. Su objetivo es la comprensión de lenguaje estadístico y aspectos asociados al diseño, análisis y resultados expuestos en trabajos de investigación. ...
... 2. Los profesionales de Ciencias de la Salud Humana realizan investigación principalmente cuando cursan un posgrado y regularmente forman parte de equipos de investigación multidisciplinares, lo cual se evidenció en la revisión bibliográfica y en las respuestas de los coordinadores. 3. El análisis de artículos con aplicación explícita de la estadística en cada campo profesional favorece la motivación de los estudiantes y permite evidenciar la estructura del proceso investigativo, así como su articulación con diferentes aspectos de la estadística (Rao, 2008). La profundidad del análisis depende del momento del semestre y de las características de los estudiantes. ...
... Cabe mencionar que el cuestionario a los coordinadores incluía preguntas abiertas, entre ellas citar una publicación propia de su profesión como sugerencia para el curso, todos hicieron comentarios favorables al respecto. 4. Una adecuada articulación de estrategias pedagógicas (vinculando las sugerencias de Hayat, 2014;Oster y Enders, 2018;Rao, 2008) propende por el desarrollo de competencias y conocimientos, tanto conceptuales como procedimentales; además promueve el balance entre motivación, compromiso, tiempo y esfuerzo destinado a lo largo del periodo académico. ...
A curriculum is proposed for undergraduate students, under conditions of high academic diversity, sociodemographics, and professional interests. Curriculum design has been based on international recommendations and a consultation with program coordinators of a Colombian public university on statistical training needs. This curriculum aims to develop a selection of fundamental statistical competences required for future professional practices in health sciences, recognizing the importance in the training process of both the role of the learner and the role of the teacher. Se propone un currículo para estudiantes de pregrado, bajo condiciones de alta diversidad académica, sociodemográfica y de intereses profesionales. El diseño curricular se ha basado en recomendaciones internacionales y una consulta a los coordinadores de programa, de una universidad pública colombiana, sobre necesidades de formación estadística. Este currículo pretende desarrollar una selección de competencias estadísticas fundamentales requeridas para las futuras prácticas profesionales en Ciencias de la Salud, reconociendo la importancia en el proceso formativo tanto del rol del aprendiz como del rol del docente.
... PH has a great deal of overlap with CTS with regard to statistical competencies and the statistical competencies for PH [4] and CTS [3] have been combined [5] and continue to evolve [1]. In contrast to CTS and PH that seek to train independent medical researchers, EBM and GME are concerned with training medical professionals to critically evaluate the medical research literature and to incorporate research findings into practice-based learning environments [6]. Published statistical competencies for EBM and GME do not include as much detail as those for CTS and PH, and are generally much more limited in scope [4,7,8]. ...
... Indeed, our study demonstrates a limitation in current descriptions of competencies needed to evaluate the literature in EBM and GME. Within EBM, there is a cognitive skill associated with applying knowledge of study designs and statistical methods to the evaluation of the research literature [6], but these are not described in the detail we offer. Within GME, these elements are missing. ...
Introduction
It is increasingly essential for medical researchers to be literate in statistics, but the requisite degree of literacy is not the same for every statistical competency in translational research. Statistical competency can range from ‘fundamental’ (necessary for all) to ‘specialized’ (necessary for only some). In this study, we determine the degree to which each competency is fundamental or specialized.
Methods
We surveyed members of 4 professional organizations, targeting doctorally trained biostatisticians and epidemiologists who taught statistics to medical research learners in the past 5 years. Respondents rated 24 educational competencies on a 5-point Likert scale anchored by ‘fundamental’ and ‘specialized.’
Results
There were 112 responses. Nineteen of 24 competencies were fundamental. The competencies considered most fundamental were assessing sources of bias and variation (95%), recognizing one’s own limits with regard to statistics (93%), identifying the strengths, and limitations of study designs (93%). The least endorsed items were meta-analysis (34%) and stopping rules (18%).
Conclusion
We have identified the statistical competencies needed by all medical researchers. These competencies should be considered when designing statistical curricula for medical researchers and should inform which topics are taught in graduate programs and evidence-based medicine courses where learners need to read and understand the medical research literature.
... International assessments have found poor numeracy to be widespread in a number of countries [5]. Low numeracy has also been identified as a problem in both patients [6] and health professionals [7][8][9]. Patients with low numeracy will have difficulties assessing the likelihood, and probability, of any harms and benefits associated with treatment options. Health professionals with low numeracy will have difficulty explaining numerical information needed to guide patients through difficult treatment decisions. ...
... Patient decision aids that are designed to reflect the needs of patients with different levels of health literacy may not meet the needs of patients with different levels of numeracy. Academic institutions that train health professionals, health policymakers, and clinical practice managers need to: (a) support the regular assessment of health professional's numeracy skills and provide training to meet existing deficits, and (b) ensure that health professionals are trained to communicate numerical information to patients in a manner that is easily and accurately understood by patients during the treatment decision making process [7,9]. Health professionals must actively seek to maintain and improve their own numeracy as well as the numeracy of their patients. ...
... Since a large number of studies published in the medical literature have clinical applications, and since proper understanding of these studies and their potential impact on clinical practice is crucial to being a good practitioner, physicians need to develop certain capabilities and information management strategies for handling the volume of new information that they constantly receive from various sources. A number of intellectual competencies, which are necessary "to understand the quantitative aspects of clinical medicine, [and] original research" [13]-generally referred to as "physician numeracy" skills [13] -are indispensable for the practice of modern medicine. Examples of such skills include the ability to interpret standard deviation, relative risk, confidence interval and statistical significance, and p value; recognize power, sample size, and bias; and determine strength of evidence for risk factors [14]. ...
... Since a large number of studies published in the medical literature have clinical applications, and since proper understanding of these studies and their potential impact on clinical practice is crucial to being a good practitioner, physicians need to develop certain capabilities and information management strategies for handling the volume of new information that they constantly receive from various sources. A number of intellectual competencies, which are necessary "to understand the quantitative aspects of clinical medicine, [and] original research" [13]-generally referred to as "physician numeracy" skills [13] -are indispensable for the practice of modern medicine. Examples of such skills include the ability to interpret standard deviation, relative risk, confidence interval and statistical significance, and p value; recognize power, sample size, and bias; and determine strength of evidence for risk factors [14]. ...
Since a large number of studies published in the medical literature have clinical applications, and since proper understanding of these studies and their potential impact on clinical practice is crucial to being a good practitioner, physicians need to develop certain capabilities and information management strategies for handling the volume of new information that they constantly receive from various sources. A number of intellectual competencies, which are necessary “to understand the quantitative aspects of clinical medicine, [and] original research” [13]—generally referred to as “physician numeracy” skills [13] —are indispensable for the practice of modern medicine. Examples of such skills include the ability to interpret standard deviation, relative risk, confidence interval and statistical significance, and p value; recognize power, sample size, and bias; and determine strength of evidence for risk factors [14]. However, the fact is that many physicians do not have the necessary competencies for understanding the results of scientific research and appraising medical literature [14-16]. In addition, most physicians seem to lack a clear information management strategy to process the information, distinguish between high- and low-quality information, and integrate high-quality information into patient care [17, 18].
... This finding puts extra pressure on medical educators to teach evidence-based medicine, incorporating higher-order skills to interpret original research papers such as study design and statistical interpretation. 36 Increasingly, family physicians will need to critically appraise emerging literature before it is peer reviewed, whether they encounter it in their own searches or when a patient presents information they found before an appointment. ...
Background and objectives:
Online publication of prereviewed manuscripts disseminates research simultaneously to scientists, clinicians, and patients, enabling the media and public to act as scientific reviewers for studies that are not yet endorsed by the scientific and clinical community. This study describes the reach of prereview literature and frames it within the pursuit to teach evidence-based medicine.
Methods:
In this deductive content analysis, the primary unit of analysis was the individual preprint manuscript submitted to the medRxiv preprint server during the emergence of the COVID-19 pandemic. The coding scheme included study design, negative or positive findings, dissemination status (whether it was withdrawn from the server or eventually published), and three levels of reach: user engagement, news media coverage, and social media engagement.
Results:
Prereviewed manuscripts describe a variety of study methods. Dissemination status was significantly related to abstract views, manuscript views, news coverage, and social media exposure. Studies with negative findings had higher counts of abstract views, manuscript views, and news coverage, but no significant relationships were detected.
Conclusions:
Results demonstrate that not only are scientists publishing negative findings, but that those studies reach a wide audience. Notably, eventually-withdrawn manuscripts, potentially containing incomplete or uncertain science, is reaching the public domain. Increasingly, family physicians will need to critically appraise emerging literature before it is peer reviewed, whether they encounter it in their own searches or when a patient presents information they found before an appointment.
... Neben der zeitlichen Ansiedlung ist auch deren Umfang und Ausrichtung unklar: Je nach Standort bieten Methoden-Abteilungen postgraduelle Weiterbildungen von mehreren Stunden oder auch mehreren Wochen an; speziell im letzteren Kontext werden Pflicht-Weiterbildungen sicherlich umso kritischer diskutiert. Reed und Rao publizierten in den 2000er-Jahren notwendige Inhalte zu solchen Weiterbildungen, in Anlehnung an die gängigsten statistischen Konzepte und Methoden in medizinischen Zeitschriften [13,14]. Entscheidend für eine Bemessung des möglichen Volumens solcher Weiterbildungsangebote kann jedoch letztlich nur eine Messung des bestehenden Wissensund damit des Weiterbildungsbedarfssein. Vor diesem Hintergrund sollte ein multidisziplinärer Survey unter klinisch tätigen Ärztinnen und Ärzten eines Universitätsklinikums zur Messung der Vertrautheit mit gängigen Termini und Kenngrößen sowie grafischen Methoden der EbM durchgeführt werden. ...
Background:
Patient care according to the principles of evidence-based medicine (EbM) is defined by the Social Code (SGB, inter alia § 137f SGB V). As part of quality assurance in medicine, a mandatory further education for doctors is demanded. The aim of this investigation was to evaluate how familiar medical practitioners are with common terms and parameters of EbM. Another question was, at which point of the medical training a possible postgraduate training on EbM methods should take place.
Material and methods:
In February 2017, an anonymous survey (paper questionnaire) was conducted at four clinics of the Witten/Herdecke University. The questionnaire offered 12 application scenarios for EbM methods; all items could be clearly answered with "right" or "wrong"; the refusal of an answer to single items was allowed. The investigation's primary endpoint was the individual proportion [%] of correctly among all available answers for each survey participant. The primary analysis sought to estimate this proportion of correct answers by means of a two-sided 95 % confidence interval.Results A total of 70 evaluable questionnaires could be evaluated (15-19 per clinic). The median individual proportion of correct answers was 47 % (95 % CI 40 %; 50 %). The four clinics did not differ significantly with respect to this proportion of correct answers (Kruskal/Wallis p = 0.388) and showed median proportions between 40 % and 50 %.
Discussion:
Medical practitioners at university hospitals can be expected to have certain knowledge on common methods of clinical research. However, less than 50 % of the answer were correct and a need for further training in EbM-specific methods appears to be indicated. The survey did not provide information on the optimal timing of such offers during medical training.
... Many decisions about health involve the consideration of complex numerical information about risks and benefits. On such occasions, medical professionals are expected to be risk literate decision makers and advisors to their patients [1][2][3]. Risk literacy broadly refers to one's a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 are often reinforced by screening campaigns that fail to specify the extent of benefit and important risks like false positive tests or overdiagnosis [16]. Such campaigns may create the impression that cancer screening is always the best choice and not a matter of careful evaluation of the evidence of benefits and harms. ...
We investigated what factors may foster or hinder physicians’ cancer screening risk literacy–specifically the ability to understand evidence regarding screening effectiveness and make evidence-based recommendations to patients. In an experiment, physicians in training (interns and residents) read statistical information about outcomes from screening for cancer, and had to decide whether to recommend it to a patient. We manipulated the effectiveness of the screening (effective vs. ineffective at reducing mortality) and the demand of the patient to get screened (demand vs. no demand). We assessed participants’ comprehension of the presented evidence and recommendation to the patient, as well as a-priori screening beliefs (e.g., that screening is always a good choice), numeracy, science literacy, knowledge of screening statistics, statistical education, and demographics. Stronger positive a-priori screening beliefs, lower knowledge of screening statistics, and lower numeracy were related to worse comprehension of the evidence. Physicians recommended against the ineffective screening but only if they showed good comprehension of the evidence. Physicians’ recommendations were further based on the perceived benefits from screening but not on perceived harms, nor the patient’s demands. The current study demonstrates that comprehension of cancer screening statistics and the ability to infer the potential benefits for patients are essential for evidence-based recommendations. However, strong beliefs in favor of screening fostered by promotion campaigns may influence how physicians evaluate evidence about specific screenings. Fostering physician numeracy skills could help counteract such biases and provide evidence-based recommendations to patients.
... Physicians and medical students need to comprehend and understand the importance of numerical and graphical information related to health issues (e.g., risks of different diseases and treatments) in order to order for adequate information to be transmitted to patients. [1][2][3] To make an effective decision, health personnel must be aware of the risks and benefits of different medical treatments, diagnostic tests and lifestyle modifications; however, different investigations indicate that numerical and graphical concepts are not always understood, and these are essential skills for health issues to be understood and communicated. [4][5][6][7] To investigate these skills, scales have been developed in different countries such as the United States, Germany and Spain, [8][9][10] which comply with appropriate psychometric properties to be applied in different clinical and research scenarios. ...
... Los médicos y estudiantes de medicina necesitan comprender y entender la importancia de la información numérica y gráfica relacionada con temas de salud (por ejemplo, riesgos de diferentes enfermedades y tratamientos), para transmitir información adecuada a los pacientes. [1][2][3] Para tomar una decisión efectiva, el personal de salud debe entender los riesgos y beneficios de diferentes tratamientos médicos, pruebas diagnósticas y modificaciones en los estilos de vida; sin embargo, diversas investigaciones indican que no siempre comprende conceptos numéricos y gráficos, habilidades esenciales para el entendimiento y comunicación en los temas de salud. [4][5][6][7] Para la investigación de estas habilidades se han desarrollado escalas en diversos países como Estados Unidos, Alemania y España, [8][9][10] las cuales cumplen con propiedades psicométricas apropiadas para ser aplicadas en diversos escenarios clínicos y de investigación. ...
Introduction:
Physicians and medical students need to understand numerical and graphical health data in order to provide patients with correct information.
Objective:
The graphical and numerical skills of medical students and residents from a private university in Lima, Peru, were determined in this investigation.
Method:
Cross-sectional, descriptive study. The Objective Numeracy, Subjective Numeracy and Graph Literacy Scales were applied to medical students at their two final years of medical school and to medical residents.
Results:
Of 169 participants, 52.07% were sixth-year and 18.34% were seventh-year students and 29.58% were residents. Mean objective numeracy score was 7.34, mean subjective numeracy was 34.12 and mean graph literacy was 10.35. A multiple linear regression analysis showed that Subjective and Objective Numeracy Scales highest means were associated with the male gender and training on research methodology (p < 0.05). Graph Literacy Scale highest means were associated with the male gender and younger age (p-value < 0.05).
Conclusion:
Numeracy and Graph Literacy Scales mean scores were high in medical students.
... To avoid such misguided recommendations, it is necessary to improve numeracy, risk literacy, and statistical skills training in medical curricula and continuing education. 62,64,65 A recent review of cancer prevention and screening recommendation statements showed that as much as 69% either did not quantify the benefits and harms of screening or presented them in an asymmetric manner. 66 This, together with the current results, suggests that there is a pressing need for well-designed decision aids to help less numerate physicians and patients understand and discuss life-altering risks and benefits. ...
Objective:
Many patients have low numeracy, which impedes their understanding of important information about health (e.g., benefits and harms of screening). We investigated whether physicians adapt their risk communication to accommodate the needs of patients with low numeracy, and how physicians' own numeracy influences their understanding and communication of screening statistics.
Methods:
UK family physicians ( N = 151) read a description of a patient seeking advice on cancer screening. We manipulated the level of numeracy of the patient (low v. high v. unspecified) and measured physicians' risk communication, recommendation to the patient, understanding of screening statistics, and numeracy.
Results:
Consistent with best practices, family physicians generally preferred to use visual aids rather than numbers when communicating information to a patient with low (v. high) numeracy. A substantial proportion of physicians (44%) offered high quality (i.e., complete and meaningful) risk communication to the patient. This was more often the case for physicians with higher (v. lower) numeracy who were more likely to mention mortality rates, OR=1.43 [1.10, 1.86], and harms from overdiagnosis, OR=1.44 [1.05, 1.98]. Physicians with higher numeracy were also more likely to understand that increased detection or survival rates do not demonstrate screening effectiveness, OR=1.61 [1.26, 2.06].
Conclusions:
Most physicians know how to appropriately tailor risk communication for patients with low numeracy (i.e., with visual aids). However, physicians who themselves have low numeracy are likely to misunderstand the risks and unintentionally mislead patients by communicating incomplete information. High-quality risk communication and shared decision making can depend critically on factors that improve the risk literacy of physicians.
... Practising evidence-based medicine (EBM) is considered important in health care, also in the field of general practice [1][2][3][4][5]. EBM is the integration of clinical expertise, patient values and the best available clinical evidence into daily clinical practice. ...
Background and Objectives: For Evidence Based Medicine (EBM) learning journal clubs are recommended, but these are not common practice. How participants discuss and learn about applying evidence in other group meetings is unknown. We examined different types of group meetings and explored the use of, and discussions about, clinical research evidence.
Methods: A mixed-methods study design was adopted. After distribution and analysis of a questionnaire about types of group meetings, interviews were conducted to better understand the most frequently occurring type.
Results: GPs have different types of meetings, but the most common group meetings where evidence wass discussed were so called quality circles, i.e. pharmacotherapy audit meetings in which GPs discuss drug prescription figures or preferred treatment together with pharmacists. Interviews showed that the source of evidence used mostly are the recommendations in the national GP guidelines. The underlying evidence or new research did not play an important role in the discussions.
Conclusions: Quality circles seem to be more goal-oriented than learning oriented. Learning discussions about controversies in clinical research or about the integration of evidence, patient values and clinical expertise occurred infrequently. To harvest the potential value of group meetings for EBM learning, quality circles in their present design are not optimal.
... While most physicians agree that a good understanding of basic statistical concepts is necessary in contemporary medicine, very few feel confident of their own skills [143]. This is a longstanding problem that has been repeatedly documented in Europe and North America [144][145][146]. Up until recently, these skills were deemed important by physicians espousing the principles of evidence-based medicine. ...
The genomic architecture of kidney disease has fascinated developmental biologists and human geneticists for over four decades. Seminal discoveries of note include the discovery of genes implicated in autosomal dominant/recessive polycystic kidney disease, nephronophthisis and nephrotic syndrome. Uncovering disease-causing genes has not only helped refine our pathogenetic understanding of many renal diseases, but in many cases it has directly translated into concrete improvement of patient care. The recent emergence of next generation sequencing strategies has dramatically accelerated the discovery process, and constitutes the cornerstone towards the realization of personalized medicine. This chapter provides a brief overview of basic genomic/genetic concepts before delving into recent advances that are pertinent to the practice of contemporary pediatric nephrologists. From laboratory methods to interpretation of genetic variants, every topic is presented within a clinical framework and is enriched with numerous examples from the pediatric nephrology literature. The major benefits of genomics on the day-to-day practice of busy clinicians are as follows: it will expediate diagnosis, clarify prognosis and guide therapeutic choices. Our overarching goals for this chapter were twofolds: to first convince clinicians “already overloaded with information” that learning about genomics is a worthwhile investment that will pay dividends in the short-term, while at the same time providing an accessible port of entry into this complex field.
... There are several limitations in this study. Most instruments that measure information and computer literacies are designed for patients and almost none target clinicians' competencies [33,45,46], so this study by its nature was exploratory. While our information literacy construct showed strong internal validity, it requires further refinement and validation in other clinician populations. ...
Background:
As healthcare moves towards technology-driven population health management, clinicians must adopt complex digital platforms to access health information and document care.
Objectives:
This study explored information literacy, a set of skills required to effectively navigate population health information systems, among primary care providers in one Veterans' Affairs (VA) medical center.
Methods:
Information literacy was assessed during an 8-month randomized trial that tested a population health (panel) management intervention. Providers were asked about their use and comfort with two VA digital tools for panel management at baseline, 16 weeks, and post-intervention. An 8-item scale (range 0-40) was used to measure information literacy (Cronbach's α=0.84). Scores between study arms and provider types were compared using paired t-tests and ANOVAs. Associations between self-reported digital tool use and information literacy were measured via Pearson's correlations.
Results:
Providers showed moderate levels of information literacy (M= 27.4, SD 6.5). There were no significant differences in mean information literacy between physicians (M=26.4, SD 6.7) and nurses (M=30.5, SD 5.2, p=0.57 for difference), or between intervention (M=28.4, SD 6.5) and control groups (M=25.1, SD 6.2, p=0.12 for difference). Information literacy was correlated with higher rates of self-reported information system usage (r=0.547, p=0.001). Clinicians identified data access, accuracy, and interpretability as potential information literacy barriers.
Conclusions:
While exploratory in nature, cautioning generalizability, the study suggests that measuring and improving clinicians' information literacy may play a significant role in the implementation and use of digital information tools, as these tools are rapidly being deployed to enhance communication among care teams, improve health care outcomes, and reduce overall costs.
... Rao argues that physician numeracy is an essential skill for practicing EBM, and proposes an elaborate curriculum for teaching numeracy during the residency period. [34] It is not surprising that the medical representative finds it easy to take a doctor for a ride with glossy presentations and glossier, but not accurate statistics on his/her drug. Interactions with colleagues have made it clear that statistics to most physicians is limited to "means" and "averages," while "median" or "mode" is out of reach. ...
... Rao argues that physician numeracy is an essential skill for practicing EBM, and proposes an elaborate curriculum for teaching numeracy during the residency period. [34] It is not surprising that the medical representative finds it easy to take a doctor for a ride with glossy presentations and glossier, but not accurate statistics on his/her drug. Interactions with colleagues have made it clear that statistics to most physicians is limited to "means" and "averages," while "median" or "mode" is out of reach. ...
Introduction: All patients have a right to the best treatment for the ailment that they suffer from, and the best treatment usually would be evidence based medicine (EBM).
Practice of EBM: This is formulated considering the physician's personal experience, that of the peers and the patients' expectations. In our country EBM is conspicuous in its absence due to poor maintenance of patients' records, poor access to journals and non-involvement of patients in therapeutic decisions. Physicians in private practice are generally not trained to evaluate external experience, whatever is available. Badly designed diagnostic approach and poor quality of information on which therapeutic decisions are based add to factors that prevent the implementation of EBM.
Conclusions: In certain conditions patient centred medicine may be superior to EBM. However, patient centred medicine for special situations can only be a reality after EBM has been made the norm for commonly encountered diseases.
... Vor allem Ärztinnen und Ärzte werden gemeinhin als diejenigen gesehen, die zu wenig Zeit für einzelne Patientinnen und Patienten aufbringen, nicht zuhören oder Informationen vorenthalten – aber selten wird hier mangelndes statistisches Wissen betrachtet (z. B. Berwick, Fineberg & Weinstein, 1981; Rao, 2008 ...
Um informierte und selbstbestimmte Entscheidungen von Patientinnen und Patienten möglich zu machen, brauchen wir risikokompetente Ärzte und Ärztinnen sowie eine risikokompetente Öffentlichkeit: Beides kann zum einen durch bessere Fähigkeiten im Umgang mit Gesundheitsstatistiken und zum anderen durch transparente Kommunikation solcher Zahlen erreicht werden.
... Most legal and psychological articles on patient-doctor communication assume that the problem lies in the patient's mind. Doctors may be said to pay insufficient attention to their patients' feelings or not listen carefully to their complaints, consult with them only 5 minutes on average, or withhold information-but rarely is it contemplated that many doctors might not understand medical evidence themselves (e.g., Berwick, Fineberg, & Weinstein, 1981;Rao, 2008). Yet most doctors we have studied do not understand health statistics and thus cannot evaluate the evidence for or against a treatment, or critically judge a report in a medical journal . ...
Efficient health care requires informed doctors and patients. The health care system we inherited from the 20th century falls short on both counts. Most doctors and patients do not understand the available medical evidence. We identify seven "sins" that have contributed to this lack of knowledge: biased funding; biased reporting in medical journals, brochures, and the media; conflicts of interest; defensive medicine; and medical schools that fail to teach doctors how to comprehend health statistics. These flaws have generated a partially inefficient system that wastes taxpayers' money for unnecessary or even potentially harmful tests and treatments, and for medical research that is of limited relevance for the patient. Raising taxes or rationing care are often thought to be the only alternatives in the face of exploding health care costs. Yet there is a third option through promoting health literacy: getting better care for less money. The 21st century should become the century of the patient. Governments and health institutions need to change course and provide honest and transparent information, creating better doctors, better patients, and, ultimately, better health care.
... 26 Most residency programs failed to develop what has been called "physician numeracy" skills in the trainees. 27 In other words, if physicians do not develop these and the other necessary intellectual virtues during their medical education, there is no chance that they will do better in their interactions with pharmaceutical representatives. As Matthews and McPherson 28 rightly said so well decades ago: ...
The pharmaceutical industry's wide range of interactions with physicians, trainees, and other medical professionals-interactions that include information transfer and financial incentives-has been the source of undue influences, especially on physicians' prescription behavior. Current literature has mainly been focused on the financial element of these influences, and the problems in medical professional-pharmaceutical industry interactions are mainly viewed in terms of conflicts of interest. There is often the assumption that physicians are intellectually competent but biased because of financial incentives.The author rejects that assumption and proposes an alternative explanation for the observed influence of the pharmaceutical industry on physicians' behavior by emphasizing the importance of the information-transfer side of the interactions and maintaining that physicians and other medical professionals need certain intellectual virtues (i.e., competencies) to properly assess the information, which is often unreliable and biased. These virtues are necessary for the practice of modern medicine and include mindfulness, the ability to understand practical implications of newly found evidence, to consider alternative explanations of data, to recognize and correct errors, to decide on the best available evidence, and to tailor that to the needs and values of individual patients. On the basis of this view, the author recommends that the best solution for the observed problems in physician-pharmaceutical industry interactions is to "vaccinate" physicians and other medical professionals by increasing efforts to inculcate the necessary intellectual virtues early in medical education and fostering them throughout those individuals' professional lives.
... Lesser and Parker 12 argue that biostatisticians should develop statistics courses specifi cally for biomedical investigators. Windish et al., 13,14 Novack et al., 15 and Rao 16 discuss the challenges that physicians and others who receive CTS training have with traditional statistical coursework. O'Brien et al. 17 discuss that when developing statistics courses for CTS learners, it is important to recognize that "conducting clinical research requires a fundamental understanding of the terminology and concepts of biostatistics, " but that "few clinical scientists need to know the mathematical and computing technicalities as covered in traditional academic courses. ...
IntroductionStatistics is an essential training component for a career in clinical and translational science (CTS). Given the increasing complexity of statistics, learners may have difficulty selecting appropriate courses. Our question was: what depth of statistical knowledge do different CTS learners require?Methods
For three types of CTS learners (principal investigator, co-investigator, informed reader of the literature), each with different backgrounds in research (no previous research experience, reader of the research literature, previous research experience), 18 experts in biostatistics, epidemiology, and research design proposed levels for 21 statistical competencies.ResultsStatistical competencies were categorized as fundamental, intermediate, or specialized. CTS learners who intend to become independent principal investigators require more specialized training, while those intending to become informed consumers of the medical literature require more fundamental education. For most competencies, less training was proposed for those with more research background.DiscussionWhen selecting statistical coursework, the learner's research background and career goal should guide the decision. Some statistical competencies are considered to be more important than others. Baseline knowledge assessments may help learners identify appropriate coursework.Conclusion
Rather than one size fits all, tailoring education to baseline knowledge, learner background, and future goals increases learning potential while minimizing classroom time.
... 24 One underlying reason for why even identically informing physicians provided information that was not fully understandable could be a lack of understanding statistical information. [25][26][27][28][29] However, we did not find any relation between the way physicians reported that they would inform patients and their subjective numeracy. It could be that subjective numeracy is not sensitive enough in this regard, although it is correlated with objective numeracy. ...
Little is known about how physicians provide statistical information to patients, which is important for informed consent.
In a survey, obstetricians and gynecologists (N = 142) received statistical information about the benefit and side effects of an antidepressant. They received information in various formats, including event rates (antidepressant v. placebo), absolute risks, and relative risks. Participants had to imagine 2 hypothetical patients, 1 for whom they believed the drug to be safe and effective and 1 for whom they did not, and select the information they would give those patients. We assessed whether the information they selected for each patient was complete, transparent, interpretable, or persuasive (i.e., to nudge patients toward a particular option) and compared physicians who gave both patients the same information with those who gave both patients different information.
A similar proportion of physicians (roughly 25% each) selected information that was 1) complete and transparent, 2) complete but not transparent, 3) not interpretable for the patient because necessary comparative information was missing, or 4) suited for nudging. Physicians who gave both patients the same information (61% of physicians) more often selected at least complete information, even if it was often not transparent. Physicians who gave both patients different information (39% of physicians), in contrast, more often selected information that was suited for nudging in line with the belief they were asked to imagine. A limitation is that scenarios were hypothetical.
Most physicians did not provide complete and transparent information. Clinicians who presented consistent information to different patients tended to present complete information, whereas those who varied what information they chose to present appeared more prone to nudging.
... The intervention addressed key skills emphasized in the literature on EBM education and information mastery: formulating clinical questions, searching EBM resources for answers, and most importantly, applying their knowledge to patient care in real time. [17][18][19] Our intervention required a single faculty member with a one half per day per week (0.1 FTE) time commitment and some expertise in EBM and basic epidemiology. This time commitment includes development of curriculum for topics covered and would likely be less at smaller residency programs. ...
Medical educators have used multiple interventions to teach evidence-based medicine (EBM) and information mastery. Most of these interventions are applied uniformly to a group of residents. We developed a curriculum to increase residents' EBM and information mastery skills that would meet individual learning needs.
Two cohorts of second year residents (n=26) in the Family Medicine Residency Program at Brown University participated in the intervention. We delivered the curriculum through monthly individual meetings with a learning coach over 1 year. Before and after participating, residents completed a survey assessing their attitudes toward EBM and a quiz assessing their knowledge of EBM. Semi-structured interviews with each resident were done after the intervention and analyzed using qualitative methods.
At the conclusion of our intervention, residents' attitudes toward EBM and information mastery were overwhelmingly positive, EBM knowledge quiz scores increased by 31.8%, and reported use of EBM in real time during patient encounters increased.
An intervention using a learning coach to provide one-on-one EBM and information mastery instruction to residents improved residents' attitudes, knowledge, and use of both in the clinical setting.
... 4 Our data, therefore, not only fit well with earlier studies' findings, but also extend the field to a new domain. The results support the growing body of evidence showing that numeracy plays an important role in medical students' ability to interpret medical data (Sheridan & Pignone, 2002 ), physicians' capacity to practice evidence-based medicine (Rao, 2008 ), and clinicians' ability to communicate risk information to patients (Gigerenzer et al., 2008; Nelson, Reyna, Fagerlin, Lipkus, & Peters, 2008). Our data also corroborate earlier indications 3 It is possible that better performance with fewer plans is driven by chance, as participants had a higher probability of guessing correctly when faced with three plans compared with 10 or 20 plans. ...
In this study, we examined the effect of choice-set size and numeracy levels on a physician-in-training's ability to choose appropriate Medicare drug plans.
Medical students and internal medicine residents (N = 100) were randomly assigned to 1 of 3 surveys, differing only in the number of plans to be evaluated (3, 10, and 20). After reviewing information about stand-alone Medicare prescription drug plans, participants answered questions about what plan they would advise 2 hypothetical patients to choose on the basis of a brief summary of the relevant concerns of each patient. Participants also completed an 11-item numeracy scale.
Ability to answer correctly questions about hypothetical Medicare Part D insurance plans and numeracy levels.
Consistent with our hypotheses, increases in choice sets correlated significantly with fewer correct answers, and higher numeracy levels were associated with more correct answers. Hence, our data further highlight the role of numeracy in financial- and health-related decision making, and also raise concerns about physicians' ability to help patients choose the optimal Part D plan.
Our data indicate that even physicians-in-training perform more poorly when choice size is larger, thus raising concerns about the capacity of physicians-in-training to successfully navigate Medicare Part D and help their patients pick the best drug plan. Our results also illustrate the importance of numeracy in evaluating insurance-related information and the need for enhancing numeracy skills among medical students and physicians.
... Most psychological, legal, and medical articles on patient–doctor communication assume that the problem lies in the patient's mind. Doctors may be said to pay insufficient attention to their patients' feelings or not listen carefully to their complaints, consult with them only 5 minutes on average, or withhold information—but rarely is it considered that many doctors might be statistically illiterate (e.g., Berwick, Fineberg, & Weinstein, 1981; Rao, 2008). Why do doctors need minimum statistical literacy? ...
Many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without noticing. Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. For instance, many citizens are unaware that higher survival rates with cancer screening do not imply longer life, or that the statement that mammography screening reduces the risk of dying from breast cancer by 25% in fact means that 1 less woman out of 1,000 will die of the disease. We provide evidence that statistical illiteracy (a) is common to patients, journalists, and physicians; (b) is created by nontransparent framing of information that is sometimes an unintentional result of lack of understanding but can also be a result of intentional efforts to manipulate or persuade people; and (c) can have serious consequences for health.
The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor–patient relationship and conflicts of interest in the healthcare system. The classic doctor–patient relation is based on (the physician's) paternalism and (the patient's) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none). We show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Without understanding the numbers involved, the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making.
What can be done? We discuss the importance of teaching statistical thinking and transparent representations in primary and secondary education as well as in medical school. Yet this requires familiarizing children early on with the concept of probability and teaching statistical literacy as the art of solving real-world problems rather than applying formulas to toy problems about coins and dice. A major precondition for statistical literacy is transparent risk communication. We recommend using frequency statements instead of single-event probabilities, absolute risks instead of relative risks, mortality rates instead of survival rates, and natural frequencies instead of conditional probabilities. Psychological research on transparent visual and numerical forms of risk communication, as well as training of physicians in their use, is called for.
Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.
... In addition to the methods of presentation (which will be discussed in detail in the next section), time constraints combined with the overwhelming amount of information that patients are required to receive create challenges for risk communication. Unfortunately, many health care providers are subject to these same biases and misunderstandings of risk (Rao, 2008). Furthermore, many do not have adequate training in risk communication (Paling, 2003;Thomson et al., 2005). ...
EVIDENCE THAT LAY PEOPLE FREQUENTLY misinterpret risk raises concerns for the ethical conduct of human research, which requires adequate disclosure, understanding, and appreciation of risk information. Review of the risk communication research literature suggests new directions for empirical research on human research ethics: Investigation is needed on how to best assess and improve potential and enrolled subjects' understanding of risk information. Preferences regarding the presentation of risk information and the effects of alternative presentation formats and decision aids on knowledge, trust, satisfaction, risk/benefit analysis, and perceptions of respectful treatment should be studied. Research is also needed on the effects of payment for research participation, the order in which study information is presented, and having one's own physician present risk information.
Introduction:
Health numeracy (numerical literacy) refers to an individual's ability to use numerical information to make effective health decisions. Numeracy is fundamental in the role of a health care provider, forming the basis of evidence-based medicine and effective patient-provider communication. Despite a high level of education, many health care providers struggle with numeracy. Numeracy is often integrated into training curricula; however, teaching modality, competencies covered, learner satisfaction, and effectiveness of these educational interventions varies.
Method:
A scoping review was conducted to explore and summarize what is known about numeracy skills education programs for health care providers. A comprehensive literature search was conducted from January 2010 to April 2021 in 10 databases. Controlled vocabulary terms and text words were used. The search was restricted to human studies, adults, and the English language. Articles were included if they were related to numeracy education for health care providers or trainees and provided details regarding methods, evaluation, and results.
Results:
The literature search retrieved 31,611 results and 71 met the inclusion criteria. Most interventions were conducted in a university setting, and targeted nursing students, medical students, resident physicians, and pharmacy students. Common numeracy concepts included statistics/biostatistics, medication calculations, evidence-based medicine, research methodology, and epidemiology. A variety of teaching modalities were used, which most often combined active approaches (eg, workshops, laboratories, small-group exercises, and discussion boards) with passive approaches (eg, traditional lectures and didactic teaching). Measured outcomes included knowledge and skills, self-efficacy, attitudes, and engagement.
Discussion:
Although efforts have been made to incorporate numeracy into training curricula, greater emphasis should be placed on developing strong numeracy skills in health care providers, particularly given the role numerical information plays in clinical decision making, evidence-based practices, and patient-provider communication.
In an age of competing medical priorities, why should healthcare professionals who are already overloaded with information develop core competencies in genetics and genomics? [1]
The genomic architecture of kidney disease has fascinated developmental biologists and human geneticists for over four decades. Seminal discoveries of note include the discovery of genes implicated in autosomal dominant/recessive polycystic kidney disease, nephronophthisis, and nephrotic syndrome. Uncovering disease-causing genes has helped refine our pathogenetic understanding of many renal diseases, and in many cases, it has directly translated into a concrete improvement of patient care. The recent emergence of next-generation sequencing strategies has dramatically sped up the discovery process and constitutes the cornerstone towards the realization of personalized medicine. This chapter summarizes basic genomic/genetic concepts before delving into recent advances that are pertinent to the practice of contemporary pediatric nephrologists. From laboratory methods to the interpretation of genetic variants, we present every topic within a clinical framework enriched with many examples from the pediatric nephrology literature. The major benefits of genomics to the day-to-day practice of busy clinicians are: it will expedite diagnosis, clarify prognosis, and guide therapeutic choices. Our overarching goals for this chapter were two-fold: to first convince clinicians “already overloaded with information” that learning about genomics is a worthwhile investment that will pay dividends in the short-term, while providing an accessible port of entry into this complex field.KeywordsGenetic testingMendelian conditionPathogenic variantFamily historyNext-generation sequencing
Every day thousands of individuals need to make critical decisions about their health based on numerical information, yet recent surveys have found that over half the population of the United States is unable to complete basic math problems. How does this lack of numerical ability (also referred to as low numeracy, quantitative illiteracy or statistical illiteracy) impact healthcare? What can be done to help people with low numeracy skills? Numerical Reasoning in Judgments and Decision Making about Health addresses these questions by examining and explaining the impact of quantitative illiteracy on healthcare and in specific healthcare contexts, and discussing what can be done to reduce these healthcare disparities. This book will be a useful resource for professionals in many health fields including academics, policy makers, physicians and other healthcare providers.
Statistics is an important component of the knowledge base for health care professionals. In this essay, it is argued that statistical knowledge for teaching (SKT) should be considered an important component of their preparation as well. Health care professionals often must help others understand the statistical basis for recommendations they make. A COVID-19 press briefing is used to illustrate the need for SKT when making high-stakes recommendations related to public health. It is conjectured that efforts to educate the public during the press briefing would have been enhanced if the presenters had deeper knowledge of the general public’s common statistical thinking patterns, the typical statistics curriculum experienced by members of their audience, and contemporary tools for teaching statistics. The importance of such knowledge to support smaller-scale individual interactions is also discussed. A call for SKT-centered partnerships between educational researchers and medical researchers is made; such partnerships could be mutually beneficial to the development of both fields and to society at large.
Ninety-seven clinicians answered at least 1 item, providing 939 responses; 72 answered all 11 items. The mean percentage of correct answers was 61% (95% confidence interval [CI], 57%-65%). Overall, doctor performance was better than performance by nurses and other respondents (68% [95% CI, 63%-73%] vs. 57% [95% CI, 52%-62%], P = 0.003). In basic numeracy, mean percent correct was 82% (95% CI, 77%-87%). For risk-adjustment numeracy, the mean percent correct was 70% (95% CI, 64%-76%). Risk-adjustment interpretation had the lowest average percent correct, 43% (95% CI, 37%-49%). All pairwise differences between concepts were statistically significant at
P <0.05.
Background
Despite widespread calls for its use, there are challenges to the implementation of evidence-based medicine (EBM) in clinical practice.
Methods
In response to the challenges of finding timely, pertinent information on diagnostic test accuracy, we developed an online, crowd-sourced Wiki on diagnostic test accuracy called Get the Diagnosis (GTD, http://www.getthediagnosis.org).
Results
Since its launch in November 2008 till October 2015, GTD has accumulated information on 300 diagnoses, with 1617 total diagnostic entries. There are a total of 1097 unique diagnostic tests with a mean of 5.4 tests (range 0–38) per diagnosis. 73% of entries (1182 of 1617) have an associated sensitivity and specificity and 89% of entries (1432 of 1617) have associated peer-reviewed literature citations. Altogether, GTD contains 474 unique literature citations. For a sample of three diagnoses, the search precision (percentage of relevant results in the first 30 entries) in GTD was 100% as compared with a range of 13.3%–63.3% for PubMed and between 6.7% and 76.7% for Google Scholar.
Conclusion
GTD offers a fast, precise and efficient way to look up diagnostic test accuracy. On three selected examples, GTD had a greater precision rate compared with PubMed and Google Scholar in identifying diagnostic test information. GTD is a free resource that complements other currently available resources.
Although biostatistics and clinical epidemiology are essential for comprehending medical evidence, research has shown consistently low and variable knowledge among postgraduate medical trainees. Simultaneously, there has been an increase in the complexity of statistical methods among top-tier medical journals.
To develop the Biostatics and Clinical Epidemiology Skills (BACES) assessment by (1) establishing content validity evidence of the BACES; (2) examining the model fit of the BACES items to an Item Response Theory (IRT) model; and (3) comparing IRT item estimates with those of traditional Classical Test Theory (CTT) indices.
Thirty multiple choice questions were written to focus on interpreting clinical epidemiological and statistical methods. Content validity was assessed through a four-person expert review. The instrument was administered to 150 residents across three academic medical centres in southern USA during the autumn of 2013. Data were fit to a two-parameter logistic IRT model and the item difficulty, discrimination and examinee ability values were compared with traditional CTT item statistics.
147 assessments were used for analysis (mean (SD) score 14.38 (3.38)). Twenty-six items, 13 devoted to statistics and 13 to clinical epidemiology, successfully fit a two-parameter logistic IRT model. These estimates also significantly correlated with their comparable CTT values.
The strength of the BACES instrument was supported by (1) establishing content validity evidence; (2) fitting a sample of 147 residents' responses to an IRT model; and (3) correlating the IRT estimates with their CTT values, which makes it a flexible yet rigorous instrument for measuring biostatistical and clinical epidemiological knowledge.
Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
In order to best care for women, obstetrician-gynecologists (ob-gyns) must be able to diagnose and treat sexually transmitted infections (STIs), as well as effectively communicate risks regarding STIs. This article provides a narrative review of studies primarily conducted by the Research Department at the American College of Obstetricians and Gynecologists, about missed opportunities for STI risk communication. Missed opportunities include the omission or partial completion of STI risk assessment and counseling, failure to offer screening or testing, lack of follow-up on STI testing/vaccination refusals, and a failure to comply with existing guidelines. We also discuss knowledge level, time constraints, and gaps in statistical literacy as barriers to STI communication. The aim of this article is to highlight common barriers to risk communication, discuss their potential impact, and suggest means by which these obstacles can be addressed. Future directions for training, education, and research are discussed.
Summary
Evidence about the benefits and harms of health care interventions has the potential to improve quality of care. Syntheses of the best available evidence - systematic reviews - can provide valuable information for health care clinicians and policy makers, but these publications are often too lengthy to be useful for decision makers in time-pressed contexts. Condensed summaries of reviews may help, but there is little research about how to design such summaries (what content to include and how to present it), how they will be experienced by users, and their effect. Earlier research to create summaries of Cochrane Reviews has also shown that condensing complex data can be challenging and lead to misrepresentation.
This thesis builds on a series of studies examining these issues from several perspectives, with the designed artifact as the main focal point. An underlying assumption for all of this work is that the design of artifacts in which evidence is embedded may shape or influence users’ experiences of this evidence. Given that these artifacts are, in effect, mediating between the domains of evidence production and evidence use (by policymakers, clinicians or the public), they warrant far more attention that they appear to have received to date.
Multiple methods are employed, including a framework from design practice domain. The first study is a set of user tests, examining health professionals’ user experience of the Cochrane Library (where Cochrane Reviews are published). The second study involves exploring user and stakeholder feedback to inform iterations of a Summary of Findings Table for Cochrane Reviews. The third study is an evaluation of the effect of including a Summary of Findings Table in a Cochrane Review (compared to a Review with no table) on user satisfaction, understanding and time spent to find key messages. The fourth study explores user and stakeholder feedback to inform the development of a template for short summaries of systematic reviews that are tailored for health policy makers in low and middle-income countries.
Findings from the user feedback uncovered many comprehension problems, for instance:
• Health professionals and policy makers may not know what a systematic review is or may confuse it with another type of report (e.g. single trial). This can lead to unrealistic expectations or critical misunderstanding of content.
• Unfamiliar language (both foreign language and jargon, abbreviations and unfamiliar terms) may lead to frustration and alienation, or cause critical barriers while reading and searching. Non-native English speakers are at an extra disadvantage.
• Outcome effects may be misunderstood when presented as absolute differences in a table; the use of absolute risks can solve this problem.
• Outcome effects presented as continuous outcomes are often difficult to understand when the scales that are used are unfamiliar.
However, in two small trials we demonstrated that correct comprehension of the main results in a Cochrane Review were improved by including a Summary of Findings Table (in comparison to a review with no table). These trials also showed that key findings were quicker to find in a review with a table.
User studies also helped us understand how summaries of evidence from reviews could be made more useful:
• The layering of the information, through use of a graded entry format, is well-received by policy makers
• Making summaries useful for policy makers also includes broadening the scope of information, such as adding author’s comments about applicability or a wider range of references, as well as information about the information.
• Making summaries useful for clinicians includes better descriptions of interventions and risk groups.
One of the main challenges we encountered in development work was resolving the tension between stakeholders’ concern for precision in data presentation and users’ needs for simplicity. This can be reframed in light of fuzzy traces theory, where people are seen to process information along a gist-verbatim continuum, with a gist preference. Artifacts presenting evidence should be designed with the aim of enabling easy gist extraction.
Drawing on findings across the studies, I evaluated the suitability of the user experience framework we employed and present a revised version. I suggest that user experience frameworks may need to be tailored for specific use contexts and product types.
At the end of the thesis I reflect on my experiences as a designer participating in this work. I discuss, among other things, the challenges to designer role and identity – such as when design-related tasks are increasingly shared with non-designers or when the product disappears out of sight. The results of design processes of complex digital information may not be strikingly visible, but rather subtle, hard to freeze in time and claim ownership to. I propose a move towards redefining design as more than a “reflective conversation with the materials”, and discuss how legitimate output from design should not be limited to the “things” that we make, but also include knowledge gained underway.
Research training has enabled academic clinicians to contribute significantly to the body of medical research literature. Biostatistics represents a critical methodological skill for such researchers, as statistical methods are increasingly a necessary part of medical research. However, there is no validated knowledge and skills assessment for graduate level biostatistics for academic medical researchers. In this paper, I review graduate level statistical competencies and existing instruments intended to assess physicians' ability to read the medical literature and for undergraduate statistics for their alignment with core competencies necessary for successful use of statistics. This analysis shows a need for a new instrument to assess biostatistical competencies for medical researchers.
To measure the effects of a summary-of-findings (SoF) table on user satisfaction, understanding, and time spent finding key results in a Cochrane review.
We randomized participants in an evidence-based practice workshop (randomized controlled trial [RCT] I) and a Cochrane Collaboration entities meeting (RCT II) to receive a Cochrane review with or without an SoF table. In RCT I, we measured user satisfaction. In RCT II, we measured correct comprehension and time spent finding key results.
RCT I: Participants with the SoF table (n=47) were more likely to "agree" or "strongly agree" that it was easy to find results for important outcomes than (n=25) participants without the SoF table-68% vs. 40% (P=0.021). RCT II: Participants with the SoF table (n=18) were more likely to correctly answer two questions regarding results than (n=15) those without the SoF table: 93% vs. 44% (P=0.003) and 87% vs. 11% (P<0.001). Participants with the SoF table spent an average of 90 seconds to find key information compared with 4 minutes for participants without the SoF table (P=0.002).
In two small trials, we found that inclusion of an SoF table in a review improved understanding and rapid retrieval of key findings compared with reviews with no SoF table.
Incorrectly concluding that the probability of a joint event (such as a combination of clinical findings) is greater than the probability of any one of these events alone is known as the conjunction fallacy. It is one type of cognitive error in estimation of probability to which physicians are known to be prone. The purpose of this study was to determine whether beginning medical students are also prone to the conjunction fallacy.
A total of 134 beginning medical students completed a written exercise in which they assigned percent probabilities to symptoms and combinations of symptoms based on a case vignette of a young woman with the common cold. Rates of violation of the conjunction rule were calculated for the entire sample and calculated and compared for the subsets of men, women, and those with different prior educational and statistics backgrounds.
The conjunction rule was violated by 47.8% of the students. Rates of violation did not differ significantly between men and women, among students with different types of prior education, or between students who had and had not taken at least one statistics course.
A significant proportion of medical students were prone to the conjunction fallacy. This proportion, however, was lower than that reported previously for practicing physicians, who may rely more upon "representativeness" to make probability judgments. Teaching medical students about the conjunction fallacy has the potential to reduce the predisposition to this error that may develop during medical training.
Consumers of research (researchers, administrators, educators and clinicians) frequently use standard critical appraisal tools to evaluate the quality of published research reports. However, there is no consensus regarding the most appropriate critical appraisal tool for allied health research. We summarized the content, intent, construction and psychometric properties of published, currently available critical appraisal tools to identify common elements and their relevance to allied health research.
A systematic review was undertaken of 121 published critical appraisal tools sourced from 108 papers located on electronic databases and the Internet. The tools were classified according to the study design for which they were intended. Their items were then classified into one of 12 criteria based on their intent. Commonly occurring items were identified. The empirical basis for construction of the tool, the method by which overall quality of the study was established, the psychometric properties of the critical appraisal tools and whether guidelines were provided for their use were also recorded.
Eighty-seven percent of critical appraisal tools were specific to a research design, with most tools having been developed for experimental studies. There was considerable variability in items contained in the critical appraisal tools. Twelve percent of available tools were developed using specified empirical research. Forty-nine percent of the critical appraisal tools summarized the quality appraisal into a numeric summary score. Few critical appraisal tools had documented evidence of validity of their items, or reliability of use. Guidelines regarding administration of the tools were provided in 43% of cases.
There was considerable variability in intent, components, construction and psychometric properties of published critical appraisal tools for research reports. There is no "gold standard' critical appraisal tool for any study design, nor is there any widely accepted generic tool that can be applied equally well across study types. No tool was specific to allied health research requirements. Thus interpretation of critical appraisal of research reports currently needs to be considered in light of the properties and intent of the critical appraisal tool chosen for the task.
A multiple choice test with nine statistical questions was sent to a random sample of Danish doctors to assess their knowledge of elementary statistical expressions (SD, SE, p < 0·05, p > 0·05 and r). One hundred and forty eight (59 per cent) of 250 doctors answered the questions. The test was also completed by 97 participants in postgraduate courses in research methods, mainly junior hospital doctors. The median number of correct answers was 2·4 in the random sample and 4·0 in the other sample of doctors. It is concluded that the statistical knowledge of most doctors is so limited that they cannot be expected to draw the right conclusions from those statistical analyses which are found in papers in medical journals. Sixty-five per cent of the doctors in the random sample stated that it is very important that this problem is raised.
A Statistical Skills Self-Assessment Questionnaire (SAQ) was developed using hypothetical clinical questions to explore respondents' mastery of vocabulary and rules of inference that seem relevant to the use of quantitative information. The SAQ was administered to 281 subjects, including 36 medical students, 45 interns and residents, 49 physicians engaged in research and 151 physicians in full-time practice. All groups of subjects showed frequent lack of consensus on the meaning of terms in common use (e.g., "false-positive rate" and "p values") and unfamiliarity with some important principles in quantitative inference (e.g., the Central Limit Theorem and Regression to the Mean). Subjects often seemed willing to draw conclusions unsupported by available data. Performance on the SAQ was inversely correlated with length of time since graduation from medical school, and practicing physicians tended to err more frequently than the other three groups.
A review was conducted of 585 articles in the medical literature, and the articles were categorized by study design and by statistical techniques employed. In the majority, statistics were utilized primarily for data description and hypothesis-testing. Descriptive studies were most frequent, and only a minority were epidemiological investigations. A questionnaire designed to assess knowledge of biostatistics and epidemiology relevant to this literature was administered to internists and medical house staff at a teaching hospital. Of 229 questionnaire recipients, 141 responded and achieved a mean score of 7.4 with 10.0 possible. Practicing physicians had a significantly lower mean score than house staff and full-time faculty. Scores were higher for the 44 percent of respondents who reported prior training in epidemiology or in biostatistics. The results of the journal review suggest that knowledge of basic biostatistics and of study design is important for reading medical literature. Physicians with prior training in epidemiology and in biostatistics appear to be better prepared for this task.
To review the goals, organization, and teaching methods of journal clubs, summarize elements of successful clubs, and evaluate their effect on reading habits, and effectiveness in meeting teaching goals. Examples of clubs that utilize principles of adult learning are reviewed.
English language articles identified through a MEDLINE search (1966-1997) using the MeSH terms "internship" and "residency," and text words "journal club" and "critical appraisal."
Articles on learning goals and organization were included if they represented national or regional surveys with a response rate of 65% or greater. Articles that evaluated teaching effectiveness were included if they used a controlled, educational design, or if they exemplified important adult learning principles.
Data were manually extracted from selected studies and reviews.
A major goal for most clubs is to teach critical appraisal skills. Clubs with high attendance and longevity are characterized by mandatory attendance, availability of food, and perceived importance by the program director. Residents who are taught critical appraisal report paying more attention to the methods and are more skeptical of the conclusions, and have increased knowledge of clinical epidemiology and biostatistics, but studies have failed to demonstrate that these residents read more, or read more critically. Reading guidelines may be useful for teaching critical appraisal skills, and may be associated with increased resident satisfaction.
Journal club formats are educationally diverse, can incorporate adult learning principles, and are an adaptable format for teaching the "new basic sciences."
To systematically review the published literature on graduate medical education (GME) curricula in clinical epidemiology, critical appraisal, and evidence-based medicine (EBM).
The author searched the Medline and Educational Resources Information Center (ERIC) databases from 1973 through 1998, and also searched the references of the captured papers. The author reviewed all peer-reviewed reports of GME curricula (with or without effectiveness studies) in critical appraisal, clinical epidemiology, or evidence-based medicine, extracting objectives, formats, and evaluations (including effectiveness, process, and satisfaction). For effectiveness evaluations, he also identified the outcomes, outcome measures, methodologic characteristics, and results.
The search produced 18 reports. The most common objective of the curricula described in the reports was improving critical skills; the most common format was resident-directed small-group seminar. The most common outcome-evaluation measure was a multiple-choice examination. Only seven of the reports evaluated the curricula's effectiveness, and only four met a minimum methodologic standard of a pretest-posttest controlled trial. The impacts on critical appraisal skills of the curricula in those four reports ranged from no effect to a 23% net absolute increase in test scores.
These reports provide useful guides for medical educators, but many suffered from incomplete descriptions and inadequate evaluations of their curricula. The curricula themselves often focused on critical appraisal to the exclusion of other EBM skills and had limited effectiveness. In addition to increased methodologic rigor, future studies should focus on more meaningful outcome evaluations. Curricula should use residents' actual clinical experiences and teach EBM skills in real time in existing clinical and educational venues.
Medical students must learn the principles of epidemiology and biostatistics to critically evaluate the medical literature. However, this subject has traditionally been difficult to teach. In 1997 at the Albert Einstein College of Medicine, the required first-year course in epidemiology and biostatistics was revised to use the case-discussion teaching method. In preparation for the course, experienced faculty participated in an intensive, two-day training workshop. The course, taught to 163 first-year medical students, was structured in two parts: (1) three lectures complemented by a detailed syllabus, followed by a multiple-choice midterm exam; and (2) six case-discussion seminars, followed by a short answer/essay final exam. There were seven case-discussion groups with 23-24 students each. The program was evaluated using subjective faculty feedback, examination scores, and student evaluation questionnaires. Faculty noted excellent student preparation and participation. Multiple-choice exam scores were comparable to those from earlier years, and a short answer/essay exam demonstrated good student mastery of the required material. Student evaluation was overwhelmingly positive, and significantly improved from prior years of the course. Positive student evaluations of the course using this teaching method continued over the next four years; National Board of Medical Examiners examination scores indicated success in mastery of the material; and student assessment of the course improved on the AAMC Graduation Questionnaire. This favorable experience suggests that case-discussion teaching can be employed successfully in teaching principles of epidemiology and biostatistics to medical students.
Incorporating evidence-based medicine (EBM) into clinical practice is an important competency that residency training must address. Residency program directors, and the clinical educators who work with them, should develop curricula to enhance residents' capacity for independent evidence-based practice. In this article, the authors argue that residency programs must move beyond journal club formats to promote the practice of EBM by trainees. The authors highlight the limitations of journal club, and suggest additional curricular approaches for an integrated EBM curriculum. Helping residents become effective evidence users will require a sustained effort on the part of residents, faculty, and their educational institutions.
To determine the types of information sources that evidence-based medicine (EBM)-trained, family medicine residents use to answer clinical questions at the point of care, to assess whether the sources are evidence-based, and to provide suggestions for more effective information-management strategies in residency training.
In 2005, trained medical students directly observed (for two half-days per physician) how 25 third-year family medicine residents retrieved information to answer clinical questions arising at the point of care and documented the type and name of each source, the retrieval location, and the estimated time spent consulting the source. An end-of-study questionnaire asked 37 full-time faculty and the participating residents about the best information sources available, subscriptions owned, why they use a personal digital assistant (PDA) to practice medicine, and their experience in preventing medical errors using a PDA.
Forty-four percent of questions were answered by attending physicians, 23% by consulting PDAs, and 20% from books. Seventy-two percent of questions were answered within two minutes. Residents rated UptoDate as the best source for evidence-based information, but they used this source only five times. PDAs were used because of ease of use, time factors, and accessibility. All examples of medical errors discovered or prevented with PDA programs were medication related. None of the participants' residencies required the use of a specific medical information resource.
The results support the Agency for Health Care Research and Quality's call for medical system improvements at the point of care. Additionally, it may be necessary to teach residents better information-management skills in addition to EBM skills.
Physicians depend on the medical literature to keep current with clinical information. Little is known about residents' ability to understand statistical methods or how to appropriately interpret research outcomes.
To evaluate residents' understanding of biostatistics and interpretation of research results.
Multiprogram cross-sectional survey of internal medicine residents.
Percentage of questions correct on a biostatistics/study design multiple-choice knowledge test.
The survey was completed by 277 of 367 residents (75.5%) in 11 residency programs. The overall mean percentage correct on statistical knowledge and interpretation of results was 41.4% (95% confidence interval [CI], 39.7%-43.3%) vs 71.5% (95% CI, 57.5%-85.5%) for fellows and general medicine faculty with research training (P < .001). Higher scores in residents were associated with additional advanced degrees (50.0% [95% CI, 44.5%-55.5%] vs 40.1% [95% CI, 38.3%-42.0%]; P < .001); prior biostatistics training (45.2% [95% CI, 42.7%-47.8%] vs 37.9% [95% CI, 35.4%-40.3%]; P = .001); enrollment in a university-based training program (43.0% [95% CI, 41.0%-45.1%] vs 36.3% [95% CI, 32.6%-40.0%]; P = .002); and male sex (44.0% [95% CI, 41.4%-46.7%] vs 38.8% [95% CI, 36.4%-41.1%]; P = .004). On individual knowledge questions, 81.6% correctly interpreted a relative risk. Residents were less likely to know how to interpret an adjusted odds ratio from a multivariate regression analysis (37.4%) or the results of a Kaplan-Meier analysis (10.5%). Seventy-five percent indicated they did not understand all of the statistics they encountered in journal articles, but 95% felt it was important to understand these concepts to be an intelligent reader of the literature.
Most residents in this study lacked the knowledge in biostatistics needed to interpret many of the results in published clinical research. Residency programs should include more effective biostatistics training in their curricula to successfully prepare residents for this important lifelong learning skill.
Many physicians admit to having some degree of innumeracy--difficulty understanding and working with numbers. Yet, increasingly, physicians in all specialties are committing to practicing evidence-based medicine (EBM) and, as a result, must learn to discern quantitative differences and address statistical significance. Although no one expects a practicing physician to be able to evaluate a statistician's choice of methods or conduct an independent rendering of a clinical study's analysis, practitioners of EBM must learn how to assess the importance of results found in a clinical study. Since 2001, Mayo Clinic has been teaching its residents in pediatric and adolescent medicine the skills required for EBM. This article describes the 5 steps involved in practicing EBM, focusing on the interpretation of study results.
Statistics at square one. www.bmj.com/statsbk
- Tdv Swinscow
Swinscow TDV. Statistics at square one.
www.bmj.com/statsbk/. Accessed January
22, 2008.
Chief residents' skills with evidence-based medicine: still hazy after all these years
- R Watkins
- Wc Wilson
- S Richardson
Watkins R, Wilson WC, Richardson S.
Chief residents' skills with evidence-based
medicine: still hazy after all these years.
Users' guides to the medical literature: a manual for evidencebased clinical practice
- G Guyatt
- D Rennie
Guyatt G, Rennie D, eds. Users' guides to
the medical literature: a manual for evidencebased clinical practice. Chicago: AMA Press,
2002.