Article

Increasing Understanding of Diagnostic Test Outcomes

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

Available at: http://journals.sagepub.com/doi/full/10.1177/0272989X18758293 Background: Understanding diagnostic test outcomes requires determining the positive predictive value (PPV) of the test, which most laypeople and medical professionals struggle to do. Despite advances found with frequency formats and visual aids, less than 40% of people can typically identify this value. This study tests the impact of using congruent reference classes in problem-question pairings, evaluates the role of numeracy, and assesses how diagnostic value estimates affect reported likelihood-to-use the test. Method: A 3 x 2, Pairing (congruent test-focus, congruent condition-focus, incongruent) x Response Format (frequency, percentage) mixed design experiment was conducted in which participants answered diagnostic questions about seven medical problems presented in a format focusing either on the reference class of those who test positive or those who have the condition. Answer accuracy, numeracy, and ratings of likelihood to use estimates were assessed. Results: Focusing on the congruent test reference class allowed 87% of participants to consistently identify the PPV, and focusing on the congruent condition reference class led 63% of participants to consistently identify the sensitivity (SEN). Aligning reference classes was especially beneficial for those with lower numeracy, increasing accuracy on problems from 21% for incongruent pairings to 66% for congruent pairings. Ratings of likelihood to use the test were closely tied to participants’ estimates of diagnostic values, regardless of the accuracy of those estimates. Conclusions: Although often overlooked, a straightforward mapping of reference classes from the relevant diagnostic information to the question of interest reduces confusion and substantially increases accuracy in estimates of diagnostic values. These findings can be used to strengthen training in the assessment of uncertainties associated with medical test results.
Content may be subject to copyright.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Three explanations have been advanced to explain the most popular error on probabilistic inference word problems, which is the use of the reliability of the evidence p(E/H) as the response when p(H/E) is requested. Production system simulations of each explanation are applied to data from a study in which 265 subjects judged the probability of an hypothesis after receiving each of three pertinent pieces of information—the evidence, the baserate, and the reliability of the evidence—in all possible combinations. The explanation that subjects consider the baserate to be irrelevant is rejected. Data are consistent with both the explanation that 265 subjects confuse p(E/H) with p(H/E) and the explanation that they interpolate between the baserate probability and 1.0 and then select their response from among nearby numbers that are available in the word problem.
Article
Full-text available
The number and complexity of clinical laboratory tests is rapidly expanding, presenting primary care physicians with challenges in accurately, efficiently, and safely ordering and interpreting diagnostic tests. The objective of this study was to identify challenges primary care physicians face related to diagnostic laboratory testing and solutions they believe are helpful and available to them. In this study, sponsored by the Centers for Disease Control and Prevention, a random sample of general internal medicine and family medicine physicians from the American Medical Association Masterfile were surveyed in 2011. 1768 physicians (5.6%) responded to the survey. Physicians reported ordering diagnostic laboratory tests for an average of 31.4% of patient encounters per week. They reported uncertainty about ordering tests in 14.7% and uncertainty in interpreting results in 8.3% of these diagnostic encounters. The most common problematic challenges in ordering tests were related to the cost to patients and insurance coverage restrictions. Other challenges included different names for the same test, tests not available except as part of a test panel, and different tests included in panels with the same names. The most common problematic challenges in interpreting and using test results were not receiving the results and confusing report formats. Respondents endorsed a variety of information technology and decision support solutions to improve test selection and results interpretation, but these solutions were not widely available at the time of the survey. Physicians infrequently sought assistance or consultation from laboratory professionals but valued these consultations when they occurred. Primary care physicians routinely experience uncertainty and challenges in ordering and interpreting diagnostic laboratory tests. With more than 500 million primary care patient visits per year, the level of uncertainty reported in this study potentially affects 23 million patients per year and raises significant concerns about the safe and efficient use of laboratory testing resources. Improvement in information technology and clinical decision support systems and quick access to laboratory consultations may reduce physicians' uncertainty and mitigate these challenges.
Article
Full-text available
Objectives: The natural frequency effect (NF effect)-whereby framing health risks using information presented as natural frequencies (NFs), instead of conditional probabilities (CPs), results in improved diagnostic problem solving-has led to the recommended use of NFs in clinical practice. This experiment tests, via incentivization of a lab-based decision, the hypothesis that the NF effect reflects differential motivation applied to solving problems that differ in complexity. The study examines if incentive effects are moderated by task complexity and expertise and also explores the extent to which NF frames improve diagnostic understanding. Methods: Three-hundred and 25 participants (235 novices and 90 medical experts) were randomly allocated to a frame (NF vs. CP) by task difficulty (short vs. standard menus) by incentive (present vs. absent) between-subjects design. The task was to calculate the positive predictive value (PPV) of the hemoccult test for colorectal cancer. Effort, self-efficacy, and diagnostic understanding were assessed. Results: Incentives increased correct problem solving and the NF effect was replicated. The NF effect did not vary as a function of incentivization, but was slightly attenuated by task complexity and expertise. There was no evidence that effort mediated the incentive effect. The correct PPV (which is low) was associated with reduced trust in the test's diagnostic accuracy. For those who committed errors, NF frames increased the likelihood of underestimating the PPV, with underestimation associated with greater trust in the test. Conclusions: The NF effect is robust to incentives supporting the use of NF frames in clinical settings. When errors occur, however, NF frames are linked to underestimation.
Article
Full-text available
A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.
Article
Full-text available
The presentation of a Bayesian inference problem in terms of natural frequencies rather than probabilities has been shown to enhance performance. The effect of individual differences in cognitive processing on Bayesian reasoning has rarely been studied, despite enabling us to test process-oriented variants of the two main accounts of the facilitative effect of natural frequencies: The ecological rationality account (ERA), which postulates an evolutionarily shaped ease of natural frequency automatic processing, and the nested sets account (NSA), which posits analytical processing of nested sets. In two experiments, we found that cognitive reflection abilities predicted normative performance equally well in tasks featuring whole and arbitrarily parsed objects (Experiment 1) and that cognitive abilities and thinking dispositions (analytical vs. intuitive) predicted performance with single-event probabilities, as well as natural frequencies (Experiment 2). Since these individual differences indicate that analytical processing improves Bayesian reasoning, our findings provide stronger support for the NSA than for the ERA.
Article
Full-text available
There is concern across a range of healthcare settings worldwide that trust in physicians is declining. Decreased trust may lead to lesser tolerance of prognosis uncertainty and an increased demand for tests, referrals and second opinions. Literature suggests that there has been a recent cultural shift towards decreased trust in, and increased questioning of, medical advice. We investigated the impact of varying prognosis and socioeconomic status (SES) on trust in physicians, and patient questioning of medical advice. Semistructured, audio-recorded transcribed interviews were conducted. The interview schedule was developed with reference to the Health Belief Model. Interviews were conducted between October 2008 and September 2009. Participants were recruited via general practitioner clinics and hospital cardiac rehabilitation programmes. Participants consisted of patients either receiving preventive treatment or active treatment for established cardiovascular disease. A coding structure was developed based on the aim of the research, to investigate the impact of varying prognosis and SES on trust in physicians. Older participants are more likely than their younger counterparts to be unquestioning of medical advice. Higher SES participants are more likely to question medical advice than lower SES participants. Also, unlike primary prevention participants, established pathology increased participants' trust, or decreased questioning behaviour. Participants who perceived themselves at risk of a poor or uncertain outcome were unlikely to doubt medical advice. Blind trust in physicians remains strong in older participants, participants who perceive their prognosis to be uncertain and a proportion of lower SES participants. This is important for practitioners in terms of patient agency and points to the importance of moral and ethical practice. However, physicians also need to be aware that there are a growing proportion of patients for whom trust needs to be developed, and cannot be assumed.
Article
Full-text available
Professional probabilists have long argued over what probability means, with, for example, Bayesians arguing that probabilities refer to subjective degrees of confidence and frequentists arguing that probabilities refer to the frequencies of events in the world. Recently, Gigerenzer and his colleagues have argued that these same distinctions are made by untutored subjects, and that, for many domains, the human mind represents probabilistic information as frequencies. We analyze several reasons why, from an ecological and evolutionary perspective, certain classes of problem-solving mechanisms in the human mind should be expected to represent probabilistic information as frequencies. Then, using a problem famous in the “heuristics and biases” literature for eliciting base rate neglect, we show that correct Bayesian reasoning can be elicited in 76% of subjects - indeed, 92% in the most ecologically valid condition - simply by expressing the problem in frequentist terms. This result adds to the growing body of literature showing that frequentist representations cause various cognitive biases to disappear, including overconfidence, the conjunction fallacy, and base-rate neglect. Taken together, these new findings indicate that the conclusion most common in the literature on judgment under uncertainty - that our inductive reasoning mechanisms do not embody a calculus of probability - will have to be re-examined. From an ecological and evolutionary perspective, humans may turn out to be good intuitive statisticians after all.
Article
Full-text available
Mammography is the only breast screening method, we are aware of today, which is able to reduce mortality from breast cancer. Nevertheless, this procedure carries an inherent risk of false-positive screening mammogram. The association between these results and reattendance at the next scheduled screening mammogram is controversial. The aim of this study was to examine the effect of a false-positive screening mammogram and women's characteristics on reattendance in eight regional population-based breast cancer screening programmes in Spain. This study included 1 383 032 women aged 44-67 years who were initially screened for breast cancer between 1990 and 2004. To investigate factors associated with reattendance, logistic regression models were used. The mean age of women at first screening was 53.6 years (SD = 6.1 years). Of 120 800 women with a false-positive screening mammogram, 78.3% returned for a subsequent screening mammogram compared with 81.9% of those with a negative result (P < 0.001). Multivariate analysis showed that women with a false-positive result at first screening mammogram were less likely to reattend (OR = 0.71; 95% CI 0.70-0.73) and that the likelihood was lower in those who had undergone invasive additional tests (OR = 0.56; 95% CI 0.53-0.59). A false-positive screening mammogram in the first screening negatively affected attendance at the subsequent screening. The results of this study could be useful to improve the screening process and to increase women's compliance.
Article
Full-text available
Is the mind, by design, predisposed against performing Bayesian inference? Previous research on base rate neglect suggests that the mind lacks the appropriate cognitive algorithms. However, any claim against the existence of an algorithm, Bayesian or otherwise, is impossible to evaluate unless one specifies the information format in which it is designed to operate. The authors show that Bayesian algorithms are computationally simpler in frequency formats than in the probability formats used in previous research. Frequency formats correspond to the sequential way information is acquired in natural sampling, from animal foraging to neural networks. By analyzing several thousand solutions to Bayesian problems, the authors found that when information was presented in frequency formats, statistically naive participants derived up to 50% of all inferences by Bayesian algorithms. Non-Bayesian algorithms included simple versions of Fisherian and Neyman–Pearsonian inference.
Article
Full-text available
We review the growing literature on health numeracy, the ability to understand and use numerical information, and its relation to cognition, health behaviors, and medical outcomes. Despite the surfeit of health information from commercial and noncommercial sources, national and international surveys show that many people lack basic numerical skills that are essential to maintain their health and make informed medical decisions. Low numeracy distorts perceptions of risks and benefits of screening, reduces medication compliance, impedes access to treatments, impairs risk communication (limiting prevention efforts among the most vulnerable), and, based on the scant research conducted on outcomes, appears to adversely affect medical outcomes. Low numeracy is also associated with greater susceptibility to extraneous factors (i.e., factors that do not change the objective numerical information). That is, low numeracy increases susceptibility to effects of mood or how information is presented (e.g., as frequencies vs. percentages) and to biases in judgment and decision making (e.g., framing and ratio bias effects). Much of this research is not grounded in empirically supported theories of numeracy or mathematical cognition, which are crucial for designing evidence-based policies and interventions that are effective in reducing risk and improving medical decision making. To address this gap, we outline four theoretical approaches (psychophysical, computational, standard dual-process, and fuzzy trace theory), review their implications for numeracy, and point to avenues for future research.
Article
Full-text available
We studied patients' understanding of the characteristics of diagnostic tests for six common conditions to determine what patients know about diagnostic uncertainties before they communicate with a doctor. We compared the accuracy of patients' estimates of disease probabilities and diagnostic test characteristics for diseases with which they did or did not have prior experience. To measure patients' understanding of the uncertainty of diagnostic test results, questionnaires describing diseases were given to patients in clinic waiting rooms. For each of six diseases, a 2-page questionnaire presented a case history of the disease and its diagnostic test, and asked respondents to estimate the probability that the case patient has the suspected disease, the sensitivity of test, the specificity of test, and the probability that the patient has the disease if the test result is positive. It also asked whether the patient, a close friend, or family member had ever been thought to have this disease. One hundred eighty-four patients in the clinic waiting room responded for at least one disease. Although patients judged the disease probabilities to be higher after a positive diagnostic test, each of their four judgments was essentially the same for all diseases, including those with high and low prior probabilities, and with accurate and inaccurate tests. Past experience with the disease was associated with only a minimal increase in the accuracy of patient knowledge. Patients' ignorance of the uncertainties of diseases demonstrates the need for patient education when a disease is suspected. The lack of relation between knowledge and experience suggests that this need is not being effectively met. To convey the rates or probabilities, and to help the patients understand what the probabilities are based on, a physician should speak in terms that patients can easily understand.
Article
Full-text available
This article outlines a theory of naive probability. According to the theory, individuals who are unfamiliar with the probability calculus can infer the probabilities of events in an extensional way: They construct mental models of what is true in the various possibilities. Each model represents an equiprobable alternative unless individuals have beliefs to the contrary, in which case some models will have higher probabilities than others. The probability of an event depends on the proportion of models in which it occurs. The theory predicts several phenomena of reasoning about absolute probabilities, including typical biases. It correctly predicts certain cognitive illusions in inferences about relative probabilities. It accommodates reasoning based on numerical premises, and it explains how naive reasoners can infer posterior probabilities without relying on Bayes's theorem. Finally, it dispels some common misconceptions of probabilistic reasoning.
Article
Full-text available
Numeracy, how facile people are with basic probability and mathematical concepts, is associated with how people perceive health risks. Performance on simple numeracy problems has been poor among populations with little as well as more formal education. Here, we examine how highly educated participants performed on a general and an expanded numeracy scale. The latter was designed within the context of health risks. A total of 463 men and women aged 40 and older completed a 3-item general and an expanded 7-item numeracy scale. The expanded scale assessed how well people 1) differentiate and perform simple mathematical operations on risk magnitudes using percentages and proportions, 2) convert percentages to proportions, 3) convert proportions to percentages, and 4) convert probabilities to proportions. On average, 18% and 32% of participants correctly answered all of the general and expanded numeracy scale items, respectively. Approximately 16% to 20% incorrectly answered the most straightforward questions pertaining to risk magnitudes (e.g., Which represents the larger risk: 1%, 5%, or 10%?). A factor analysis revealed that the general and expanded risk numeracy items tapped the construct of global numeracy. These results suggest that even highly educated participants have difficulty with relatively simple numeracy questions, thus replicating in part earlier studies. The implication is that usual strategies for communicating numerical risk may be flawed. Methods and consequences of communicating health risk information tailored to a person's level of numeracy should be explored further.
Article
Full-text available
How likely is a diagnosis, given a particular medical test result? This probability can be determined by using Bayes's rule; however, previous research has shown that doctors often experience problems with Bayesian inferences. These findings illustrate the need to teach statistical reasoning in medical education. A new method of teaching Bayesian reasoning is representation learning: the key idea is to instruct medical students how to translate probability information into a representation that is easier to process, namely natural frequencies. This approach was implemented in a one-hour classroom tutorial to evaluate its effectiveness in this setting and compared with a traditional rule-learning approach. Evaluation took place two months after training by testing students' ability to correctly solve a Bayesian inference task with information represented as probabilities. While both approaches improved performance, almost three times as many students were able to profit from representation training as opposed to rule training.
Article
Full-text available
Current health care policy emphasizes improving health outcomes and the efficacy of health care delivery by supporting informed consumer choices. At the same time, health information often involves uncertainty, and many people may lack the skills and knowledge to process this information, manage their health and health care, and make informed choices. Innumeracy, an element of poor health literacy, is associated with the comprehension and use of important health information. We review this literature and examine what can be done to help less numerate people act more effectively and take charge of their health.
Article
Full-text available
Basic numeracy skills are necessary before patients can understand the risks of medical treatments. Previous research has used objective measures, similar to mathematics tests, to evaluate numeracy. To design a subjective measure (i.e., self-assessment) of quantitative ability that distinguishes low- and high-numerate individuals yet is less aversive, quicker to administer, and more usable for telephone and Internet surveys than existing numeracy measures. Paper-and-pencil questionnaires. The general public (N = 703) surveyed at 2 hospitals. Forty-nine subjective numeracy questions were compared to measures of objective numeracy. An 8-item measure, the Subjective Numeracy Scale (SNS), was developed through several rounds of testing. Four items measure people's beliefs about their skill in performing various mathematical operations, and 4 measure people's preferences regarding the presentation of numerical information. The SNS was significantly correlated with Lipkus and others' objective numeracy scale (correlations: 0.63-0.68) yet was completed in less time (24 s/item v. 31 s/item, P < 0.05) and was perceived as less stressful (1.62 v. 2.69, P < 0.01) and less frustrating (1.92 v. 2.88, P < 0.01). Fifty percent of participants who completed the SNS volunteered to participate in another study, whereas only 8% of those who completed the Lipkus and others scale similarly volunteered (odds ratio = 11.00, 95% confidence interval = 2.14-56.65). The SNS correlates well with mathematical test measures of objective numeracy but can be administered in less time and with less burden. In addition, it is much more likely to leave participants willing to participate in additional research and shows much lower rates of missing or incomplete data.
Article
Shared decision making occurs when patients and clinicians reach a formulation about the presenting problem and discuss how to manage it. If there are several reasonable alternatives, the alternatives should be explicitly compared, using evidence about relevant harms and benefits. Such decisions should be informed by knowledge about the patients’ condition, about the evidence applicable to it, and the patient's goals and preferences. Eliciting patients’ views has been referred to as making a preference diagnosis,¹ and eliciting it requires a blend of science and interpersonal skills.
Article
Imagine that you have received a positive result on a routine cancer screening test. Follow-up biopsies were inconclusive, and the decision to treat aggressively or monitor conservatively is yours. Consider the following information: 0.1% of the population has a terminal version of this form of cancer, 99% of those people will appear positive on the test you have been administered, and 5% of those without terminal cancer will still have a benign condition that tests positive. Given your test result, what is the probability that you have terminal cancer and should treat it aggressively? When judging risks and trying to predict the future, how should you decide? We need to better understand the structure of risks and how the human mind creates representations of risk and probability.
Article
Available at http://onlinelibrary.wiley.com/doi/10.1002/bdm.1949/abstract Research suggests that most people struggle when asked to interpret the outcomes of diagnostic tests such as those presented as Bayesian inference problems. To help people interpret these difficult problems, we created a brief tutorial, requiring less than 10 minutes, that guided participants through the creation of an aid (either graph or table) based on an example inference problem, and then showed the correct way to calculate the positive predictive value of the problem (i.e., likelihood that positive tests correctly indicate presence of condition). Approximately 70% of those in each training condition found the correct response on at least one problem in the format for which they were trained. Just under 55% of those in the control condition (i.e., no training) were able to find the correct response on at least one table or graph problem. We demonstrated a relationship between numeracy and performance on both problem formats, though we did not find evidence for a relationship between graph literacy and performance for either problem format. Potential improvements to and applications of the tutorial are discussed.
Article
The medical profession in the United States, by some measures, has much to be pleased about. Public confidence in the honesty and ethical standards of physicians has remained high for decades, even as other professions, including bankers, the clergy, and members of Congress, have endured a steady erosion in public trust. Even though physicians may not enjoy the exalted, even unquestioned, position they did in the middle of the last century, they remain among the most trusted professionals in the United States.
Article
Background In 2011, 1.3 million sexual health screens (SHS) were conducted in genitourinary medicine clinics across the UK, a doubling of workload in the last 8 years. One approach UK clinicians have adopted in managing this increase is for minimally trained non-medical staff (health care support workers (HCSWs)) to deliver protocol driven asymptomatic screening to low risk patients. There is limited research regarding patient confidence with the service offered by HCSWs and re-attendance rates could be an indicator of patient dissatisfaction. Aim To assess whether patients who have asymptomatic screening with HCSWs were more likely to return for subsequent assessment by a clinician following discharge. Method A case controlled study of 300 asymptomatic patients attending for sexual health screening between October 2011 and April 2012. There were 2 arms with equal patient numbers, the HCSW led clinic and the clinician led clinic. Data collection and analysis for both groups included patient demographics, diagnoses, treatment, test results and time to next new diagnosis. Results No significant difference was found between the patient demographics of the two groups. The rates of Chlamydia infection between the HCSW and the clinician groups were 8% and 7.3% respectively. Within the clinician group 4 cases of syphilis, 1 new diagnosis of HIV and 1 case of Hepatitis C were also diagnosed. The HCSW clinic had 19 (12.7%) patients re-attend for further screening within 11 months compared to 16 (10.7%) patients who saw a clinician, showing no significant difference between the two groups [p = 0.124 Fishers exact test]. Only 1 patient from each group re-attended within 6 weeks for further screening due to the development of symptoms. Conclusion This study highlights low patient re-attendance rates within the HCSW group. This is an indirect marker of patient satisfaction and demonstrates patients are reassured with the service they offer.
Article
Background: Trust in physicians influences the health and well-being of older adults and is an important indicator to assess the quality of medical care. However, Asian aging populations are often underrepresented in studies of patient trust in physicians. This study aims to examine the level of trust in physicians among Chinese older adults in a community-dwelling Chinese aging population. Methods: Data were drawn from the Population Study of Chinese Elderly, a population-based survey of U.S. Chinese older adults in the Greater Chicago area. Guided by a community-based participatory research approach, a total of 3,159 Chinese older adults aged 60 and above were surveyed. An 11-item scale was used to measure participants' trust in physicians. Results: On a scale of 11-55, the level of trust in physician among U.S. Chinese older adults was 42.0 (SD = 6.3). Items related to confidence in physicians' knowledge and skills were most commonly endorsed, including trusting physicians' judgment on medical care (84.8%), trusting physicians' advice (84.2%), and trusting physicians' words that something is so and must be true (81.2%). Younger age, male gender, higher educational level, fewer years of residing in the United States and in the community, poorer self-reported health status, and poorer quality of life were associated with lower level of trust in physicians. Conclusions: Trust in physician is commonly endorsed among U.S. Chinese older adults. However, future longitudinal studies are needed to improve our understanding of risk factors and outcomes associated with trust in physicians among U.S. Chinese older adults.
Article
Research has demonstrated that individual differences in numeracy may have important consequences for decision making. In the present paper, we develop a shorter, psychometrically improved measure of numeracy—the ability to understand, manipulate, and use numerical information, including probabilities. Across two large independent samples that varied widely in age and educational level, participants completed 18 items from existing numeracy measures. In Study 1, we conducted a Rasch analysis on the item pool and created an eight-item numeracy scale that assesses a broader range of difficulty than previous scales. In Study 2, we replicated this eight-item scale in a separate Rasch analysis using data from an independent sample. We also found that the new Rasch-based numeracy scale, compared with previous measures, could predict decision-making preferences obtained in past studies, supporting its predictive validity. In Study, 3, we further established the predictive validity of the Rasch-based numeracy scale. Specifically, we examined the associations between numeracy and risk judgments, compared with previous scales. Overall, we found that the Rasch-based scale was a better linear predictor of risk judgments than prior measures. Moreover, this study is the first to present the psychometric properties of several popular numeracy measures across a diverse sample of ages and educational level. We discuss the usefulness and the advantages of the new scale, which we feel can be used in a wide range of subject populations, allowing for a more clear understanding of how numeracy is associated with decision processes.
Article
When making decisions, people must frequently take into account numerical information, but not all individuals have the ability to understand and use numbers. Less numerate individuals comprehend less numerical information; but numeracy goes beyond comprehension, relating systematically to psychological mechanisms. In particular, greater numeracy has been associated with reduced susceptibility to framing effects, less influence of nonnumerical information such as mood states, and greater sensitivity to different levels of numerical risk. This greater number sensitivity has been linked with number-related affective reactions reported by the highly numerate. I briefly discuss methods to increase number use in decisions and policy implications of numeracy research.
Article
“Numeracy,” so-called on analogy with literacy, is essential for making health and other social judgments in everyday life [Reyna, V. F., & Brainerd, C. J. (in press). The importance of mathematics in health and human judgment: Numeracy, risk communication, and medical decision making. Learning and Individual Differences.]. Recent research on numeracy in health decision making has shown that many adults fail to solve simple ratio and decimal problems, concepts that are prerequisites for understanding health-relevant risk communications. In addition, adults exhibit a ratio bias, in which higher frequencies bias probability judgments, and denominator neglect, described by Reyna and Brainerd (e.g., [Reyna, V. F. (1991). Class inclusion, the conjunction fallacy, and other cognitive illusions. Developmental Review, 11, 317–336.; Reyna, V. F., & Brainerd, C. J. (1994). The origins of probability judgment: A review of data and theories. In G. Wright & P. Ayton (Eds.), Subjective probability. (pp. 239–272). New York: Wiley.]) and independently by Epstein (e.g., [Epstein, S. (1994). Integration of the cognitive and psychodynamic unconscious. American Psychologist, 49, 709–724.]). Along with research in education and cognitive development, this work demonstrates that adults have difficulty with a broad range of ratio concepts, including fractions, proportions, risks and probabilities. The psychological mechanisms underlying this difficulty are characterized using dual-processes approaches such as fuzzy-trace theory, simple and effective interventions are described that eliminate common problem-solving errors, and implications for the effective use of numerical information in risk communication are discussed.
Article
A new algorithm called Mersenne Twister (MT) is proposed for generating uniform pseudorandom numbers. For a particular choice of parameters, the algorithm provides a super astronomical period of 219937 - 1 and 623-dimensional equidistribution up to 32-bit accuracy, while using a working area of only 624 words. This is a new variant of the previously proposed generators, TGFSR, modified so as to admit a Mersenne-prime period. The characteristic polynomial has many terms. The distribution up to ν bits accuracy for 1 ≪ ν ≪ 32 is also shown to be good. An algorithm is also given that checks the primitivity of the characteristic polynomial of MT with computational complexity O(p2) where p is the degree of the polynomial. We implemented this generator in portable C-code. It passed several stringent statistical tests, including diehard. Its speed is comparable to other modern generators. Its merits are due to the efficient algorithms that are unique to polynomial calculations over the two-element field.
Article
The first decade of the patient safety movement achieved some real gains, focused as it was on adverse events amenable to systemwide solutions, such as infections associated with health care and medication errors. However, diagnostic errors, although common and often serious, have not received comparable attention. They are challenging to measure and less amenable to systemwide solutions. Furthermore, it is difficult to hold hospitals accountable, since diagnostic errors usually result from cognitive mistakes on the part of one or more members of the medical staff. Health information technology, better training, and increasing acknowledgment of the problem hold some promise. As approaches to measuring, preventing, and mitigating harm from diagnostic errors are proven to work, it will be important to integrate these approaches into policy initiatives to improve patient safety.
Article
Patients must be informed about risks before any treatment can be implemented. Yet serious problems in communicating these risks occur because of framing effects. To investigate the effects of different information frames when communicating health risks to people with high and low numeracy and determine whether these effects can be countered or eliminated by using different types of visual displays (i.e., icon arrays, horizontal bars, vertical bars, or pies). Experiment on probabilistic, nationally representative US (n = 492) and German (n = 495) samples, conducted in summer 2008. Participants' risk perceptions of the medical risk expressed in positive (i.e., chances of surviving after surgery) and negative (i.e., chances of dying after surgery) terms. Although low-numeracy people are more susceptible to framing than those with high numeracy, use of visual aids is an effective method to eliminate its effects. However, not all visual aids were equally effective: pie charts and vertical and horizontal bars almost completely removed the effect of framing. Icon arrays, however, led to a smaller decrease in the framing effect. Difficulties with understanding numerical information often do not reside in the mind, but in the representation of the problem.
Article
Many people have difficulties grasping numerical concepts that are prerequisites for understanding treatment risk reduction. Visual aids have been proposed as a promising method for enhancing comprehension. In a survey of probabilistic, nationally representative samples in two different countries (United States and Germany), we compared the effectiveness of adding different types of visual aids (icon arrays and bar graphs representing either affected individuals only or the entire population at risk) to the numerical information in either an absolute or a relative risk reduction format. We also analyzed whether people's numeracy and graphical literacy skills affected the efficacy of the visual aids. Our results showed large improvements in accuracy both when icon arrays and when bar graphs were added to numerical information. Highest increases were achieved when the visual aids depicted the entire population at risk. Importantly, visual aids were most useful for the participants who had low numeracy but relatively high graphical literacy skills. Building on previous research showing that problems with understanding numerical information often do not reside in people's minds, but in the representation of the problem, our results show that visual aids help to modify incorrect expectations about treatment risk reduction. Our results have important implications for medical practice.
Article
Understanding information about medical screening tests often requires estimating positive predictive values (i.e., posterior probabilities), which is a notoriously difficult task. Previous studies have shown that representation of information in terms of natural frequencies (i.e., counts of occurrences that preserve base rates) facilitates judgments of positive predictive values. The objective of this study was to investigate whether natural frequencies facilitate accurate estimates in elderly people and whether performance depends on numeracy skills. Elderly people are more often than younger people required to use such information to make informed choices regarding medical procedures (e.g., screenings). This was an experimental study in which information about 2 medical screening tests was presented either as conditional probabilities or natural frequencies. Participants were 47 older adults (62-77 years of age; average numeracy score 8.6) and 115 younger adults (18-35 years of age; average numeracy score 10.3). When the screening information was presented in terms of conditional probabilities, only 15% of the younger adults and 18% of the older adults provided accurate estimates in at least 1 of the tasks. When information was presented in terms of natural frequencies, 55% of the younger adults and 58% of the elderly participants gave correct estimates. This effect occurred without explicit training. Furthermore, participants with higher numeracy scores performed better in the estimation tasks than those with lower numeracy scores. Natural frequencies help elderly and young patients-including those with lower numeracy skills-to understand positive predictive values of medical screening tests.
Article
This article has no abstract; the first 100 words appear below. AS both the number and cost of clinical laboratory tests continue to increase at an accelerating rate,¹ physicians are faced with the task of comprehending and acting on a rising flood tide of information. We conducted a small survey to obtain some idea of how physicians do, in fact, interpret a laboratory result. Methods We asked 20 house officers, 20 fourth-year medical students and 20 attending physicians, selected in 67 consecutive hallway encounters at four Harvard Medical School teaching hospitals, the following question: "If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of . . . *Adapted from Schwartz2 and originally posed to one of the authors (W.C.) by N. K. Hollenberg. † where PV is the predictive value of the positive result (i.e., the likelihood of disease), p denotes disease prevalence, Se the sensitivity (proportion of diseased persons who give positive results), and Sp the specificity (proportion of healthy persons who give negative results). We are indebted to Drs. S. J. Adelstein, M. D. Altschule, S. W. Casscells, R. Fox, A. H. Goroll, N. K. Hollenberg, G. A. Lamb, B. Lown, S. J. Reiser, G. A. Silver and D. Z. Young for advice and criticism of the manuscript. Source Information From the Cardiovascular Laboratory, Harvard School of Public Health, and the Department of Medicine, Peter Bent Brigham Hospital (address reprint requests to Mr. Casscells at the Cardiovascular Laboratory, Harvard School of Public Health, 665 Huntington Ave., Boston, MA 02115).
Article
Quantitative information about risks and benefits may be meaningful only to patients who have some facility with basic probability and numerical concepts, a construct called numeracy. To assess the relation between numeracy and the ability to make use of typical risk reduction expressions about the benefit of screening mammography. Randomized, cross-sectional survey. A simple random sample of 500 female veterans drawn from a New England registry. One of four questionnaires, which differed only in how the same information on average risk reduction with mammography was presented. Numeracy was scored as the total number of correct responses to three simple tasks. Participants estimated their risk for death from breast cancer with and without mammography. Accuracy was judged as each woman's ability to adjust her perceived risk in accordance with the risk reduction data presented. 61% of eligible women completed the questionnaire. The median age of these women was 68 years (range, 27 to 88 years), and 96% were high school graduates. Both accuracy in applying risk reduction information and numeracy were poor (one third of respondents thought that 1000 flips of a fair coin would result in < 300 heads). Accuracy was strongly related to numeracy: The accuracy rate was 5.8% (95% CI, 0.8% to 10.7%) for a numeracy score of 0, 8.9% (CI, 2.5% to 15.3%) for a score of 1, 23.7% (CI, 13.9% to 33.5%) for a score of 2, and 40% (CI, 25.1% to 54.9%) for a score of 3. Regardless of how information was presented, numeracy was strongly related to accurately gauging the benefit of mammography. More effective formats are needed to communicate quantitative information about risks and benefits.
Article
To test whether physician's diagnostic inferences can be improved by communicating information using natural frequencies instead of probabilities. Whereas probabilities and relative frequencies are normalized with respect to disease base rates, natural frequencies are not normalized. The authors asked 48 physicians in Munich and Düsseldorf to determine the positive predictive values (PPVs) of four diagnostic tests. Information presented in the four problems appeared either as probabilities (the traditional way) or as natural frequencies. When the information was presented as probabilities, the physicians correctly estimated the PPVs in only 10% of cases. When the same information was presented as natural frequencies, that percentage increased to 46%. Representing information in natural frequencies is a fast and effective way of facilitating diagnosis insight, which in turn helps physicians to better communicate risks to patients, and patients to better understand these risks.
Article
The authors present and test a new method of teaching Bayesian reasoning, something about which previous teaching studies reported little success. Based on G. Gigerenzer and U. Hoffrage's (1995) ecological framework, the authors wrote a computerized tutorial program to train people to construct frequency representations (representation training) rather than to insert probabilities into Bayes's rule (rule training). Bayesian computations are simpler to perform with natural frequencies than with probabilities, and there are evolutionary reasons for assuming that cognitive algorithms have been developed to deal with natural frequencies. In 2 studies, the authors compared representation training with rule training; the criteria were an immediate learning effect, transfer to new problems, and long-term temporal stability. Rule training was as good in transfer as representation training, but representation training had a higher immediate learning effect and greater temporal stability.
A-98 Reporting Clinical Test Results: General Guidelines
American Medical Association. CEJA Report 12-A-98 Reporting Clinical Test Results: General Guidelines; 1998. Available at: https://www.ama-assn.org/about-us/councilethical-judicial-affairs-ceja-reports
Improving Diagnosis in Healthcare
  • E P Balogh
  • B T Miller
  • J R Ball
Balogh EP, Miller BT, Ball JR. Improving Diagnosis in Healthcare. Washington, DC: The National Academies Press; 2015.
The Routledge Handbook of Health Communication
  • E M Galarce
  • S Ramandhan
  • K Viswanath
Galarce EM, Ramandhan S, Viswanath K. Health information seeking. In: Thompson TL, Parrott R, Nussbaum JF, eds. The Routledge Handbook of Health Communication. 2nd ed. New York: Routledge; 2011. p 167-180.