Bias and asymmetric loss in expert forecasts: A study of physician prognostic behavior with respect to patient survival

Institute for Quantitative Social Science, Harvard University, CGIS 1737 Cambridge Street, Cambridge, MA 02138, United States.
Journal of Health Economics (Impact Factor: 2.58). 08/2008; 27(4):1095-108. DOI: 10.1016/j.jhealeco.2008.02.011
Source: PubMed


We study the behavioral processes undergirding physician forecasts, evaluating accuracy and systematic biases in estimates of patient survival and characterizing physicians' loss functions when it comes to prediction. Similar to other forecasting experts, physicians face different costs depending on whether their best forecasts prove to be an overestimate or an underestimate of the true probabilities of an event. We provide the first empirical characterization of physicians loss functions. We find that even the physicians subjective belief distributions over outcomes are not well calibrated, with the loss characterized by asymmetry in favor of over-predicting patients' survival. We show that the physicians' bias is further increased by (1) reduction of the belief distributions to point forecasts, (2) communication of the forecast to the patient, and (3) physicians own past experience and reputation.

Full-text preview

Available from:
  • Source
    • "Doctors also display non-standard beliefs. A survey in Chicago showed that doctors are often too optimistic in their prediction of patient survival, especially when communicating with the patients and their relatives (Alexander and Christakis, 2008). Interestingly, the authors succeeded in incorporating their survey into the administrative forms, which a doctor had to complete anyway when diagnosing a patient. "

    Preview · Article · Nov 2015
  • Source
    • "As a consequence, investors tend to realize their gains more often than their losses as they sell winning stocks more readily. There are many other examples for such asymmetry, such as in bankruptcy prediction [57], behavioral finances [45], expected stock returns [2], criminal justice settings [6], physician prognostic behavior [1], product recommendations [32], and so on. Cost-sensitive regression is still not adequately addressed in the data mining literature, as most existing research in this area deals with classification problems. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Regression learning methods in real world applications often require cost minimization instead of the reduction of various metrics of prediction errors. Currently in the literature, there is a lack of white box solutions that can deal with forecasting problems where under-prediction and over-prediction errors have different consequences. To fill this gap, we introduced the Cost-sensitive Global Model Tree (CGMT), which applies a fitness function that minimizes an average misprediction cost. Proposed specialized genetic operators improve searching for optimal tree structure and cost-sensitive linear regression models in the leaves. Experimental validation is performed on loan charge-off data. It is known to be a difficult forecasting problem for banks due to the asymmetric cost structure. Obtained results show that specialized evolutionary algorithm applied to model tree induction finds significantly more accurate predictions than tested competitors. Decisions generated by the CGMT are simple, easy to interpret, and can be applied directly.
    Full-text · Article · Apr 2015 · Decision Support Systems
  • Source
    • "Asking subjects to make two forecasts also allows us to identify the higher forecast as the preferred alternative and examine how preference influences the counting rule. Forecasts may be biased in a variety of ways (Alexander, 2008; Harvey, 2007; Wolfson, Doctor & Burns, 2000). A " value induced " or preference bias is frequently mentioned in the clinical decision making literature to explain the ostensible tendency of clinicians and patients to make over-optimistic forecasts about favored alternatives ( "
    [Show abstract] [Hide abstract]
    ABSTRACT: The field of clinical decision making is polarized by two predominate views. One holds that treatment recommendations should conform with guidelines; the other emphasizes clinical expertise in reaching case-specific judgments. Previous work developed a test for a proposed alternative, that clinical judgment should systematically incorporate both general knowledge and patient-specific information. The test was derived from image theory's two phase-account of decision making and its ``simple counting rule'', which describes how possible courses of action are pre-screened for compatibility with standards and values. The current paper applies this rule to clinical forecasting, where practitioners indicate how likely a specific patient will respond favorably to a recommended treatment. Psychiatric trainees evaluated eight case vignettes that exhibited from 0 to 3 incompatible attributes. They made two forecasts, one based on a guideline recommendation, the other based on their own alternative. Both forecasts were predicted by equally- and unequally-weighted counting rules. Unequal weighting provided a better fit and exhibited a clearer rejection threshold, or point at which forecasts are not diminished by additional incompatibilities. The hypothesis that missing information is treated as an incompatibility was not confirmed. There was evidence that the rejection threshold was influenced by clinician preference. Results suggests that guidelines may have a de-biasing influence on clinical judgment. Subject to limitations pertaining to the subject sample and population, clinical paradigm, guideline, and study procedure, the data support the use of a compatibility test to describe how clinicians make patient-specific forecasts.
    Preview · Article · May 2012 · Judgment and decision making
Show more