ArticlePDF Available

The Most Dangerous Profession: A Note on Nonsampling Error

Authors:
Howard Wainer at Independent Statistician and Author
Howard Wainer
  • Independent Statistician and Author
Download full-text PDF
Read full-text
Download citation

Abstract

Nonsampling errors are subtle, and strategies for dealing with them are not particularly well known within psychology. This article provides a compelling example of an incorrect conclusion drawn from a nonrandom sample: H. C. Lombard's (1835) mortality data. This example is augmented by a second example (A. Wald, 1980) that shows how modeling the selection mechanism can correct for the bias introduced by nonsampling errors. These 2 examples are then connected to modern statistical methods that through the method of multiple imputation allow researchers to assess uncertainty in observational studies. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Content uploaded by Howard Wainer
Author content
All content in this area was uploaded by Howard Wainer
Content may be subject to copyright.
A preview of this full-text is provided by American Psychological Association.
Learn more
Content available from Psychological Methods
This content is subject to copyright. Terms and conditions apply.
... The biggest flaw in their methodology, it seems to me, is their treatment of missing data. Missing data are an enormous problem in all observational studies (see Rosenbaum [2002] for a lucid description of what can be done, Little and Rubin [1987] for a more general discussion of dealing with missing data, and Wainer [2000] for a lively discussion on the impact of missing data on inferences), and its pernicious effects enters into this problem in an interesting way. Avery et al. (2004) suggest that by using a preference ranking they eliminate many of the possible ways that schools can manipulate the rankings. ...
... These were somewhat younger than those found by Madden, but this was expected since Lombard was dealing with ordinary people rather than the "geniuses" in Madden's (the positive correlation between fame and longevity was well known even then). But Lombard's study yielded one surprise; the most dangerous profession-the one with the shortest longevity-was that of "student," with an average age of death of only 20.7! Lombard recognized the reason for this anomaly, but apparently he did not connect it to his other results, which he interpreted as if they reflected real danger (for more details, see Wainer [1999]). ...
Shopping for Colleges When What We Know Ain't
Article
Full-text available
  • Sep 2005
The cost of attending a 4-yr. private university can easily top $150,000, making education one of the most expensive purchases we make. Unfortunately, the sorts of informed opinions and objective comparisons that are widely available for other large consumer purchases, like automobiles, are lacking in education. Instead we must make due with ad hoc indexes like that published annually by U.S. News and World Report. A recent alternative to this index using preference rankings was proposed. In this essay I discuss the challenges facing this index, and other survey instruments, caused by nonignorable missing data. My conclusion is not optimistic. (c) 2005 by JOURNAL OF CONSUMER RESEARCH, Inc..
View
... Reasons for Meditating Indeed Change Over Time There was some indication in previous studies that reasons for meditating may change over time, but the evidence is largely correlational, that is, the importance of a given reason for meditating covaried with meditators' amount of practice (see above). But, as one of the meditation teachers in study 1 remarked, beginning meditators who see their initial goals unfulfilled might just stop meditating after some time, so these correlations suffer from the potential problem of nonsampling error (Wainer, 1999). Our data strongly indicate that long-term meditators might indeed consist of a systematically selected subsample of all people who begin to meditate (see Table 3). ...
Why Do People Begin to Meditate and Why Do They Continue?
Article
Full-text available
  • Jun 2020
  • Mindfulness
Objectives Studies that have examined why people begin to meditate and why they stay with it are few and inconclusive. We looked at these questions in two comprehensive exploratory studies.Methods In study 1, meditation teachers were asked why participants in their courses might begin to meditate and why they might continue to do so. A qualitative analysis of their responses, together with suggestions from the literature, yielded 77 distinct reasons that were applicable to both beginners and advanced meditators. In study 2, 245 meditators (49 who had practiced for less than 1 year) rated how strongly these reasons applied to themselves. Exploratory factor analyses of the reasons both to begin and to continue meditation were conducted.ResultsWe found a large number of different categories that go beyond those identified in previous research. Reasons changed with increasing meditation practice, although spiritual goals tended to become more important only for practitioners with a spiritual background.Conclusions Our results indicate that reasons for meditating are much more diverse than previously assumed and that they are influenced by practitioners’ spiritual background and their personality characteristics.
View
... Wald was able to solve the dilemma of bomber armor largely because he recognized that a selection effect was at work within his sample: he was able to study only those aircraft that had survived previous bombing runs (Wainer, 1999). He recognized how the missing data in his sample could inform on the conditions affecting the total population of bombers, rather than just those that had survived. ...
Selected samples: The nature of silcrete adzes in the formation of Australian stone artefact assemblages
Article
  • Jun 2017
  • JASR
Silcrete flakes and cores are abundant in the surface assemblages from western New South Wales, Australia but retouched tools made from silcrete are much less frequent, especially the heavily retouched forms like flake adzes. The distribution and abundance of these forms together with other silcrete retouched tools is investigated. While heavily retouched forms are present, forms thought to represent the intermediate stages of rejuvenation reduction are largely absent. This absence is used to consider how archaeological sites from the region should be interpreted. It is concluded that sites should not be interpreted as though they represent a material record of activities that occurred only at the place where archaeological sites are identified.
View
... Johnson & Christensen, 2012: 217). Moreover, whereas nonresponse may not necessarily bias a study's data, a single nonresponse renders a probability sample nonrandom and, thus, introduces ambiguity into the inferences that can be made (Wainer, 1999). ...
"More Than Meets the Eye": A Guide to Interpreting the Descriptive Statistics and Correlation Matrices Reported in Management Research
Article
Full-text available
  • Mar 2014
The author of this essay wonders whether in teaching our students the latest analytic techniques we have neglected to emphasize the importance of understanding the most basic aspects of a study's primary data. In response, he provides a 12-part answer to a fundamental question: "What information can be derived from reviewing the descriptive statistics and correlation matrix that appear in virtually every empirically based, nonexperimental paper published in the management discipline?" The seeming ubiquity of strained responses, to what many at first consider to be a vexed question about a mundane topic, leads the author to suggest that students at all levels, seasoned scholars, manuscript referees, and general consumers of management research may be unaware that the standard Table 1 in a traditional Results section reveals "more than meets the eye!"
View
Journal of critical reviews 4 Journal of Critical Reviews AN IMPACT OF MERGERS AND ACQUISITIONS (M&A) TO ACHIEVE INEXPENSIVE ADVANTAGE IN THE ECONOMY OF BAHRAIN (A STUDY ON ISLAMIC BANKS IN THE KINGDOM OF BAHRAIN)
Article
Full-text available
  • Apr 2020
The persistence of this paper is to assess the impact of Merger and Acquisition on the economy of Bahrain. The experiential research methodology is built on quantitative method by using online questionnaire and expressive statistics using the weighted average mean. The performance of an acquisition is measured by questionnaire distributed using selective sample technique to Head of Departments and executive management. The acquisitions were divided into three domains: Motives, Post-acquisition management and critical factors affecting the economy of Bahrain. The result of this research associates that the so-called planned and structural fit between companies complicated in M&A play an important role in refining the operative performance of the developed companies in the post-acquisition period. Effective acquirers have potential performance on the economy of the country, but they also had some skills and competencies to recover the performance of an acquired firm. The results of our study advocate the conclusion that success acquisition and mergers will lead to robust economy, particularly with regard to convinced aptitudes and skills worldwide.
View
Research Methods and Statistics in Psychology
Book
  • Dec 2018
Cambridge Core - Qualitative Research Methods - Research Methods and Statistics in Psychology - by Bernard C. Beins
View
Research Methods: A Tool for Life
Book
  • Nov 2018
Cambridge Core - Qualitative Research Methods - Research Methods - by Bernard C. Beins
View
Predicting Voluntary Turnover from Job Applicant Biodata and Other Applicant Information
Article
  • Sep 2014
  • INT J SELECT ASSESS
Research suggests that biodata (i.e., personal history information) is one of the best predictors of voluntary employee turnover. Given this fact, it is surprising that research on biodata has declined in recent years, and that biodata is not commonly used by employers for selecting employees. This article presents the results of a study that focused on the ability of biodata items and other information available on job applicants to predict voluntary turnover. With one exception, the hypotheses offered were confirmed. Specifically, applicant status (i.e., previously had applied for a job), submission of optional personal history information, employment status (i.e., job applicant was employed), and recruitment source (i.e., employee referral) all predicted voluntary turnover. The ability of applicant status, submission of a personal statement, and recruitment source to predict employee performance also was examined. Applicant status and recruitment source were found to predict subsequent performance ratings.
View
Uneducated Guesses Using Evidence to Uncover Misguided Education Policies
Book
  • Sep 2011
Uneducated Guesseschallenges everything our policymakers thought they knew about education and education reform, from how to close the achievement gap in public schools to admission standards for top universities. In this explosive book, Howard Wainer uses statistical evidence to show why some of the most widely held beliefs in education today--and the policies that have resulted--are wrong. He shows why colleges that make the SAT optional for applicants end up with underperforming students and inflated national rankings, and why the push to substitute achievement tests for aptitude tests makes no sense. Wainer challenges the thinking behind the enormous rise of advanced placement courses in high schools, and demonstrates why assessing teachers based on how well their students perform on tests--a central pillar of recent education reforms--is woefully misguided. He explains why college rankings are often lacking in hard evidence, why essay questions on tests disadvantage women, why the most grievous errors in education testing are not made by testing organizations--and much more. No one concerned about seeing our children achieve their full potential can afford to ignore this book. With forceful storytelling, wry insight, and a wealth of real-world examples,Uneducated Guessesexposes today's educational policies to the light of empirical evidence, and offers solutions for fairer and more viable future policies.
View
View
Abraham Wald's Work on Aircraft Survivability
Article
Full-text available
  • Jun 1984
While he was a member of the Statistical Research Group (SRG), Abraham Wald worked on the problem of estimating the vulnerability of aircraft, using data obtained from survivors. This work was published as a series of SRG memoranda and was used in World War II and in the wars in Korea and Vietnam. The memoranda were recently reissued by the Center for Naval Analyses. This article is a condensation and exposition of Wald's work, in which his ideas and methods are described. In the final section, his main results are reexamined in the light of classical statistical theory and more recent work.
View
The most dangerous profession: A note on nonsampling error.
Article
  • Sep 1999
Nonsampling errors are subtle, and strategies for dealing with them are not particularly well known within psychology. This article provides a compelling example of an incorrect conclusion drawn from a nonrandom sample: H. C. Lomard's (1835) mortality data. This example is augmented by a second example (A. Wald, 1980) that shows how modeling the selection mechanism can correct for the bias introduced by nonsampling errors. These 2 examples are then connected to modern statistical methods that through the method of multiple imputation allow researchers to assess uncertainty in observational studies. The APA's task force on Statistical Inference has received comments and suggestions from interested parties throughout the entire time I have served on it. These comments have always been treated by the task force with careful attention. In the most recent batch was a one-page missive from John Tukey containing seven suggestions. In the course of my professional life I have made many errors, but happily, ignoring statistical advice from John Tukey is not one of them. Tukey's fifth suggestion, in its entirety, is, "non-.ampling errors deserve greater attention, especially when randomization is absent. The formal statistical analysis treats only some of the uncertainties" (J. W. Tukey, personal communication, June 16, 1997). Indeed, but nonsampling errors are subtle, and strategies for dealing with them are not particularly well known within psychology. Thus, I think it would be worthwhile to provide a particularly interesting illustration of one and point the way toward alternative methodologies for interested readers.
View
Responsum
Article
  • Jan 1989
  • J Educ Stat
View
Safety in Caution
Article
  • Jul 1989
  • J Educ Stat
View
Eelworms, Bullet Holes, and Geraldine Ferraro: Some Problems With Statistical Adjustment and Some Solutions
Article
  • Jun 1989
There is no safety in numbers. When data are gathered from a sample in which the selection criteria are unknown, many problems can befall the unwary investigator. In this paper we explore some of these problems and discuss some solutions. Our principle example is drawn from data from students who choose to take the College Board's Scholastic Aptitude Test (SAT). We explore methods of covariance adjustment as well as more explicitly model-based adjustment methods. Among the latter we discuss Heckman's Selection Model, Rubin's Mixture Model, and Tukey's Simplified Selection Model.
View
De l'influence des professions sur la durée de la vie [On the influence of profession on the length of life]
  • Jan 1835
  • 88-131
  • H. C. Lombard