Herbert i Weisberg

Causalytics, LLC

Feature selection with high-dimensional data and a very small proportion of relevant features poses a severe challenge to standard statistical methods. We have developed a new approach (HARVEST) that is straightforward to apply, albeit somewhat computer-intensive. This algorithm can be used to pre-screen a large number of features to identify those...

There is currently much interest in generating more individualized estimates of treatment effects. However, traditional statistical methods are not well suited to this task. Post hoc subgroup analyses of clinical trials are fraught with methodological problems. We suggest that the alternative research paradigm of predictive analytics, widely used i...

RCTs have saved countless lives. But, in a world of complex medical conditions and personalised treatment plans, are they still fit for purpose? Herbert I. Weisberg looks to the past in search of a way forward RCTs have saved countless lives. But, in a world of complex medical conditions and personalised treatment plans, are they still fit for purp...

When Fisher began his tenure at Rothamsted in 1919, statistical methods were almost entirely concerned with problems in which large volumes of data were in hand. This chapter discusses the two methodological problems faced by experimentalists: confounding and random variability. By 1923, Fisher had come to realize that both of these could be solved...

Methods and media for facilitating estimation of the causal effect for individuals based on predictor factors associated with the individuals. Estimated values of the causal effect may be used to distinguish between individuals and to recommend treatment based on the predicted treatment effect for the individuals.

An original account of willful ignorance and how this principle relates to modern probability and statistical methods. Through a series of colorful stories about great thinkers and the problems they chose to solve, the author traces the historical evolution of probability and explains how statistical methods have helped to propel scientific researc...

Do cell phones cause cancer? Can a new curriculum increase student achievement? Determining what the real causes of such problems are, and how powerful their effects may be, are central issues in research across various fields of study. Some researchers are highly skeptical of drawing causal conclusions except in tightly controlled randomized exper...

Although the superior internal validity of the randomized clinical trial (RCT) is invaluable to establish causality, generalizability is far from guaranteed. In particular, strict selection criteria intended to maximize treatment efficacy and safety can impair external validity. This problem is widely acknowledged in principle but sometimes ignored...

Omitted Confounding Variables Measurement Error The Regression Effect Specifying a Mathematical Model Sampling Error Separation of Groups on a Confounding Factor Summary

Categorical or Numerical Variables Comparison of Matching and Adjustment Procedures Combining Procedures

Definition of Randomization Properties of Randomization Further Points on Randomization Reasons for the Use of Nonrandomized Studies Types of Comparative Studies Our Attitude toward Nonrandomized Studies

The primary causes of rapidly increasing auto bodily injury insurance rates in the late 1980s and early 1990s are an increase in the number of claims filed per accident and the escalation of medical costs at twice the overall rate of inflation. Changes in claiming behavior can arise from the operation of a personal injury insurance lottery. Claiman...

Various mathematical and statistical models for estimation of automobile insurance pricing are reviewed. The methods are compared on their predictive ability based on two sets of automobile insurance data for two different states collected over two different periods. The issue of model complexity versus data availability is resolved through a compa...

Statistical methods in general, and multiple regression analysis in particular, are being used increasingly to provide evidence in employment discrimination cases. While the technical issues involved in using statistical methods to detect discrimination are straightforward, the conceptual issues are much less clearly understood. This article provid...

The analysis of growth curves has long been important in biostatistics. Work has focused on two problems: the estimation of individual curves based on many data points, and the estimation of the mean growth curve for a group of individuals. This paper extends a recent approach that seeks to combine data from a group of individuals in order to impro...

This paper examines various conceptual approaches to the etiology of child abuse and reports the results of an original study. Data on 99 families with children who were court adjudicated as having suffered non-accidental trauma were analyzed by means of a cluster analysis (hierarchical nearest-neighbor clustering). Four empirically-derived cluster...

Traditional actuarial methods for predicting pure premiums on the basis of cross-classified data can be regarded as statistical estimates of model parameters. This article develops a simplified model of claim generation and uses it to illuminate the properties of various proposed cross-classification methods. Although traditional methods have natur...

Adjustment for a Confounding Factor Bias, Precision, and Statistical Significance Some Qualitative Considerations

In an earlier paper (Bryk & Weisberg, 1976), we introduced the value-added analysis. This approach built on the notion that many educational programs are dynamic interventions in natural growth processes. In particular, by modeling the natural growth process, we developed an analytic model and estimation procedure for assessing program impact on th...

Many evaluations of social interventions are based on uncontrolled assignments of individuals to treatment groups. Statistical adjustments are often used to compensate for naturally occurring differences between groups. There is much confusion and controversy about the adequacy of these statistical methods. A variety of interrelated problems have b...

The Brookline Early Education Project (BEEP) is a demonstration model in which a public school system in collaboration with a pediatric center has extended teaching and diagnostic services to the earliest days of life. In most cases, enrollment and data collection began three months before the children were born. A comprehensive diagnostic componen...

Many experimental programs aim to accelerate the growth of individuals on some dimension of interest. Typically, a group exposed to the program is compared with a control group. When it is feasible, random assignment of Ss to the program and control groups assures an unbiased comparison between treatments. Without randomization there may be importa...

Randomized experiments are rarely feasible in large-scale educational and social evaluations. Most evaluations are observational studies in which the investigators have very limited control over the assignment of individuals to treatments. Since the effect of an intervention becomes confounded with those of other influences, clear causal inferences...

This report focuses on three main questions: (1) To what extent does a Head Start experience accelerate the rate at which disadvantaged preschoolers acquire cognitive skills? (2) Are the Planned Variation models, simply by virtue of sponsorship more effective than ordinary nonsponsored Head Start programs? and (3) Are some Planned Variation models...

Dempster's method of upper and lower probabilities represents an attempt to obtain posterior probability statements about parameters without the necessity to incorporate prior information. We apply Dempster's method to the case of multinomial data for which there is some natural ordering of the categories. Inferences are derived for the sum of an a...

A Bayesian approach to the problem of comparing two multinomial distributions with k categories and a natural ordering of the categories is developed. Assuming independent Dirichlet priors, we obtain a convenient algorithm for computing the posterior probability that one distribution is stochastically larger than the other. The important special ca...