Robert Rosenthal’s research while affiliated with University of California, Riverside and other places

Experimental studies within the education field are rare. The current study used a random effects meta-analytic approach to examine the effectiveness of a teacher expectation intervention across different schools, grade levels, socioeconomic levels, ethnicities, and gender in terms of student mathematics achievement. Teachers were randomly assigned to intervention and control groups, and through professional development workshops were trained in the practices of teachers who have high expectations for all students. The intervention related to three key areas: grouping and learning experiences, class climate, and goal setting. No matter which grouping variables were employed in the random effects meta-analyses, effect sizes in mathematics achievement for the students whose teachers were part of the intervention group compared with students with control group teachers were large, ranging from r = 0.61–0.87. The usefulness of the instructional strategies that formed the basis of the intervention is discussed in light of the relevant literature and the educational implications are presented.

In this interview, we discuss my early uses of meta-analytic procedures, first to combine p-values and then to combine effect sizes as well. My interest in quantifying the magnitude and the statistical significance of the effect of interpersonal expectations probably grew out of the following: (1) a long-held interest in the concept of replication and (2) a series of controversies over the very existence of any effect of interpersonal expectations held, for example, by psychological experimenters, classroom teachers, and leaders of various organizations. Copyright © 2015 John Wiley & Sons, Ltd.

The ability to measure agreement between two independent observers is vital to any observational study. We use a unique situation, the calculation of inter-rater reliability for transcriptions of a parrot’s speech, to present a novel method of dealing with inter-rater reliability which we believe can be applied to situations in which speech from human subjects may be difficult to transcribe. Challenges encountered included (1) a sparse original agreement matrix which yielded an omnibus measure of inter-rater reliability, (2) “lopsided” $2\times 2$ 2 × 2 matrices (i.e. subsets) from the overall matrix and (3) categories used by the transcribers which could not be pre-determined. Our novel approach involved calculating reliability on two levels—that of the corpus and that of the above mentioned smaller subsets of data. Specifically, the technique included the “reverse engineering” of categories, the use of a “null” category when one rater observed a behavior and the other did not, and the use of Fisher’s Exact Test to calculate r r -equivalent for the smaller paired subset comparisons. We hope this technique will be useful to those working in similar situations where speech may be difficult to transcribe, such as with small children.

Interobserver reliability is a vital part of all psychological studies that use an observational methodology to address questions of human behavior. This article provides a brief review of some important points for clarity’s sake, as we believe the current treatment of these topics has become more a case of theory than of practice. In general, there are two types of observer reliability: within observer reliability and between observer reliability. In the event of an experiment in which only one observer is plausible, it is possible, and important, to assess reliability and estimate the likelihood of bias by briefly using a second observer to conduct interobserver reliability trials for a small, random, portion of the data. One of the most common ways of measuring reliability between two observers without the problems inherent in percentage agreement is by using Cohen’s kappa, which takes into account the chance agreement of two observers. The focused kappa greatly expands the researcher’s capability for analysis, as it can be tailored to the question of interest on a case-by-case basis. The scientific method provides a structure that emphasizes the importance of neutrality on the part of the scientist, and it is most likely valid to claim that many scientists pride themselves on their ability to remain unbiased. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

Reports an error in "Effect sizes for experimenting psychologists" by Ralph L. Rosnow and Robert Rosenthal (Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale, 2003[Sep], Vol 57[3], 221-237). A portion of the note to Table 1 was incorrect. The second sentence of the note should read as follows: Fisher’s ʐr is the log transformation of r, that is, ½ loge [(1 + r)/(1 - r)]. (The following abstract of the original article appeared in record 2003-08374-009.) [Correction Notice: An erratum for this article was reported in Vol 63(1) of Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale (see record 2009-03064-004). Correction for Note in TABLE 1 (on page 222): The second sentence should read as follows: Fisher’s zr is the log transformation of r, that is, 1⁄2 loge[(1 + r)/(1 − r)].] This article describes three families of effect size estimators and their use in situations of general and specific interest to experimenting psychologists. The situations discussed include both between- and within-group (repeated measures) designs. Also described is the counternull statistic, which is useful in preventing common errors of interpretation in null hypothesis significance testing. The emphasis is on correlation (r-type) effect size indicators, but a wide variety of difference-type and ratio-type effect size estimators are also described. (PsycINFO Database Record (c) 2012 APA, all rights reserved)

This book is really three-books-in-one, dealing with the topic of artifacts in behavioral research. It is about the problems of experimenter effects which have not been solved. Experimenters still differ in the ways in which they see, interpret, and manipulate their data. Experimenters still obtain different responses from research participants (human or infrahuman) as a function of experimenters' states and traits of biosocial, psychosocial, and situational origins. Experimenters' expectations still serve too often as self-fulfilling prophecies, a problem that biomedical researchers have acknowledged and guarded against better than have behavioral researchers; e.g., many biomedical studies would be considered of unpublishable quality had their experimenters not been blind to experimental condition. Problems of participant or subject effects have also not been solved. Researchers usually still draw research samples from a population of volunteers that differ along many dimensions from those not finding their way into our research. Research participants are still often suspicious of experimenters' intent, try to figure out what experimenters are after, and are concerned about what the experimenter thinks of them. That portion of the complexity of human behavior that can be attributed to the social nature of behavioral research can be conceptualized as a set of artifacts to be isolated, measured, considered, and, sometimes, eliminated. This book examines the methodological and substantive implications of sources of artifacts in behavioral research and strategies for improving this situation.

This book is really three-books-in-one, dealing with the topic of artifacts in behavioral research. It is about the problems of experimenter effects which have not been solved. Experimenters still differ in the ways in which they see, interpret, and manipulate their data. Experimenters still obtain different responses from research participants (human or infrahuman) as a function of experimenters' states and traits of biosocial, psychosocial, and situational origins. Experimenters' expectations still serve too often as self-fulfilling prophecies, a problem that biomedical researchers have acknowledged and guarded against better than have behavioral researchers; e.g., many biomedical studies would be considered of unpublishable quality had their experimenters not been blind to experimental condition. Problems of participant or subject effects have also not been solved. Researchers usually still draw research samples from a population of volunteers that differ along many dimensions from those not finding their way into our research. Research participants are still often suspicious of experimenters' intent, try to figure out what experimenters are after, and are concerned about what the experimenter thinks of them. That portion of the complexity of human behavior that can be attributed to the social nature of behavioral research can be conceptualized as a set of artifacts to be isolated, measured, considered, and, sometimes, eliminated. This book examines the methodological and substantive implications of sources of artifacts in behavioral research and strategies for improving this situation.

This book is really three-books-in-one, dealing with the topic of artifacts in behavioral research. It is about the problems of experimenter effects which have not been solved. Experimenters still differ in the ways in which they see, interpret, and manipulate their data. Experimenters still obtain different responses from research participants (human or infrahuman) as a function of experimenters' states and traits of biosocial, psychosocial, and situational origins. Experimenters' expectations still serve too often as self-fulfilling prophecies, a problem that biomedical researchers have acknowledged and guarded against better than have behavioral researchers; e.g., many biomedical studies would be considered of unpublishable quality had their experimenters not been blind to experimental condition. Problems of participant or subject effects have also not been solved. Researchers usually still draw research samples from a population of volunteers that differ along many dimensions from those not finding their way into our research. Research participants are still often suspicious of experimenters' intent, try to figure out what experimenters are after, and are concerned about what the experimenter thinks of them. That portion of the complexity of human behavior that can be attributed to the social nature of behavioral research can be conceptualized as a set of artifacts to be isolated, measured, considered, and, sometimes, eliminated. This book examines the methodological and substantive implications of sources of artifacts in behavioral research and strategies for improving this situation.