Is there any review of the effects of participants' sample restriction (e.g. only students or young adults) in (cognitive) psychology experiments?
Namely, in many experiments the participants were college students, which are not a typical subsample of a population (e.g., young adults with supposedly higher IQ).
Moreover, one can argue, that typical participants might have other confounds: e.g., more open to experience (e.g., some of our participants come just because they are interested in psychology and want to see how does research look like). Other possibility - some participants come only because of money compensation, hence, lower socio-economic-status* (SES)?
Therefore, could someone please recommend a study that investigates how this restriction might affect the outcomes of the experiment? For example, a sample was compared to a "true random" sample (e.g., taken by calling people from a phone book at random). Or where intelligence / personality / SES were simultaneously controlled for.
I'm mostly interested in memory research, but would be grateful for any information. I just keep noticing that, say, in a typical memory experiment, the participants are almost never controlled for IQ or personality.
* p.s. I'm aware that SES is a contradictionary construct which is affected by IQ and personality for sure, but I've just used it for an example.
Not specifically about memory, but a recent paper in Plos One (Hanel & Vione, 2016) has emphasized the problems of using student participants in general because not only do students differ considerably from the general public, they differ country by country.
It's also notable that student samples are rather homogeneous. This may reduce variability in data sets. A consequence could be that detecting experimental effects is easier, but effect sizes are overestimated. On the other hand correlations between variables will be more difficult to detect with underestimated effect sizes.
Hanel, P. H., & Vione, K. C. (2016). Do Student Samples Provide an Accurate Estimate of the General Public?. PloS one, 11(12), e0168354.
The authors show from two concrete examples that when comparing different subgroups attained from different total groups it is in general impossible to draw any conclusions about the difference between the total groups. That will hold, of course, with the assumption that the subgroups are attained by natural self‐selection instead of random samplin...
This paper considers the estimation of ratio of population mean when some observations on study variable and auxiliary variables are missing in the case of stratified random sampling. Four estimator are presented and their bias and mean square error are formulated. Here the problem of stratified random sampling in the case of missing observations f...