No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Die Replikationskrise vermitteln, aber Vertrauen erhalten
Abstract
Das Positionspapier von Brachem et al. (2022) liefert einen wertvollen Beitrag zum Umgang mit der Replikationskrise und verwandten Themen in der Lehre. Eine zentrale Anregung des Positionspapiers, nämlich die Replikationskrise verstärkt in der Lehre zu thematisieren, möchten wir in diesem Kommentar kritisch beleuchten. Wir stimmen zu, dass Kenntnisse über fragwürdige Forschungspraktiken und Replizierbarkeit vermittelt werden sollten: Nur so können Studierende bestehende Forschung korrekt einordnen und wir können vermeiden, dass zukünftige Wissenschaftsgenerationen die Fehler der Vergangenheit wiederholen. Wie aber verändert ein solcher Fokus auf die Replikationskrise die Einstellung von Psychologiestudierenden zu ihrem eigenen Fach? [...]
Zusammenfassung. In den letzten Jahren gab es innerhalb der Psychologie eine intensive Auseinandersetzung mit den Auswirkungen der Replikationskrise sowie dem hieraus entstandenen Diskurs über die Weiterentwicklung der Disziplin. Als ein Grund für die mangelnde Replizierbarkeit psychologischer Forschung wurde die Verwendung fragwürdiger Forschungspraktiken (eng. QRPs) identifiziert. Während es umfangreiche Untersuchungen zur Prävalenz von QRPs unter Wissenschaftler*innen gibt, ist bisher wenig über die Verbreitung dieser Praktiken unter Studierenden bekannt. Mit der hier vorgestellten Arbeit wurde erstmals eine größere Befragung unter 1397 Psychologie-Studierenden im deutschsprachigen Raum durchgeführt, um die Verbreitung von QRPs in studentischen Projekten sowie den aktuellen Stand der akademischen Lehre in Bezug auf die Replikationskrise und Open Science zu erheben. Die gemeinsame Betrachtung der Lehre und des Einsatzes fragwürdiger Forschungspraktiken versprechen Aufschluss darüber, wie die psychologische Lehre mit dem empirischen Vorgehen der Studierenden zusammenhängt. Die Ergebnisse zeigen, dass QRPs auch in studentischen Projekten vorkommen, wobei große Unterschiede in der Verbreitung einzelner QRPs bestehen. Auch zwischen den verschiedenen Projekttypen zeigten sich Unterschiede, so war die Anwendung von QRPs in Experimentalpraktika am stärksten und in Masterarbeiten am schwächsten ausgeprägt. Unsere Daten weisen insgesamt darauf hin, dass die selbstberichtete Verbreitung von QRPs über den Studienverlauf abnimmt. Zudem scheint ein Großteil der Studierenden bereits mit der Thematik der Replikationskrise in der Lehre in Berührung gekommen zu sein. Deren Behandlung findet größtenteils in der Methodenlehre und weniger in inhaltlich spezialisierten Lehrveranstaltungen statt. Wir geben abschließend Impulse zur Weiterentwicklung der psychologischen Lehre, in denen die Prinzipien der Offenheit, Transparenz und Kollaboration beim Hervorbringen inhaltlich robuster Forschung bereits während des Studiums im Vordergrund stehen.
Science should be self-correcting. However, researchers often hesitate to admit errors and to adopt reforms in their own work. In two studies (overall N = 702), we test whether scientific self-criticism and reform intentions expressed by researchers damage or rather improve their reputation in the eyes of the public (i.e. perceivers). Across both studies, such self-correction (compared to no self-correction) increases perceivers’ epistemic trustworthiness ascriptions, credibility perceptions, and willingness to further engage with science. Study 2 revealed that these effects were largely driven by the no self-criticism condition. In addition, researchers’ commitment to implementing reforms had positive effects and rejecting reforms had negative effects on perceptions, irrespective of the extent of these reforms. These findings suggest that researchers’ fear that self-criticism and expressing reform intentions may damage their reputation may be unfounded.
Several meta-analytical attempts to reproduce results of empirical research have failed in recent years, prompting scholars and news media to diagnose a “replication crisis” and voice concerns about science losing public credibility. Others, in contrast, hoped replication efforts could improve public confidence in science. Yet nationally representative evidence backing these concerns or hopes is scarce. We provide such evidence, conducting a secondary analysis of the German “Science Barometer” (“Wissenschaftsbarometer”) survey. We find that most Germans are not aware of the “replication crisis.” In addition, most interpret replication efforts as indicative of scientific quality control and science’s self-correcting nature. However, supporters of the populist right-wing party AfD tend to believe that the “crisis” shows one cannot trust science, perhaps using it as an argument to discredit science. But for the majority of Germans, hopes about reputational benefits of the “replication crisis” for science seem more justified than concerns about detrimental effects.
Replication failures of past findings in several scientific disciplines, including psychology, medicine, and experimental economics, have created a “crisis of confidence” among scientists. Psychological science has been at the forefront of tackling these issues, with discussions about replication failures and scientific self-criticisms of questionable research practices (QRPs) increasingly taking place in public forums. How this replicability crisis impacts the public’s trust is a question yet to be answered by research. Whereas some researchers believe that the public’s trust will be positively impacted or maintained, others believe trust will be diminished. Because it is our field of expertise, we focus on trust in psychological science. We performed a study testing how public trust in past and future psychological research would be impacted by being informed about (i) replication failures (replications group), (ii) replication failures and criticisms of QRPs (QRPs group), and (iii) replication failures, criticisms of QRPs, and proposed reforms (reforms group). Results from a mostly European sample (N = 1129) showed that, compared to a control group, people in the replications, QRPs, and reforms groups self-reported less trust in past research. Regarding trust in future research, the replications and QRPs groups did not significantly differ from the control group. Surprisingly, the reforms group had less trust in future research than the control group. Nevertheless, people in the replications, QRPs, and reforms groups did not significantly differ from the control group in how much they believed future research in psychological science should be supported by public funding. Potential explanations are discussed.
In the current psychological debate, low replicability of psychological findings is a central topic. While the discussion about the replication crisis has a huge impact on psychological research, we know less about how it impacts public trust in psychology. In this article, we examine whether low replicability damages public trust and how this damage can be repaired. Studies 1–3 provide correlational and experimental evidence that low replicability reduces public trust in psychology. Additionally, Studies 3–5 evaluate the effectiveness of commonly used trust-repair strategies such as information about increased transparency (Study 3), explanations for low replicability (Study 4), or recovered replicability (Study 5). We found no evidence that these strategies significantly repair trust. However, it remains possible that they have small but potentially meaningful effects, which could be detected with larger samples. Overall, our studies highlight the importance of replicability for public trust in psychology.
In methodological and practical debates about replications in science, it is (often implicitly) assumed that replications will affect public trust in science. In this preregistered experiment (N = 484), we varied (a) whether a replication attempt was successful or not and (b) whether the replication was authored by the same, or another lab. Results showed that ratings of study credibility (e.g. evidence strength, ηP2 = .15) and researcher trustworthiness (e.g. expertise, ηP2 = .15) were rated higher upon learning of replication success, and lower in case of replication failure. The replication’s author did not make a meaningful difference. Prior beliefs acted as covariate for ratings of credibility, but not trustworthiness, while epistemic beliefs regarding the certainty of knowledge were a covariate to both. Hence, laypeople seem to notice that successfully replicated results entail higher epistemic significance, while possibly not taking into account that replications should be conducted by other labs.
The credibility revolution (sometimes referred to as the “replicability crisis”) in psychology has brought about many changes in the standards by which psychological science is evaluated. These changes include (a) greater emphasis on transparency and openness, (b) a move toward preregistration of research, (c) more direct-replication studies, and (d) higher standards for the quality and quantity of evidence needed to make strong scientific claims. What are the implications of these changes for productivity, creativity, and progress in psychological science? These questions can and should be studied empirically, and I present my predictions here. The productivity of individual researchers is likely to decline, although some changes (e.g., greater collaboration, data sharing) may mitigate this effect. The effects of these changes on creativity are likely to be mixed: Researchers will be less likely to pursue risky questions; more likely to use a broad range of methods, designs, and populations; and less free to define their own best practices and standards of evidence. Finally, the rate of scientific progress—the most important shared goal of scientists—is likely to increase as a result of these changes, although one’s subjective experience of making progress will likely become rarer.
Over the past 10 years, crises surrounding replication, fraud, and best practices in research methods have dominated discussions in the field of psychology. However, no research exists examining how to communicate these issues to undergraduates and what effect this has on their attitudes toward the field. We developed and validated a 1-hr lecture communicating issues surrounding the replication crisis and current recommendations to increase reproducibility. Pre- and post-lecture surveys suggest that the lecture serves as an excellent pedagogical tool. Following the lecture, students trusted psychological studies slightly less but saw greater similarities between psychology and natural science fields. We discuss challenges for instructors taking the initiative to communicate these issues to undergraduates in an evenhanded way.
- Brachem J.
The "replication crisis" in the public eye: Germans' awareness and perceptions of the (ir) reproducibility of scientific research. Public Understanding of Science
- N G Mede
- M S Schäfer
- R Ziegler
- M Weißkopf
Mede, N. G., Schäfer, M. S., Ziegler, R. & Weißkopf, M. (2020). The
"replication crisis" in the public eye: Germans' awareness and
perceptions of the (ir) reproducibility of scientific research. Public Understanding of Science, 30 (1), 91 -102. London: SAGE
Publications Sage UK. https://doi.org/https://doi.org/10.1177/
0963662520954370