Shall we really do it again? The powerful concept of replication is neglected in the social sciences.
... Se requieren mucha información y materiales acerca del experimento para que un experimentador realice una replicación (Schmidt, 2009). Previa a la ejecución de la replicación del experimento, algunos o la totalidad de los materiales pueden requerir cambios produciendo nuevas o modificadas versiones de estos. ...
Revista electrónica de Computación, Informática, Biomédica y Electrónica es una publicación periódica de la División de Electrónica y Computación del CUCEI (Centro Universitario de Ciencias Exactas e Ingenierías) de la Universidad de Guadalajara que publica artículos de interés para la comunidad científica en las áreas de las Ciencias de la Computación e Informática, Biomédica y Electrónica. ReCIBE es semestral y publica artículos inéditos, arbitrados, en inglés y español, que abordan resultados, análisis reflexivos y revisiones del estado del arte de áreas específicas de investigación y desarrollo de tecnología. Nuestra audiencia principal son maestros, investigadores, estudiantes y profesionales de la industria de las áreas de: Computación, Informática, Biomédica, Electrónica y Comunicaciones.
... Regarding the contribution of our work, it is important to highlight that the last twenty years have been characterized by a remarkable awareness of the importance of replicability and reproducibility of psychological research, associated with the crisis of confidence in results that are not replicated (Cesario, 2014;Nelson et al., 2018;Nosek et al., 2022;Schmidt, 2016). In our study, the need to test whether the results of previous research on the relationship of gender and age with depressive symptomatology are replicated is warranted; especially as these results may change as a function of cultural, climatic, geographic, economic, and social context. ...
The objective of this study was to analyze the relationship of gender and age with depressive symptomatology in adolescents. The participants were 1,212 adolescents, from 1st (53.9%) and 2nd (46.1%) year in Compulsory Secondary Education, belonging to seven schools in the Region of Murcia (Spain), and 104 parents. The percentage of adolescents with clinically significant depressive symptoms is similar to that found in other studies in Spain. The girls presented greater depressive symptomatology. The gender difference was greater in the group with high depressive symptomatology. Depressive symptomatology increased with age, consistent with previous research. The difference was highlighted in adolescent self-reports but not in parent reports. The main conclusion is that the development of depression prevention programs should take gender and age into consideration.
... Third, this study used the same wrongful conviction rates as those used in Wu's (2021) experiment in the American context. However, as a conceptual follow-up to a previous study (Fabrigar & Wegener, 2016;Schmidt, 2016), the actual wrongful conviction rates in Japan should be used. Fourth, the duration for which information on wrongful conviction rate reduces the support for the death penalty is not clear; in both Wu's (2021) study and ours, the procedure involved a continuous sequence of steps from presenting the information on wrongful conviction rates to collecting the response. ...
This study tested the hypothesis that presenting people with information about wrongful convictions that are considered an unacceptable cost of administering justice lowers support for the death penalty. We recruited 815 participants using an online crowdsourcing service in Japan, randomly assigned them to eight groups with varying combinations of presence or absence of information emphasizing losses caused by wrongful convictions and asked them to answer questions on the death penalty. Results of the preregistration experiment showed no effect of loss emphasis, but the groups provided with information on wrongful conviction rates were less likely to support the death penalty than those with no information; the higher the wrongful conviction rate, the lower was the support. This study’s findings, which replicated a previous study’s findings with non-U.S. participants, support the acceptability hypothesis and confirm the institutional and cultural universality of the effect of wrongful conviction information on public attitude toward capital punishment.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
Astrophysics is often cast as an observational science, devoid of traditional experiments, along with astronomy and cosmology. Yet, a thriving field of experimental research exists called laboratory astrophysics. How should we make sense of this apparent tension? I argue that approaching the epistemology of astrophysics by attending to the production of empirical data and the aims of the research better illuminates both the successes and challenges of empirical research in astrophysics than evaluating the epistemology of astrophysics according to the presence or absence of experiments.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
With the increased use of simulations as investigative tools in various scientific fields, the question naturally arises as to how these simulations are epistemically justified. One natural approach is to insist that the numerical aspects of simulation justification be performed separately from the physical aspects, but Winsberg (2010) has argued that this is impossible for highly complex simulations. Based on a survey and close examination of a range of astrophysical MHD codes and their attendant literature, I argue that insisting on a strict separation of these aspects of simulation justification is neither epistemically necessary nor advisable.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
Observational astronomy is plagued with selection effects that must be taken into account when interpreting data from astronomical surveys. Because of the physical limitations of observing time and instrument sensitivity, datasets are rarely complete. However, determining specifically what is missing from any sample is not always straightforward. For example, there are always more faint objects (such as galaxies) than bright ones in any brightness-limited sample, but faint objects may not be of the same kind as bright ones. Assuming they are can lead to mischaracterizing the population of objects near the boundary of what can be detected. Similarly, starting with nearby objects that can be well observed and assuming that objects much farther away (and sampled from a younger universe) are of the same kind can lead us astray. Demographic models of galaxy populations can be used as inputs to observing system simulations to create “mock” catalogues that can be used to characterize and account for multiple, interacting selection effects. The use of simulations for this purpose is common practice in astronomy, and blurs the line between observations and simulations; the observational data cannot be interpreted independent of the simulations. We will describe this methodology and argue that astrophysicists have developed effective ways to establish the reliability of simulation-dependent observational programs. The reliability depends on how well the physical and demographic properties of the simulated population can be constrained through independent observations. We also identify a new challenge raised by the use of simulations, which we call the “problem of uncomputed alternatives.” Sometimes the simulations themselves create unintended selection effects when the limits of what can be simulated lead astronomers to only consider a limited space of alternative proposals.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
A theoretical astrophysicist discusses the principles and rules-of-thumb underlying the construction of models and simulations from the perspective of an active practitioner, where it is emphasised that they are designed to address specific scientific questions. That models are valid only within a restricted space of parameters and degenerate combinations of parameter values produce the same observable outcome are features, and not bugs, of competent practice that fit naturally within a Bayesian framework of inference. Idealisations within a model or simulation are strongly tied to the questions they are designed to address and the precision at which they are confronted by data. If the practitioner visualises a hierarchy of models of varying sophistication (which is standard practice in astrophysics and climate science), then de-idealisation becomes an irrelevant concept. Opportunities for future collaborations between astrophysicists and philosophers of science are suggested.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
Next-generation observational surveys in astronomy provide empirical data with increasingly high resolution and precision. After presenting the basic methods of population synthesis (via Conroy C, Ann Rev Astronom Astrophys 51:393–455, 2013; Maraston C, Mon Not Royal Astronom Soc 362:799–825, 2005), this paper argues for several related conclusions. The increased precision of the new methods requires the development of improved theoretical resources and models to provide the richest interpretation of the new data (as argued by Maraston C, Strömbäck G, Monthly Not Royal Astronom Soc 418:2785–2811, 2011). The measurement of physical variables and parameters in population synthesis is best understood using a model-based account along the lines of (Tal E, The epistemology of measurement: a model-based approach. Dissertation, The University of Toronto, 2012) and (Parker WS, Br J Philos Sci 68:273–304, 2017). Finally, in the case of population synthesis, improved empirical data does not dispense with the need for theoretical reasoning in post-data analysis. In fact, the high-resolution data used in next-generation population synthesis demands ever richer theories and models, a process that results in hybrid enrichment of theoretical and observational methods and results.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
We discuss three philosophically interesting epistemic peculiarities of black hole astrophysics: (1) issues concerning whether and in what sense black holes do exist; (2) how to best approach multiplicity of available definitions of black holes; (3) short (i.e., accessible within an individual human lifespan) dynamical timescales present in many of the recent, as well as prospective, observations involving black holes. In each case we argue that the prospects for our epistemic situation are optimistic.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
It is generally accepted that science sometimes involves reasoning with analogies. Often, this simply means that analogies between disparate objects of study might be used as heuristics to guide theory development. Contemporary black hole physics, however, deploys analogical reasoning in a way that seems to overreach this traditional heuristic role. In this chapter, I describe two distinct pieces of analogical reasoning that are quite central to the contemporary study of black holes. The first underpins arguments for the existence of astrophysical Hawking radiation, and the second underpins arguments for black holes being ‘genuinely’ thermodynamical in nature. I argue that while these are distinct analogical arguments, they depend on one another in an interesting way: the success of the second analogical argument presupposes the success of the first. This induces a tension for those who wish to take black hole thermodynamics seriously, but who are sceptical of the evidence provided for astrophysical Hawking radiation by the results of analogue gravity. I consider three ways to resolve this tension, and show that each fails.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
This chapter examines some of the philosophical literature on idealizations in science and the epistemic challenges idealizations potentially pose for astrophysical methodology, particularly its use of computer simulations. We begin by surveying philosophical literature on idealization connected to (1) kinds of idealizations deployed in science, (2) the aims of idealization in science, and (3) various strategies for de-idealization. Using collisional ring galaxy simulations as a case study, we examine how these three themes play out in the context of astrophysical computer simulations. Ultimately, we argue that deploying de-idealization strategies is central to bolstering epistemic confidence in simulations in astrophysics. We conclude with some remarks on the role of idealization in the context of astrophysical computer simulations more generally.
... The Replication Crisis, 26 according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018, and Fletcher (2021) before going back to our current issue. The typology I suggest orders replication along four categories. ...
Measuring the rate at which the universe expands at a given time–the ‘Hubble constant’– has been a topic of controversy since the first measure of its expansion by Edwin Hubble in the 1920s. As early as the 1970s, Sandage and de Vaucouleurs have been arguing about the adequate methodology for such a measurement. Should astronomers focus only on their best indicators, e.g., the Cepheids, and improve the precision of this measurement based on a unique object to the best possible? Or should they “spread the risks”, i.e., multiply the indicators and methodologies before averaging over their results? Is a robust agreement across several uncertain measures, as is currently argued to defend the existence of a ‘Hubble crisis’ more telling than a single 1% precision measurement? This controversy, I argue, stems from a misconception of what managing the uncertainties associated with such experimental measurements require. Astrophysical measurements, such as the measure of the Hubble constant, require a methodology that permits both to reduce the known uncertainties and to track the unknown unknowns. Based on the lessons drawn from the so-called Hubble crisis, I sketch a methodological guide for identifying, quantifying and reducing uncertainties in astrophysical measurements, hoping that such a guide can not only help to re-frame the current Hubble tension, but serve as a starting point for future fruitful discussions between astrophysicists, astronomers and philosophers.
... To develop theory in this area, we examined the effects of justice events (i.e., vicarious mistreatment) on overall justice perceptions while identifying noteworthy mediating mechanisms and novel boundary conditions of this relationship across three complementary studies. These studies employed two methodologies to operationalize vicarious mistreatment using a variety of behavioral indicators (e.g., incivility, ostracism, and discrimination), which allowed us to achieve a constructive replication of our results (Schmidt, 2016) and also helps to alleviate common method bias concerns. ...
Though we would like to believe that people universally consider workplace mistreatment to be an indicator of injustice, we describe why bystanders can react to justice events (in this study, vicariously observing or becoming aware of others being mistreated) with diverging perceptions of organizational injustice. We show that a bystander’s gender and their gender similarity to the target of mistreatment can produce identity threat, which affects whether bystanders perceive the overall organization to be rife with gendered mistreatment and unfairness. Identity threat develops via two pathways—an emotion-focused reaction and a cognitive-focused processing of the event—and each pathway distally relates to different levels of bystanders’ justice perceptions. We test these notions in three complementary studies: two laboratory experiments (N = 563; N = 920) and a large field study (N = 8,196 employees in 546 work units). Results generally show that bystanders who are women or similar in gender to the target of mistreatment reported different levels of emotional and cognitive identity threat that related to psychological gender mistreatment climate and workplace injustice following the incident as compared to men and those not similar in gender to the target. Overall, by integrating and extending bystander theory and dual-process models of injustice perceptions, through this work, we provide a potentially overlooked reason why negative behaviors like incivility, ostracism, and discrimination continue to occur in organizations.
... In the context of the results of this review, with a majority of studies coming from Northern Europe, there is a need to examine the results of these studies in different socio-economic settings. To achieve this 'conceptual replication' (Schmidt 2009) of studies needs to occur. 'If education research is to be relied upon to develop sound policy and practice, then conducting replications on important findings is essential to moving towards a more reliable and trustworthy understanding of educational environments' (Makel and Plucker 2014). ...
The Conceiving – Designing – Implementing – Operating (CDIO) Initiative identifies itself as a global educational framework for producing the next generation of engineers. The purpose of this systematic review is to provide an overview of trends and to consider how these may be optimised for the continued evolution of the initiative. This systematic review follows PRISMA guidelines and is preregistered on The Open Science Framework (OSF). The review includes all publications within the CDIO knowledge library between 2010 and 2020 (N = 898). Each of the publications was categorised as Advances in CDIO, CDIO Implementation and Engineering Educational Research. The initial screening identified the popularity of publishing articles discussing CDIO implementation and the notable decline in CDIO publications. A second screening took place that included all the publications in the Engineering Educational Research category. Some of the findings include: 1) 43% of publications have links with Nordic institutions. 2) Sweden is the most active country. 3) 81% of the publications are completed collaboratively; however, only 22% are cross institutional collaborations. The paper concludes with three main suggestions for future research: 1) Enhancing evidence-based practice 2) Support of blended learning research and 3) Further development of collaboration & replication efforts.
... Second, while it is widely accepted that an empirical finding does not count as scientific evidence until it is independently and directly replicated by other scientists (Open Science Collaboration, 2015;Pashler & Harris, 2012;Schmidt, 2016;Zwaan et al., 2018), direct replication attempts are rare. According to one estimate, replication attempts make up roughly 1% of published psychological research (Makel et al., 2012). ...
The amount of neuroimaging evidence introduced in courts continues to increase. Meanwhile, neuroimaging research is in the midst of a reproducibility crisis, as many published findings appear to be false positives. The problem is mostly due to small sample sizes, lack of direct replications, and questionable research practices. There are concerns that a significant proportion of neuroimaging evidence introduced in court may therefore be unreliable. Guidelines governing the admissibility of scientific evidence—Frye and Daubert—are not designed to weed out such data. We propose supplementing Frye and Daubert with minimal reproducibility criteria that allow judges to make informed admissibility decisions about neuroimaging research. To demonstrate how this could work, we subjected functional magnetic resonance imaging (fMRI) findings on psychopathy—evidence that has been admitted in court—to a minimal reproducibility test. A systematic PRISMA search found 64 relevant studies but no sufficiently powered, directly replicated evidence of a psychopathy-related neurobiological profile. This illustrates two things: (a) the probability of false positives in this data set is likely to be unacceptably high and (b) the reproducibility of similar neuroimaging evidence can be evaluated in a straightforward way. Our findings suggest an urgent need to modify admissibility guidelines to exclude low-quality neuroimaging data.
... The replicability of scientific studies has played a critical role in the development of science [11,25]. Scientific results that fail to replicate will misguide the research advancement, and impair the credibility of the research community. ...
The low replication rate of published studies has long concerned the social science community, making understanding the replicability a critical problem. Several studies have shown that relevant research communities can make predictions about the replicability of individual studies with above-chance accuracy. Follow-up work further indicates that laypeople can also achieve above-chance accuracy in predicting replicability when experts interpret the studies into short descriptions that are more accessible for laypeople. The involvement of scarce expert resources may make these methods expensive from financial and time perspectives. In this work, we explored whether laypeople can predict the replicability of social science studies without expert intervention. We presented laypeople with raw materials truncated from published social science papers and elicited their answers to questions related to the paper. Our results suggested that laypeople were engaged in this technical task, providing reasonable and self-contained answers. The majority of them also demonstrated a good understanding of the material. However, the solicited information had limited predictive power on the actual replication outcomes. We further discuss several lessons we learned compared to the approach with expert intervention to inspire future works.
... The Replication Crisis 27 , according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018) andFletcher (2021) before going back to our current issue. The typology I suggest decline replication along four categories. ...
Measuring the rate at which the universe expands at a given time -- the 'Hubble constant' -- has been a topic of controversy since the first measure of its expansion by Edwin Hubble in the 1920's. As early as the 1970's, Sandage et de Vaucouleurs have been arguing about the adequate methodology for such a measurement. Should astronomers focus only on their best indicators, e.g., the Cepheids, and improve the precision of this measurement based on a unique object to the best possible? Or should they 'spread the risks', i.e., multiply the indicators and methodologies before averaging over their results? Is a robust agreement across several uncertain measures, as is currently argued to defend the existence of a 'Hubble crisis' more telling than a single one percent precision measurement? This controversy, I argue, stems from a misconception of what managing the uncertainties associated with such experimental measurements require. Astrophysical measurements, such as the measure of the Hubble constant, require a methodology that permits both to reduce the known uncertainties and to track the unknown unknowns. Based on the lessons drawn from the so-called Hubble crisis, I sketch a methodological guide for identifying, quantifying and reducing uncertainties in astrophysical measurements, hoping that such a guide can not only help to re-frame the current Hubble tension, but serve as a starting point for future fruitful discussions between astrophysicists, astronomers and philosophers.
... The Replication Crisis 27 , according to which many findings in social, behavioral and biomedical sciences have failed to replicate at alarming rates, has led to many interesting developments when it comes to understand what is a replication and what purpose it can serve. I will briefly present a typology of replication adapted from Schmidt and Oh (2016), Schmidt (2016), Zwaan et al. (2018) andFletcher (2021) before going back to our current issue. The typology I suggest decline replication along four categories. ...
Measuring the rate at which the universe expands at a given time--the "Hubble constant"-- has been a topic of controversy since the first measure of its expansion by Edwin Hubble in the 1920's. As early as the 1970's, Sandage et de Vaucouleurs have been arguing about the adequate methodology for such a measurement. Should astronomers focus only on their best indicators, e.g., the Cepheids, and improve the precision of this measurement based on a unique object to the best possible? Or should they “spread the risks”, i.e., multiply the indicators and methodologies before averaging over their results? Is a robust agreement across several uncertain measures, as is currently argued to defend the existence of a "Hubble crisis" more telling than a single 1 % precision measurement? This controversy, I argue, stems from a misconception of what managing the uncertainties associated with such experimental measurements require. Astrophysical measurements, such as the measure of the Hubble constant, require a methodology that permits both to reduce the known uncertainties and to track the unknown unknowns. Based on the lessons drawn from the so-called Hubble crisis, I sketch a methodological guide for identifying, quantifying and reducing uncertainties in astrophysical measurements, hoping that such a guide can not only help to re-frame the current Hubble tension, but serve as a starting point for future fruitful discussions between astrophysicists, astronomers and philosophers.
... Quality could be assessed much more easily and effectively if replications were commonplace (Schmidt, 2016). Unfortunately, Psychology journals typically are reluctant to accept pure replications, with replications accounting for a miniscule of the published psychology literature. ...
... It may appear logical to claim that a replication study cannot be considered new or important knowledge [9]. This study, on the other hand, is predicated on the assumption that replication in scientific research is a valuable strategy that can show scholars whether certain insights are acceptable in circumstances other than those anticipated in the initial study, rather than merely the result of specific circumstances not repeated in other studies [10]. According to [11], replicating scientific investigations is a good strategy for improving dependability and credibility while expediting discoveries. ...
The advancement of technology is making university campuses smarter every single day. Despite the benefits of these advanced technologies, the literature concerning the adoption of smart campuses is significantly lacking increased knowledge to provide effective smart campus solutions. This study aims to prioritize the adoption factors of an IoT-based smart campus. The study applied an analytical hierarchical process (AHP) on 25 factors drawn from the literature. The factors were classified into technology specific factors (TSF), organizational specific factors (OSF), environmental specific factors (ESF), and end-user specific factors (USF). Based on the results obtained, the most significant contributing factors were government support, privacy concerns, social influence, facilitating conditions, and service collaboration, whereas the least significant contributing factors were enjoyment, availability, reliability, mobility, and compatibility. Moreover, based on the global ranking computation, 12 factors from the OSF, ESF, and USF categories appeared to be more significant than TSF. The findings of this study could help university administrators, manufacturers, and policy-makers to understand the critical factors of smart campuses in order to improve the adoption and utilization of these solutions effectively.
... During negative feedback processing, enhanced DLPFC activation has been associated with subsequent aggression reduction (Riva et al., 2015;Achterberg et al., 2016Achterberg et al., , 2018. Given that replicability of fMRI findings in childhood is still relatively understudied and taskbased fMRI shows low test-retest reliability (Elliott et al., 2020), the first aim of this study was to replicate previously observed neural responses to social feedback and the relation to individual differences in aggression in a large sample of children aged 7-9 years (Open Science Collaboration, 2015;Schmidt, 2016;van IJzendoorn and Bakermans-Kranenburg, 2021). ...
This study investigated behavioral and neural correlates underlying social feedback processing and subsequent aggressive behaviors in childhood in two age cohorts (test sample: n=509/n=385 and replication sample: n=354/n=195, 7-9 years old). Using a previously validated Social Network Aggression Task (Achterberg et al., 2020), we showed that negative social feedback resulted in most behavioral aggression, followed by less aggression after neutral and least aggression after positive feedback. Receiving positive and negative social feedback was associated with increased activity in the insula, medial prefrontal cortex and ventrolateral prefrontal cortex. Responding to feedback was associated with additional activation in the dorsolateral prefrontal cortex following positive feedback. This DLPFC activation correlated negatively with aggression. Furthermore, age analyses showed that older children showed larger reductions in aggression following positive feedback and more neural activation in dorsolateral prefrontal cortex when responding to positive feedback compared to younger children. To assess the robustness of our results, we examined these processes in two independent behavioral/fMRI samples using equivalence testing, thereby contributing to replicable reports. Together, these findings demonstrate an important role of social saliency and regulatory processes where regulation of aggression rapidly develops between ages 7-9 years.
How Does the German Educational Science ›Tick‹ and What Does It Do?
Pedagogical perspectives on the one hand and perspectives of educational science perspectives on the other have different motives, objectives and forms of legitimation. It makes little sense to conceive them as opposites, nor is it convincing to try to reconcile them. The article is intended as an essay, parts of it are speculative and some passages seem to lack decency, this is especially true of the third part, which is entitled »Family Feuds and Family Resemblances«. Educational science can be seen as a large family. As in every family, its members are preoccupied with themselves and sometimes have nothing to say to each other, or they argue for a long time without any winners emerging. Do we need to worry about this family? Yes, in almost every respect, except for its existence.
Zusammenfassung
Als konzeptuelle Replikation von Dumont et al. (2017) untersucht diese Studie Effekte der Schulform auf das akademische Selbstkonzept, die Einschätzung eigener Ausbildungschancen und die schulische Motivation von Schüler:innen an nicht-gymnasialen Schulzweigen gegen Ende der Pflichtschulzeit in Klasse 9. Dabei wird anhand von Mehrebenenanalysen geprüft, inwiefern der antizipierte Schulabschluss (mittlerer Schulabschluss [MSA] vs. Hauptschulabschluss [HSA]) und die Klassenzusammensetzung zur Erklärung der Schulformeffekte beitragen. Grundlage der Analysen bilden NEPS-Daten der Startkohorte 3 aus fünf Bundesländern. Um zu prüfen, ob die Effekte über die Zeit variieren, wird zusätzlich zu Klasse 9 ( N = 1277) noch ein weiterer Messzeitpunkt in Klasse 5 herangezogen ( N = 1572). Die Ergebnisse bestätigen, dass Schüler:innen, die einen mittleren Schulabschluss anstrebten, unter Kontrolle ihrer eigenen Testleistung, der Testleistungen ihrer Mitschüler:innen und der besuchten Schulform, nicht nur höhere Selbstkonzepte im Bereich Lesen und allgemein Schule berichteten, sondern auch eine höhere leistungs- und berufsbezogene Lernmotivation vorwiesen, als Schüler:innen, die planten, die Schule mit einem Hauptschulabschluss zu beenden. Dabei fielen die Ergebnisse zu beiden Messzeitpunkten ähnlich aus. Lehrkräfte sollten also dafür sensibilisiert werden, dass schon die Antizipation des Erwerbs eines niedrigeren Schulabschlusses, über Effekte von Schulform und Klassenzusammensetzung hinaus, Nachteile für die motivationale Entwicklung der Schüler:innen mit sich bringen kann.
Several studies confirm the importance of the role of students’ interest in learning mathematics. This article describes the process of conceptual replication of Rellensmann and Schukajlow’s (2017) research on how the connection to the reality of a mathematical problem affects the interest in solving it. Our study distinguishes between intramathematical problems, word problems and modelling problems. It was implemented with 80 Spanish ninth-grade students and 80 pre-service teachers. The results show that Spanish students are more interested in intramathematical problems and less interested in modelling problems, while pre-service teachers are more interested in problems connected to reality, especially word problems. We also provide data regarding the performance of students and prospective teachers, which is higher in word problems. In addition, we find that there are significant relationships between performance and task-specific interest. These results complement the original study, as they allow us to contrast whether there are differences with German students and to explain the German pre-service teachers’ judgements of students’ interest in problems with and without a connection to reality.
The impact sheet to this article can be accessed at 10.6084/m9.figshare.25507636 .
The current study investigated how the natural number bias might hamper secondary students by replicating Van Hoof et al. (2013) previous research with a Mexican sample ( n = 105). We asked grades 1 and 3 students to solve an online fraction comparison task with items consistent or inconsistent with the natural number bias. Students’ accuracies decreased in incongruent items compared to congruent items. Also, they needed more time to solve incongruent items correctly, suggesting the activation of analytical processes. However, there were no differences between the reaction times of incorrectly answered incongruent items and those of correctly answered congruent items, which indicated that students did not detect conflicts while solving incongruent items. These results partially replicate Van Hoof et al. (2013) and further support the Dual-process account of the natural number bias. We discuss how replication studies might help to deepen the understanding of the natural number bias.
The Impact Sheet to this article can be accessed at https://doi.org/10.6084/m9.figshare.24220003
In this paper, we report on a conceptual replication of a fractions test originally administered in the 1970s in England as part of the ‘Concepts in Secondary Mathematics and Science’ ( CSMS ) research programme documenting lower secondary students’ understanding of mathematics. We present data from a recent study conducted in Denmark with 336 students aged between 12 and 14 years, presenting descriptive results as in the original research. In addition, we use Rasch modelling — a technique that was not widely used in the 1970s — to further analyse the students’ understanding of fractions and to validate the test. Our findings indicate that Danish students’ fractional understanding in word or diagrammatic contexts is slightly stronger in comparison to that of the English students in the original study, whereas the case for computations involving fractions is the reverse. We discuss the possible reasons for these differences. Our study provides evidence that fractional knowledge involves a number of components and suggest ways in which the test might be improved.
The Impact Sheet to this article can be accessed at 10.6084/m9.figshare.24219673
Auxiliary hypotheses AHs are indispensable in hypothesis-testing, because without them specification of testable predictions and consequently falsification is impossible. However, as AHs enter the test along with the main hypothesis, non-corroborative findings are ambiguous. Due to this ambiguity, AHs may also be employed to deflect falsification by providing “alternative explanations” of findings. This is not fatal to the extent that AHs are independently validated and safely relegated to background knowledge. But this is not always possible, especially in the so-called “softer” sciences where often theories are loosely organized, measurements are noisy, and constructs are vague. The Systematic Replications Framework (SRF) provides a methodological solution by disentangling the implications of the findings for the main hypothesis and the AHs through pre-planned series of systematically interlinked close and conceptual replications. SRF facilitates testing alternative explanations associated with different AHs and thereby increases test severity across a battery of tests. In this way, SRF assesses whether the corroboration of a hypothesis is conditional on particular AHs, and thus allows for a more objective evaluation of its empirical support and whether post hoc modifications to the theory are progressive or degenerative in the Lakatosian sense. Finally, SRF has several advantages over randomization-based systematic replication proposals, which generally assume a problematic neo-operationalist approach that prescribes exploration-oriented strategies in confirmatory contexts.
Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings.
This paper explores judgements about the replicability of social and behavioural sciences research and what drives those judgements. Using a mixed methods approach, it draws on qualitative and quantitative data elicited from groups using a structured approach called the IDEA protocol (‘investigate’, ‘discuss’, ‘estimate’ and ‘aggregate’). Five groups of five people with relevant domain expertise evaluated 25 research claims that were subject to at least one replication study. Participants assessed the probability that each of the 25 research claims would replicate (i.e. that a replication study would find a statistically significant result in the same direction as the original study) and described the reasoning behind those judgements. We quantitatively analysed possible correlates of predictive accuracy, including self-rated expertise and updating of judgements after feedback and discussion. We qualitatively analysed the reasoning data to explore the cues, heuristics and patterns of reasoning used by participants. Participants achieved 84% classification accuracy in predicting replicability. Those who engaged in a greater breadth of reasoning provided more accurate replicability judgements. Some reasons were more commonly invoked by more accurate participants, such as ‘effect size’ and ‘reputation’ (e.g. of the field of research). There was also some evidence of a relationship between statistical literacy and accuracy.
Both common pain and anxiety problems are widespread, debilitating and often begin in childhood-adolescence. Twin studies indicate that this co-occurrence is likely due to shared elements of risk, rather than reciprocal causation. A joint genome-wide investigation and pathway/network-based analysis of adolescent anxiety and pain problems can identify genetic pathways that subserve shared etiopathogenetic mechanisms. Pathway-based analyses were performed in the independent samples of: The Quebec Newborn Twin Study (QNTS; 246 twin pairs and 321 parents), the Longitudinal Study of Child Development in Quebec (QLSCD; n = 754), and in the combined QNTS and QLSCD sample. Multiple suggestive associations (p<1×10⁻⁵), and several enriched pathways were found after FDR correction for both phenotypes in the QNTS; many nominally-significant enriched pathways overlapped between pain problems and anxiety symptoms (uncorrected p<0.05) and yielded results consistent with previous studies of pain or anxiety. The QLSCD and the combined QNTS and QLSCD sample yielded similar findings. We replicated an association between the pathway involved in the regulation of myotube differentiation (GO:0010830) and both pain and anxiety problems in the QLSDC and the combined QNTS and QLSCD sample. Although limited by sample size and thus power, these data provide an initial support to conjoint molecular investigations of adolescent pain and anxiety problems. Understanding the etiology underlying pain and anxiety co‐occurrence in this age range is relevant to address the nature of comorbidity and its developmental pathways, and shape intervention. The replication across samples implies that these effects are reliable and possess external validity.
Language and music are two human-unique capacities whose relationship remains debated. Some have argued for overlap in processing mechanisms, especially for structure processing. Such claims often concern the inferior frontal component of the language system located within "Broca's area." However, others have failed to find overlap. Using a robust individual-subject fMRI approach, we examined the responses of language brain regions to music stimuli, and probed the musical abilities of individuals with severe aphasia. Across 4 experiments, we obtained a clear answer: music perception does not engage the language system, and judgments about music structure are possible even in the presence of severe damage to the language network. In particular, the language regions' responses to music are generally low, often below the fixation baseline, and never exceed responses elicited by nonmusic auditory conditions, like animal sounds. Furthermore, the language regions are not sensitive to music structure: they show low responses to both intact and structure-scrambled music, and to melodies with vs. without structural violations. Finally, in line with past patient investigations, individuals with aphasia, who cannot judge sentence grammaticality, perform well on melody well-formedness judgments. Thus, the mechanisms that process structure in language do not appear to process music, including music syntax.
Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.
This commentary, adopting literature from the sociology of science and basic thinking in the interdisciplinary space of science and technology studies, responds and reacts to Derbyshire et al.'s (2022) claims about replication in science. This commentary piece constitutes a modicum of context for thinking about replicability in science and some warnings for the futures and foresight academic community if the reproduction and replication of scientific research is to become a norm in the field.
An already pressing need to evidence the effectiveness of futures and foresight tools has been further amplified by the coronavirus pandemic, which highlighted more mainstream tools' difficulty with uncertainty. In light of this, the recent discussion in this journal on providing futures and foresight science with a stronger scientific basis is welcome. In this discussion critical realism has been proffered as a useful philosophical foundation and experiments a useful method for improving this field's scientific basis. Yet, experiments seek to isolate specific causal effects through closure (i.e., by controlling for all extraneous factors) and this may cause it to jar with critical realism's emphasis on uncertainty and openness. We therefore extend the recent discussion on improving the scientific basis of futures and foresight science by doing three things. First, we elaborate on critical realism and why the experimental method may jar with it. Second, we explain why the distinction between a conceptual and a direct replication can help overcome this jarring, meaning experiments can still be a valuable research tool for a futures and foresight science underpinned by critical realism. Third, we consider the appropriate unit of analysis for experiments on futures and foresight tools. In so doing, we situate the recent discussion on improving the scientific basis of futures and foresight science within the much longer running one on improving the scientific basis of business, management and strategy research more broadly. We use the case of scenario planning to illustrate our argument in relation to futures and foresight science.
Purpose : To examine the consistency of findings between two studies examining the impact of sport education on Chinese physical education preservice teachers’ content knowledge and performance of volleyball and basketball. Methods : One hundred and six preservice teachers’ from a university in central China participated in six semester-long courses of basketball taught using either a Multi-Activity or Sport Education model of instruction. Pre- and postcourse measures of game performance were recorded for common content knowledge and specialized content knowledge. Results : After controlling for preintervention scores, statistically significant differences favoring Sport Education were found for common content knowledge as well as specialized content knowledge. Students in Sport Education had 62 times higher odds of reaching the specialized content knowledge benchmark depth for acceptable content development. Conclusion : These findings provide support for the idea that the accountability mechanisms specific to Sport Education, together with the tasks related to designing team training plans, serve to promote students’ ability to design and sequence tasks based on their team’s needs.
Research concerning rational emotive behaviour therapy (REBT) and autonomous and controlled motivation within athletic settings is burgeoning. It is proposed that irrational beliefs (i.e., illogical, rigid, and extreme) together with controlled forms of motivation, can determine psychological well-being and physical health in these contexts. For example, research indicates that extreme negative self-evaluation (i.e., self-depreciation) is related to more controlled (less autonomous) motivation regulation, which may underpin poorer health. Though, research is yet to understand the concomitant influence of both irrational beliefs and motivation regulation on work related variables such as presenteeism, persistence and turnover intention, as well as non-work-related variables such as life satisfaction and mental-wellbeing. The present two study paper examines the latent profile structure of irrational beliefs and motivation regulation, and how these latent profiles relate to health and work-related variables. Across studies 1 and 2, results indicated a two-class profile whereby class 1 is characterised by low irrational beliefs and high self-determined motivation (low irrational engagement), and class 2 is characterised by high irrational beliefs and low self-determined motivation (high irrational engagement). Those in Class 2 reported poorer life satisfaction, persistence, and presenteeism, as well as greater depression, anxiety, stress, intention to quit, and absenteeism than those in class 1. Thus, findings indicate that poorer work and health outcomes are associated with greater irrational work engagement. The findings are discussed in relation to the practical implications for occupational workers.
Purpose: The aim of this study was to critically examine previous studies’ claims about the magnitude of gender differences and gender inequality in physical education (PE) in Sweden. Method: The data were based on students’ (N = 39,980) perceptions of PE and were gathered from four large research projects in Sweden. Three effect size measures (Cramer’s V, r squared, and Cohen’s d) were calculated for gender differences. Results: In general, there are small gender differences; and after controlling for students’ grade, “sports capital,” and parents’ “educational capital,” the differences are practically irrelevant. Conclusion: This study provides compelling evidence that there are small, or even irrelevant, gender differences in students’ perceptions of PE in Sweden. Moreover, given that previous research asserts that large gender differences can be used as an indicator of inequality, this study suggests that gender inequality issues related to students’ perceptions of PE are relatively small.
Pre-registration is an open research practice that can mitigate against questionable research practices and contribute to enhanced research outcomes, such as increased research transparency. This paper explores barriers and enablers to pre-registration, and develops an evidence-based behaviour change intervention specification to increase its uptake. The Behaviour Change Wheel (BCW) framework of intervention development and COM-B model of behaviour change were used to inform the development of a mixed-methods online questionnaire, assessing barriers and enablers to pre-registration. Data were collected from 18-05-2020 to 12-07-2020, and explored using descriptive statistics, reflexive thematic analysis, and COM-B. BCW was used to develop an intervention specification. Respondents were researchers (n = 105) who were mostly engaged in psychological research (71%) and had pre-registered before (75%). Insufficient knowledge and skill (psychological capability), social support (social opportunity), time (physical opportunity), and incentivisation (reflective motivation) were the most substantial barriers to pre-registration, whereas belief in pre-registration contributing to desirable research outcomes (reflective motivation) was the most substantial enabler. These findings informed the development of an intervention specification to increase pre-registration uptake by researchers. This paper demonstrates the strong potential of BCW to facilitate open research practices. The identified barriers and enablers, intervention specification, and the behaviour change approach outlined, may be used to increase pre-registration uptake; for example, developing new or refining existing training and incentivisation interventions. This paper may inspire others to consider the strong potential of BCW to facilitate open research practices and so contribute to enhanced research outcomes.
The replication of existing research studies and theories is considered a foundational pillar of knowledge accumulation and an important instrument of discourse across research disciplines. Although replication has a long tradition in natural and behavioral science research, the design science research (DSR) community has yet to adopt it, especially the replication of design theories (DTs). However, it is unclear how the DSR community could (or even would) benefit from the replication of DTs. Similarly, the goal of DTs to obtain utility instead of truth raises questions regarding the transferability of replication into the domain of DSR. Against this background, in this work, we reflect on the function, outcome, and impact of replications to understand whether the replication of DTs is possible and necessary. We propose that replication can be an important catalyst for reuse and knowledge accumulation in DSR because it provides evidence on the boundaries of a DT. Specifically, replication can increase or decrease the level of confidence and projectability associated with a DT.
In scientific context, the first step for scientific theoretical and methodological production is the epistemological analysis. What are assumptions for interaction between oncology and psychology? What are the conditions for psycho-oncological contribution in treating cancer? Furthermore, what are epistemological observations about the current developments in cancer field? And what are implications for sciences treating patients with neoplasms? Due to advances in oncology and in sciences supporting oncology, epistemological questions focus on the object of study of the integration between oncology and psychology. Therefore, the purpose of this study is to describe a proposal of theoretical and methodological frame suitable for current clinical and research needs in cancer patients asking for psychological support. Epistemological analysis lets the health professionals and researchers observe there are sciences using formal languages and sciences using ordinary language. Currently, personalized approach is pursued by oncology, identifying specific patients’ characteristics to define the proper treatment process: not only tumor characteristics but also behavioral and psychological features. Cancer patients features can be found in patients’ narrations about neoplasms: narration represents the core of clinical and research in psychoncology. Therefore when formalized, language provides the connection between oncology and psychology. Language used by patients and all the roles involved in the care of cancer patients can become a measure of these patients’ features. Dialogics science measuring the ordinary language allows the ordinary language formalization, pursuing a personalized medicine.
Recent years have introduced major shifts in scientific reporting and publishing. The scientific community, publishers, funding agencies, and the public expect research to adhere to principles of openness, reproducibility, replicability, and repeatability. However, studies have shown that scientists often have neither the right tools nor suitable support at their disposal to meet these modern science challenges. In fact, even the concrete expectations connected to these terms may be unclear and subject to field-specific, organizational, and personal interpretations. Based on a narrative literature review of work that defines characteristics of open science, reproducibility, replicability, and repeatability, as well as a review of recent work on researcher-centered requirements, we find that the bottom-up practices and needs of researchers contrast top-down expectations encoded in terms related to reproducibility and open science. We identify and define reproducibility as a central term that concerns the ease of access to scientific resources, as well as their completeness, to the degree required for efficiently and effectively interacting with scientific work. We hope that this characterization helps to create a mutual understanding across science stakeholders, in turn paving the way for suitable and stimulating environments, fit to address the challenges of modern science reporting and publishing.
We evaluated two quantitative models that attempt to account for the effects of amount on probability discounting. Sixty-four undergraduate psychology students completed four probability discounting tasks that differed in the amount of the probabilistic outcome (20, 3,000, 500,000). In an amount-independent hyperboloid model, the exponent (s) systematically increased with reward amount. These systematic changes were not explained by the model. Myerson et al. (2011) reasoned that s increases as a power function of the amount of the probabilistic reward. We found support for the amount-dependent hyperboloid model at the aggregate and individual levels of analysis. The results replicate and extend previous research and suggest that the value function of a probabilistic outcome is weighted by, or depends on, its amount.
The replicability crisis refers to the apparent failures to replicate both important and typical positive experimental claims in psychological science and biomedicine, failures which have gained increasing attention in the past decade. In order to provide evidence that there is a replicability crisis in the first place, scientists have developed various measures of replication that help quantify or “count” whether one study replicates another. In this nontechnical essay, I critically examine five types of replication measures used in the landmark article “Estimating the reproducibility of psychological science” (Open Science Collaboration, Science, 349, ac4716, 2015) based on the following techniques: subjective assessment, null hypothesis significance testing, comparing effect sizes, comparing the original effect size with the replication confidence interval, and meta-analysis. The first four, I argue, remain unsatisfactory for a variety of conceptual or formal reasons, even taking into account various improvements. By contrast, at least one version of the meta-analytic measure does not suffer from these problems. It differs from the others in rejecting dichotomous conclusions, the assumption that one study replicates another or not simpliciter. I defend it from other recent criticisms, concluding however that it is not a panacea for all the multifarious problems that the crisis has highlighted.
Reproducibility is a confused terminology. In this paper, I take a fundamental view on reproducibility rooted in the scientific method. The scientific method is analysed and characterized in order to develop the terminology required to define reproducibility. Furthermore, the literature on reproducibility and replication is surveyed, and experiments are modelled as tasks and problem solving methods. Machine learning is used to exemplify the described approach. Based on the analysis, reproducibility is defined and three different degrees of reproducibility as well as four types of reproducibility are specified.
This article is part of the theme issue ‘Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico ’.
Although reproducibility-the idea that a valid scientific experiment can be repeated with similar results-is integral to our understanding of good scientific practice, it has remained a difficult concept to define precisely. Across scientific disciplines, the increasing prevalence of large datasets, and the computational techniques necessary to manage and analyze those datasets, has prompted new ways of thinking about reproducibility. We present findings from a qualitative study of a NSF-funded two-week workshop developed to introduce an interdisciplinary group of domain scientists to data-management techniques for data-intensive computing, with a focus on reproducible science. Our findings suggest that the introduction of data-related activities promotes a new understanding of reproducibility as a mechanism for local knowledge transfer and collaboration, particularly as regards efficient software reuse.
Beset by detection of replication failures and questionable research practices over the last decade, psychology has been deemed by many to be in crisis. The situation is exceptional not only for the many investigative practices being scrutinized but also for the attention given to the undue influence of psychologists' psychology on those practices. Comparative analysis of 2 crises finds that the earlier one focused on the experimenters' activities within the laboratory, whereas the current concerns center on experimenters' postexperimental work. Whereas the previous crisis did include deep concerns about experimenters, the currently offered psychologies of fellow psychologists are distinctive in their frequency, intensity, and considerable reliance upon established knowledge about human thought and behavior. In so utilizing scientific psychology to assess psychology, the current appraisals give richer evidence of the circuitry of psychological knowledge as it travels from the laboratory outward and back. They give considerable attention to the scientists' moral characteristics, whereas the earlier crisis generated concerns about experimenters' conduct in the laboratory and the politics surrounding the application of psychological knowledge. Through their direct discussions of personal and moral conduct, the assessments also resonate with uncertainties about scientists' self-control, normative ethics, and emotions. Taken together, the psychologies and attendant uncertainties illuminate present conditions of psychology's scientific self and invite reflection on the close connections between that self, ethos, and epistemology. (PsycINFO Database Record (c) 2020 APA, all rights reserved).
Introduction:
Irrespective of the type of psychotherapy used, the abstinence-oriented treatment of drug abusers is less successful than that for alcohol abusers. If, on the other hand, the two groups are parallelized in such a way that the patients are identical with respect to the five characteristics of gender, age, schooling, work situation and partner situation, then there is no difference between the success rates of the drug and alcohol abusers. The aim of this study is to determine whether this result can be replicated in another therapeutic institution.
Method:
Retrospective field study of 320 abusers of illegal drugs and 320 alcohol abusers who were treated with behaviour therapy. By combining the binary characteristics gender, work situation and age, the drug-dependent patients were divided into 23=8 groups, and the same number of alcohol abusers were randomly selected for each group. The scheduled period of inpatient treatment was 90 days for the alcohol abusers and 120 days for the drug abusers. Every week the patients had one session of individual psychotherapy and four to five group therapy sessions. According to the indications, the certified behaviour therapists implemented the following interventions including behaviour analysis, relapse prevention, cognitive therapy, self-management and behavioural family therapy. Comparison of the success rates was carried out using the Chi2 test, and changes in the psychological findings were tested with one-way variance analysis.
Results:
There was no difference between drug and alcohol abusers with respect to the rate of therapy termination according to plan (around 80%). A total of 48% of the drug abusers and 41 % of the alcohol abusers who could be followed up had been continuously abstinent at the one-year catamnesis without a single relapse. There were also no differences between the two groups when it was assumed that the patients who could not be followed up had relapsed. In the case of both the drug and alcohol abusers the abstinence rate was highest in over-29-year-old employed men (57.6%; 48.4%). The abstinence rate was lowest in employed female drug abusers (27.8%) and young, unemployed female drug abusers (0%, n=11).
Discussion:
What appears to influence the abstinence rate after inpatient treatment is not only the type of substance consumed but also sociodemographic characteristics. In addition to individually tailored therapy, our results confirm the importance of a highly differentiated presentation of the outcomes of therapy in the specialist literature. An average rate of abstinence (e.g. 30%) is insufficient to evaluate an intervention unless information is also provided about the patients for which the intervention is suitable and those for which it is not. In accordance with the Reproducibility Project, we consider replication studies essential in psychotherapy, even though in practice the considerable methodical requirements can only be partially fulfilled.
Background:
There has long been debate about the balance between fidelity to evidence-based interventions (EBIs) and the need for adaptation for specific contexts or particular patients. The debate is relevant to virtually all clinical areas. This paper synthesises arguments from both fidelity and adaptation perspectives to provide a comprehensive understanding of the challenges involved, and proposes a theoretical and practical approach for how fidelity and adaptation can optimally be managed.
Discussion:
There are convincing arguments in support of both fidelity and adaptations, representing the perspectives of intervention developers and internal validity on the one hand and users and external validity on the other. Instead of characterizing fidelity and adaptation as mutually exclusive, we propose that they may better be conceptualized as complimentary, representing two synergistic perspectives that can increase the relevance of research, and provide a practical way to approach the goal of optimizing patient outcomes. The theoretical approach proposed, the "Value Equation," provides a method for reconciling the fidelity and adaptation debate by putting it in relation to the value (V) that is produced. The equation involves three terms: intervention (IN), context (C), and implementation strategies (IS). Fidelity and adaptation determine how these terms are balanced and, in turn, the end product - the value it produces for patients, providers, organizations, and systems. The Value Equation summarizes three central propositions: 1) The end product of implementation efforts should emphasize overall value rather than only the intervention effects, 2) implementation strategies can be construed as a method to create fit between EBIs and context, and 3) transparency is vital; not only for the intervention but for all of the four terms of the equation. There are merits to arguments for both fidelity and adaptation. We propose a theoretical approach, a Value Equation, to reconciling the fidelity and adaptation debate. Although there are complexities in the equation and the propositions, we suggest that the Value Equation be used in developing and testing hypotheses that can help implementation science move toward a more granular understanding of the roles of fidelity and adaptation in the implementation process, and ultimately sustainability of practices that provide value to stakeholders.
ResearchGate has not been able to resolve any references for this publication.