ArticlePDF Available

Abstract and Figures

Research-based learning (RBL) is regarded as a panacea when it comes to effective instructional formats in higher education settings. It is said to improve a wide set of research-related skills and is a recommended learning experience for students. However, whether RBL in the social sciences is indeed as effective as has been postulated for other disciplines has not yet been systematically examined. We thus administered a pre–post-test study to N = 952 students enrolled in 70 RBL courses at 10 German universities and examined potential changes in cognitive and affective-motivational research dispositions. Latent change score modelling indicated that students increased their cognitive research dispositions, whereas most affective-motivational research dispositions decreased. The instructors’ interest in the students’ work served as a significant predictor of changes in research interest and joy. Practical implications for designing RBL environments can be inferred from the results.
Content may be subject to copyright.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=cshe20
Studies in Higher Education
ISSN: 0307-5079 (Print) 1470-174X (Online) Journal homepage: https://www.tandfonline.com/loi/cshe20
Is research-based learning effective? Evidence
from a pre–post analysis in the social sciences
Insa Wessels, Julia Rueß, Christopher Gess, Wolfgang Deicke & Matthias
Ziegler
To cite this article: Insa Wessels, Julia Rueß, Christopher Gess, Wolfgang Deicke & Matthias
Ziegler (2020): Is research-based learning effective? Evidence from a pre–post analysis in the
social sciences, Studies in Higher Education, DOI: 10.1080/03075079.2020.1739014
To link to this article: https://doi.org/10.1080/03075079.2020.1739014
© 2020 The Author(s). Published by Informa
UK Limited, trading as Taylor & Francis
Group
View supplementary material
Published online: 12 Mar 2020.
Submit your article to this journal
View related articles
View Crossmark data
Is research-based learning eective? Evidence from a prepost
analysis in the social sciences
Insa Wessels
a,b
, Julia Rueß
a
, Christopher Gess
a
, Wolfgang Deicke
a
and Matthias Ziegler
b
a
bologna.lab, Humboldt-Universität zu Berlin, Berlin, Germany;
b
Department of Psychology, Humboldt-Universität zu
Berlin, Berlin, Germany
ABSTRACT
Research-based learning (RBL) is regarded as a panacea when it comes to
eective instructional formats in higher education settings. It is said to
improve a wide set of research-related skills and is a recommended
learning experience for students. However, whether RBL in the social
sciences is indeed as eective as has been postulated for other
disciplines has not yet been systematically examined. We thus
administered a prepost-test study to N= 952 students enrolled in 70
RBL courses at 10 German universities and examined potential changes
in cognitive and aective-motivational research dispositions. Latent
change score modelling indicated that students increased their
cognitive research dispositions, whereas most aective-motivational
research dispositions decreased. The instructorsinterest in the students
work served as a signicant predictor of changes in research interest
and joy. Practical implications for designing RBL environments can be
inferred from the results.
KEYWORDS
Research-based learning;
research knowledge; research
literacy; aective-
motivational research
dispositions; social sciences
didactics
Introduction
Teaching and research can be linked through a variety of well-dened instructional formats. One
of these is research-based learning (RBL), in which students conduct their own research with the
help of a supervisor. RBL is currently seen as a panacea for addressing a range of demands within
higher education, e.g. a lack of meaningful learning experiences and the need for stimulating
instructional formats. Accordingly, several authors and institutions claim that RBL should be
incorporated into the curriculum of many if not every academic study programme (e.g. Healey
and Jenkins 2009). Indeed, a growing number of programmes have attempted to implement
RBL in a range of disciplines and forms, e.g. the REU programme by the US National Science
Foundation. The main goal of these endeavours is to provide students with an opportunity to
experience participation in research. In science, technology, engineering, and mathematics
(STEM) disciplines, there is evidence that RBL does indeed live up to its promises and constitutes
an eective learning experience (Linn et al. 2015). However, outside the STEM disciplines, it is still
unclear which research dispositions RBL fosters. Thus, this study aims to examine whether RBLs
eectiveness regarding the acquisition of various cognitive and aective-motivational research
dispositions can be generalised to the social sciences.
© 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://
creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the
original work is properly cited, and is not altered, transformed, or built upon in any way.
CONTACT Insa Wessels insa.wessels@hu-berlin.de
Supplemental data for this article can be accessed at https://doi.org/10.1080/03075079.2020.1739014.
STUDIES IN HIGHER EDUCATION
https://doi.org/10.1080/03075079.2020.1739014
Theoretical background
Positioning research-based learning in relation to other forms of research-related
teaching
Teaching and research can be linked in dierent ways. In a popular model, Healey and Jenkins (2009)
distinguish among dierent instructional formats for engaging students in research along two axes.
The rst axis describes whether the research results or the research process is emphasised. The other
axis describes whether students take on an active role as participants or a passive role as audience.
These two axes can be combined into four dierent formats: research-tutored, research-led, research-
oriented and research-based learning. In RBL, teaching focuses on the research process, and students
actively conduct research and inquiry. However, this description fails to describe the exact nature of
studentsinvolvement in research. Huber (2014) further denes RBL as an instructional format in
which students work through the entire research process in a self-regulated manner, guided by
their own research questions. The instructor takes on a facilitating role. This theoretically derived
denition was replicated in an empirical classication of research-related formats (Rueß, Gess, and
Deicke 2016) and serves as the underlying denition of RBL in the current study.
1
The eectiveness of research-based learning
Conducting ones own research project involves various cognitive, behavioural, and aective experi-
ences (Lopatto, 2009, 29), which in turn lead to a wide range of benets associated with RBL.
RBL is associated with long-term societal benets because it can foster scientic careers: Students
participating in RBL reported a greater interest in pursuing postgraduate education or PhDs (Lopatto
2007; Russell, Hancock, and McCullough 2007) and were more likely to be engaged in scientic
careers six years after graduation (Hernandez et al. 2018).
In addition, RBL fosters research skills that are also necessary for occupations outside academia
(British Academy 2012). RBL is said to facilitate the development of a researchers mindset’–the
ability to objectively examine data or a situation and nding enjoyment in solving problems
(Wood 2003). A researchers mindset can be eective in a wide range of professional activities. For
example, in the eld of psychotherapy, therapists could draw upon their research knowledge to
consult evidence on new therapeutic approaches (Levant et al. 2006). Hence, the acquisition of
research-related knowledge and skills is a prerequisite for successfully engaging in both scientic
and non-scientic careers making it an appropriate focus for our article.
Successfully engaging in a task requires both cognitive dispositions, such as knowledge, and
aective-motivational dispositions to put this knowledge into practice (Blömeke, Gustafsson, and
Shavelson 2015). Disposition serves as an umbrella term to denote a range of latent, personal
resources (e.g. attitudes, traits and abilities) that determine how an individual will normally act in a
certain situation (Schmidt-Atzert and Amelang 2012, 63). Accordingly, competent performance in
the research domain requires various cognitive (e.g. knowledge) and aective-motivational (e.g.
interest) research dispositions. Whether RBL is eective at facilitating the development of dierent
cognitive and aective-motivational research dispositions has been the focus of previous studies.
The existing evidence will be introduced in the following sections.
Cognitive gains
Most empirical studies on the eectiveness of RBL focus on cognitive research dispositions. However,
the majority of these studies assessed STEM students (e.g. Linn et al. 2015; Seymour et al. 2004), with
only a few studies investigating the eect of RBL in the social sciences. In a study from the eld of
social work, students gained domain-general research knowledge (Whipple, Hughes, and Bowden
2015). Taraban and Logue (2012) found evidence for a range of cognitive benets of psychology stu-
dentsparticipation in research, such as improved research methods skills. Participation in RBL can
2I. WESSELS ET AL.
also lead to increased understanding of the scientic process as a whole (Lloyd, Shanks, and Lopatto
2019).
Other researchers have examined specic skills pertaining to individual research steps, e.g. the
ability to use statistics software (Whipple, Hughes, and Bowden 2015) and communicating and pre-
senting ones research (Stanford et al. 2017). RBL also seems to facilitate more general cognitive dis-
positions like critical thinking (Hunter, Laursen, and Seymour 2007; Kilgo, Ezell Sheets, and Pascarella
2015) and the ability to work independently (Stanford et al. 2017).
Thus, while RBL in the social sciences seems to be eective at facilitating a range of dierent cog-
nitive dispositions, these results can only serve as preliminary evidence. A problem concerning the
interpretability of these and other studies in the eld lies in their methodological designs: Most exist-
ing studies focus on subjective ex-post assessments and self-evaluated skill gains (e.g. Stanford et al.
2017). However, self-assessments are often distorted by personality (John and Robins 1994) or skill
levels themselves (unskilled students overestimate their abilities, see Kruger and Dunning 1999).
Large-scale investigations using objective measures provide more substantial conclusions, but
have so far only been completed for STEM students (e.g. Russell, Hancock, and McCullough 2007).
Linn et al. (2015) note that the underlying problem is a lack of valid measures to objectively investi-
gate the eectiveness of RBL. To address this problem, the Social-scientic Research Competency Test,
an objective measure of cognitive research dispositions in the social sciences, was developed by Gess
and colleagues (Gess, Geiger, and Ziegler 2018; Gess, Wessels, and Blömeke 2017). The instrument is
based on a coherent model of dierent areas of research knowledge necessary to conduct critical
steps in the research process (see Appendix 1, online supplemental data). In validation studies, the
instrument has been shown to be suitable for evaluating social-scientic research education and
could serve as an objective measure of the cognitive benets of RBL.
Aective-motivational gains
Higher education research is increasingly acknowledging the importance of aective-motivational
aspects for learning (e.g. Postareand Lindblom-Ylänne 2011). Reecting this general trend,
aective-motivational gains have also drawn increased attention in research on RBL.
Evidence on RBLs potential to alter aective-motivational research dispositions often stems from
studies with multidisciplinary samples. Demonstrated benets include higher research self-ecacy
(Deicke, Gess, and Rueß 2014; Whipple, Hughes, and Bowden 2015), increased intellectual curiosity
(Bauer and Bennett 2003) and a higher tolerance for obstacles in the research process (Lloyd,
Shanks, and Lopatto 2019). Furthermore, a study with STEM students demonstrated a greater
desire to learn and an increased disposition towards working with ambiguity (Ward, Bennett, and
Bauer 2003).
The few existing studies all examine individual aective-motivational research dispositions, often
in an exploratory manner. However, conducting research is an especially demanding task that
requires students to handle uncertainties and manifold frustrations (John and Creighton 2011).
Thus, it can be assumed that successfully conducting research requires a range of dierent
aective-motivational dispositions to cope with the challenges of the research process. A coherent,
empirically grounded model of the aective-motivational research dispositions necessary for student
research in the social sciences has been recently developed (Wessels et al. 2018). It encompasses dis-
positions that are necessary to begin and to sustain the research process: for example, research inter-
est is needed to initiate a research process, while sustaining it requires frustration tolerance to cope
with inevitable setbacks. It is unclear whether RBL is eective in developing these research
dispositions.
Overall, studies on the nature and eectiveness of RBL in the social sciences are generally scarce
and often based on weak methodological designs in contrast to studies from other disciplines.
However, one cannot assume that the evidence gained in studies with STEM students easily trans-
lates to the social sciences. First, research seems more important to university programmes in the
natural sciences than in the social sciences (cf. Taraban and Logue 2012). Second, most research
STUDIES IN HIGHER EDUCATION 3
experiences within STEM disciplines occur in structured lab environments that might have a dierent
pedagogical culture (Rand 2016). Third, if discipline-specic outcome variables are to be investigated,
a study needs to be conducted in that specic discipline.
Another open question pertains to the processes by which RBL in the social sciences aects
changes in dierent research dispositions. In studies with STEM students, the main predictors of
learning gains are the duration and intensity of the research experience: longer-lasting and more
intense research experiences lead to stronger increases in skill levels (Bauer and Bennett 2003).
Another study found that students with higher levels of autonomy in the research process, e.g.
the autonomy to make their own methodological decisions, showed stronger learning gains
(Gilmore et al. 2015). However, which characteristics of RBL courses in the social sciences aect
changes in dierent research dispositions has not been studied yet.
Research questions and hypotheses
The objective of this paper is to analyse the eectiveness of RBL courses in the social sciences. Two
main research questions guided our work: (1) Does research-based learning have a positive eect on
cognitive and aective-motivational research dispositions? (2) How do dierent course characteristics
relate to changes in these research dispositions?
Pertaining to the rst research question, the following hypotheses were tested:
Hypothesis 1a: As previous studies have found associations between student research experiences and self-eval-
uated knowledge gains (Taraban and Logue 2012), we predict that students will have signicantly higher post-
test scores than pre-test scores for research knowledge (knowledge of methods, knowledge of methodologies
and research process knowledge).
Hypothesis 1b: As previous studies have found associations between student research experiences and a higher
tolerance for obstacles in the research process (Lloyd, Shanks, and Lopatto 2019) as well as an increased ability to
work with ambiguity (Ward, Bennett, and Bauer 2003), we predict that students will have signicantly higher post-
test scores than pre-test scores for aective-motivational research dispositions.
Pertaining to the second research question, the following hypotheses were tested:
Hypothesis 2a: Since studies in STEM disciplines have demonstrated that longer and more intense research experi-
ences (Bauer and Bennett 2003) have a positive inuence on the eect of participation in RBL, we predict that the
intensity of the research experience, i.e. the number of research steps performed, will inuence changes in
research knowledge.
Hypothesis 2b: Since studies in STEM disciplines have demonstrated that higher levels of autonomy in the research
process (Gilmore et al. 2015) positively impact the eect of participation in RBL, we predict that studentsauton-
omy, i.e. ability to freely choose a research question and a research method, will positively aect changes in
aective-motivational research dispositions.
Hypothesis 2c: We predict that dierent motivating factors, e.g. studentsself-ecacy, the perception that they are
doing real research, perceived instructor interest in the studentswork, and the perceived usefulness of RBL for
their later career will positively aect changes in aective-motivational dispositions.
Methods
To answer our research questions, paper-based measurements were conducted at the beginning and
the end of RBL courses oered in dierent social scientic disciplines at 10 dierent universities.
Procedure
As the objective was to study comparable RBL courses in the social sciences, only the curricula of
study programmes employing empirical social science research methods were considered. These
4I. WESSELS ET AL.
included sociology, political science, psychology, and education science (see also Gess, Wessels, and
Blömeke 2017).
Suitable RBL courses were identied via their course descriptions. Only courses that allowed stu-
dents to experience a full research cycle in a self-regulated manner were considered, in line with our
denition of RBL. The instructors of 146 courses were contacted via email and asked to participate in
the study; 65 agreed to participate, 50 did not wish to participate, mostly due to time constraints in
the course, and the remaining 31 instructors did not respond. Pre-tests were scheduled for the rst
two weeks of the course, and post-tests for the last two weeks of the course.
Altogether, pre- and post-measurements were conducted in N= 70 RBL courses at 10 universities
across Germany. All universities included were state-funded public universities with 10,000-50,000
students oering degrees in a wide range of disciplines.
The testing itself was conducted during class time by one of the authors of this article, who
explained the procedure and general purpose of the study. The questionnaires were administered
in the form of printed booklets. A personal 6-digit code based on non-sensitive information, e.g. birth-
day month, was used to match pre- and post-test questionnaires while granting anonymity. Filling in
the questionnaire took approximately 25 min. The post-test followed the same procedure. Addition-
ally, a brief instructor survey on characteristics of the course instruction was administered.
Sample
The sample encompassed N= 952 students (74.1% female, 23.5% male), of which 881 participated in
the rst measurement and 539 participated in the second measurement. Higher participation rates at
the rst measurement point were due to higher course attendance at the beginning of the semester.
The mean age of the participating students was M= 24.38 years (SD = 4.79). 61.6% of the students
were enrolled in a bachelors programme, while 29.5% were enrolled in a masters programme. Fifty
students were enrolled in other study programmes, such as the traditional German university
diploma, and were treated as either bachelors or masters students depending on their study pro-
gress. Bachelors students were near the end of their second year of study on average; the mean
number of semesters completed was M= 3.33 (SD = 1.67). Masters students were at the beginning
of their second year of masters studies on average, with M= 2.57 (SD = 1.63) semesters of the
degree completed on average.
The students were enrolled in dierent elds of study, namely educational science (31.4% of the
students), psychology (22.4%), sociology (10.3%), communication science (8.6%), and political science
(5.5%) The remaining students were studying other, more specic social scientic subjects (i.e. media
studies).
The students were enrolled in one of 70 RBL courses. Participation was often a mandatory part of
the studentsstudy programmes: 41.8% of the students were required to enrol in this specic course;
an additional 35.7% could have chosen a dierent RBL course, while only 17.6% could have chosen a
course not involving the instructional format of RBL. The average number of participants per course
was M= 13.54 (SD = 12.62). The majority of students were enrolled in one-semester courses (77.7%);
22.3% of the students were enrolled in two-semester courses. The courses were led by 65 dierent
instructors or co-teaching teams. Fifty two of these instructors participated in the instructors
survey at the end of the course.
Measures
Research knowledge
A 9-item short version of the social-scientic research competence measure by Gess, Wessels, and
Blömeke (2017) was used to assess research knowledge in the social sciences. This test assesses
knowledge of research methods, knowledge of methodologies and research process knowledge
with items referring to both quantitative and qualitative research. The test uses short vignettes
STUDIES IN HIGHER EDUCATION 5
coupled with multiple choice questions on dierent research problems (see sample item in Appen-
dix 1, online supplemental data). The instrument has gone through several validation studies and is
suitable for the evaluation of research courses in the social sciences in both bachelorsandmasters
degree programmes (Gess, Geiger, and Ziegler 2018; Gess, Wessels, and Blömeke 2017). Since the
full 27-item measure takes 35 min to complete and in-class time was sparse, a 9-item short version
reecting the full breadth of the original test in terms of content areas was developed based on
the discrimination parameters, item diculty, reliability and correlation with the long version.
The correlation of the person scores for the short version and the person scores for the long
version is r= 0.86, which indicates that the two versions measure a similar construct. However, it
mustbenotedthattheshortversionofthetesthasnotundergonethesamevalidationprocedure
as the long version. The studentsanswers were coded as either correct (1) or incorrect (0), such that
the nal data consisted of 9 dichotomous items. The reliability was acceptable, with weighted
omega h=0.69(see Table 1).
Aective-motivational research dispositions
The model of aective-motivational research dispositions (Wessels et al. 2018) encompasses nine
necessary dispositions for pursuing research in the social sciences, of which four were selected to
be investigated in the present study. (1) Value-related interest in research subsumes beliefs about
the usefulness of research. (2) Finding joy in conducting research denotes the joy experienced with
respect to dierent research activities. (3) Research-related uncertainty tolerance is the disposition
to handle uncertainties in the research process. (4) Research-related frustration tolerance is the dispo-
sition to endure setbacks in the research process.
Self-assessment scales (sample items and basic descriptive data can be found in Table 1) were
developed in a multistep process following deductive and inductive test construction procedures
(Burisch 1984). First, at least 20 items per disposition were constructed according to xed theory-
driven construction principles (Wilson 2005). The items were selected and rened based on a pilot
study with N= 250 students from the social sciences. The nal instruments encompass 4 or 5
items per disposition and exhibit acceptable or good reliabilities (weighted omega h= 0.680.82).
The response format for all aective-motivational measures was a ve-point Likert scale ranging
from 1 (completely disagree) to 5 (completely agree).
Table 1. Sample items, means, standard deviation and weighted omega of the variables at T1 and T2.
Disposition with sample item
Number of
items Mean (SD)
Weighted
Omega h
1. Research knowledge T1
Sample item see Appendix 1, online supplemental data
9 0.46 (0.16) .69
2. Research knowledge T2 9 0.50 (0.16) .66
3. Value-related interest in research T1
Compared to other topics, I assign a high value to research.
5 3.96 (0.46) .80
4. Value-related interest in research T2 5 3.86 (0.50) .80
5. Joy in working with scientic literature T1
I enjoy reading the scientic literature on a topic.
4 3.20 (0.80) .77
6. Joy in working with scientic literature T2 4 3.03 (0.85) .82
7. Joy in working with empirical data T1
I enjoy analyzing data.
4 3.44 (0.47) .68
8. Joy in working with empirical data T2 4 3.45 (0.57) .74
9. Uncertainty tolerance T1
Ind it disturbing that before I start my research project, I dont know
whether everything will work out as I imagine it will.
4 2.71 (0.73) .73
10. Uncertainty tolerance T2 4 2.83 (0.78) .75
11. Frustration tolerance T1
If my data analysis turns out to be incorrect and I have to start all over
again, I would probably despair.
4 2.57 (0.54) .71
12. Frustration tolerance T2 4 2.55 (0.56) .76
6I. WESSELS ET AL.
Instructor and course characteristics
Student survey. During the pre- and post-test, students were asked for additional information. At
pre-test, this included their self-assessed research self-ecacy (6 items on a 5-point scale, e.g. I
am sure I can nd suitable assessment tools for a quantitative study, even if the main variable is
dicult to operationalize). At post-test, students were asked about the research steps (e.g. searching
for relevant literature) they had completed so far, their perception of the instructors interest in their
research project, their perception of whether they were doing realresearch, and the perceived use-
fulness of the course for their later career (all measured with one item each on a 5-point scale).
Instructor survey. The post-test was also used to gather information about the courses instructional
concept from the instructors perspective. A 5-minute questionnaire distributed to the instructors
asked about studentsautonomy in choosing their own research question and method (two items
on a ve-point scale).
Statistical analysis
In a rst step, studentspre- and post-test data were matched via their personal six-digit code. We
used SPSS 23 to conduct data checks and descriptive analyses of the manifest variables. To investi-
gate changes in the dierent variables over time, we employed latent change score modelling
(McArdle 2009; LCM) and multiple regressions. LCM and all necessary preceding analyses were per-
formed with Mplus version 8 (Muthén and Muthén 2017). The following three steps were performed:
Dimensionality tests
To conrm the assumed factor structures and allow for a meaningful interpretation of the data, we
conducted conrmatory factor analyses on all variables (see Appendix 2, online supplemental data).
For almost all variables, the unidimensional model exhibited better model t. The only exception was
the variable Finding joy in conducting research, which exhibited inadequate model ts in both the
unidimensional and the three-dimensional solution. Hence, subsequent analyses were conducted
with two separate factors for this construct to ensure a meaningful interpretation of the data. The
rst factor describes joy in working with scientic literature, while the second describes joy in
working with empirical data.
Measurement invariance tests
A prerequisite for latent change score modelling is strong factorial invariance (McArdle 2009). Only if
strong factorial invariance is given can all factor loadings and intercepts be xed to the same values
for all measurement points. Following Meredith and Horn (2001), the CFI values of increasingly con-
strained models were compared (see Appendix 3, online supplemental data). For all variables, either
strong factorial invariance or partial measurement invariance was established, meaning that the sub-
sequent analyses can be meaningfully interpreted.
Latent change score modelling
We then employed LCMs to examine changes in our variables over time. In LCM, change is modelled
with latent dierence variables that express the change across two or more measurement points (see
Figure 1). This approach enables us to observe interindividual dierences in intraindividual change
free from measurement error (McArdle 2009).
LCM analyses were performed in two steps: In the rst step, we specied univariate LCMs (with
two measurement points, T1 and T2) for each variable. The latent change variable indicates intrain-
dividual changes from T1 to T2. Therefore, this variable was interpreted to test hypotheses H1a and
H1b (eectiveness). The variance of the latent change variable indicates interindividual dierences,
i.e. whether studentsresearch dispositions develop in dierent ways. When signicant
STUDIES IN HIGHER EDUCATION 7
interindividual dierences were found, in a second step, the latent change variable was regressed on
six dierent course characteristics. The regression coecients were then interpreted to test hypoth-
eses H2a, H2b and H2c (impact of course characteristics).
To account for the nested structure of the data (N= 952 students nested in 70 courses), we used
the course as a cluster variable with the Mplus command TYPE = COMPLEX. Additionally, auto-corre-
lated errors were included to account for method variance resulting from the use of the same items
over the two measurement points. Missing data were handled using full-information maximum like-
lihood estimation (FIML). The criteria suggested by Hu and Bentler (1999) were used as a reference
point for determining good model t: models with a CFI > 0.95 and a RMSEA < 0.06 were considered
to have adequate t.
Results
Univariate latent change score models: changes in individual cognitive and aective-
motivational research dispositions over time (hypotheses H1a and H1b)
Research knowledge
The LCM for research knowledge exhibited good model t (see Table 2). The mean of the change
variable was small but signicant (ΔM= 0.04, p< .01), indicating a signicant change from T1 to
T2. This means that after taking the RBL course, students were able to correctly answer 0.45 questions
Figure 1. Illustrative latent change model for two measurement points and three items.
Table 2. Model ts of all univariate latent change score models.
Model Χ
2
(df) pRMSEA CFI
1. Research knowledge 199.22 (141) .001 0.02 0.93
2. Value-related interest in research 132.61 (37) .001 0.05 0.95
3. Joy in working with scientic literature 62.18 (21) .001 0.05 0.98
4. Joy in working with empirical data 91.69 (20) .001 0.06 0.92
5. Uncertainty tolerance 39.29 (21) .01 0.03 0.98
6. Frustration tolerance 38.37 (20) .01 0.03 0.98
8I. WESSELS ET AL.
more on average (out of nine questions) than at T1. Thus, the data supported hypothesis H1a. The
variance of the change variable was very small and not signicant (σ
2
= 0.001, p= .8), indicating
that there were no interindividual dierences.
The univariate LCMs for all aective-motivational dispositions had very good model ts (see Table
2). The dispositions diered in their development from T1 to T2:
Value-related interest in research
The results revealed a signicant decrease from T1 to T2 (ΔM=0.14, p< .01). The signicant variance
of the change variable (σ
2
= 0.33, p< .01) indicates the presence of interindividual dierences in
changes in interest.
Joy with respect to research activities
As described above, this variable consisted of two distinct factors whose development was examined
individually. The results suggest a signicant decrease in joy in working with scientic literaturefrom
T1 to T2 (ΔM=0.17, p< .01). The signicant variance of the change variable (σ
2
= 0.36, p< .01) indi-
cates that there were dierences in studentstrajectories. No signicant change was observed for the
second factor, joy in working with empirical data(ΔM=0.05, p= .25). The signicant variance indi-
cates the presence of interindividual dierences in studentstrajectories (σ
2
= 0.15, p< .01).
Uncertainty tolerance
The results suggest a signicant increase from T1 to T2 (ΔM= 0.12, p< .01). The signicant variance
(σ
2
= 0.38, p< .01) indicates that there were substantial interindividual dierences in students
trajectories.
Frustration tolerance
The results show that frustration tolerance did not change signicantly from T1 to T2 (ΔM= 0.03, p
= .24). The signicant variance was indicative of interindividual dierences (σ
2
= 0.12, p< .01).
Therefore, the data supports hypothesis H1b only with respect to uncertainty tolerance. For value-
related interest in research and joy in working with scientic literature, signicant decreases were
found.
Inuence of other variables on changes in dierent research dispositions over time
(hypotheses H2a, H2b and H2c)
Next, predictors of the change variable were analysed for the research dispositions for which the uni-
variate LCMs showed evidence of interindividual dierences. This was the case for value-related inter-
est for research, joy in working with scientic literature and uncertainty tolerance.
Value-related interest for research
The multiple regression revealed two signicant and positive predictors of the latent change in value-
related interest in research: the perceived usefulness of the course for ones later career and the
instructors perceived interest in the studentswork. The overall variance explained by this regression
model was 10% (see Table 3).
Joy in working with scientic literature
The perceived usefulness of the course served as a signicant predictor of the latent change in joy
from T1 to T2. Students who perceived the course as useful for their later career experienced
greater increases in joy in working with scientic literature. The full regression model explained
5% of the variance in the change in joy (see Table 3).
STUDIES IN HIGHER EDUCATION 9
Uncertainty tolerance
Uncertainty tolerance was signicantly predicted by research self-ecacy at T1. Self-ecacy served
as a negative predictor: the higher a students self-ecacy, the more uncertainty tolerance decreased
or the less it increased. The overall variance explained by this regression model was 6% (see Table 3).
These ndings are in line with hypothesis H2c, which examined the inuence of additional motiv-
ating factors. Hypotheses H2a and H2b were not supported.
Discussion and implications
Our study examined the eectiveness of RBL in the social sciences. By applying prepost measure-
ments in 70 courses, we examined changes in dierent cognitive and aective-motivational research
dispositions through participation in RBL. Research knowledge increased signicantly, but no inter-
individual dierences were observed that could be further investigated. Research-related uncertainty
tolerance increased, whereas research interest and joy in working with scientic literature decreased
over the course of RBL participation. Subsequent regression analyses showed that the change in
uncertainty tolerance was signicantly predicted by research self-ecacy. The changes in interest
and joy were predicted by the perceived usefulness of the course for ones later career, while the
change in interest was also predicted by the instructors perceived interest in the studentswork.
Contrary to our expectations, the number of research steps performed and the autonomy students
were given during the RBL experience did not have an eect on changes to any of the aective-moti-
vational research dispositions.
Research knowledge
Overall, research knowledge increased signicantly over the course of RBL participation (see hypoth-
esis 1a). Previous studies with students from individual social scientic disciplines have reported com-
parable results (e.g. Taraban and Logue 2012). We were able to conrm these ndings using an
objective test instrument assessing three sub-areas of research knowledge: knowledge of
methods, knowledge of methodologies and research process knowledge in the social sciences.
However, the students in our sample did not exhibit substantial interindividual dierences in their
improvement and no further analyses could be conducted to explain dierences in the observed
change with reference to other variables. This lack of interindividual dierences might have been
Table 3. Multiple regression analysis for aective-motivational research dispositions.
Change of value-related
interest in research
Change of joy in working
with scientic literature
Change of uncertainty
tolerance
Predictor variables (and time point of
measurement) B(SE) βB(SE) βB(SE) β
Research self-ecacy (T1) 0.01 (0.01) 0.08 0.01
(0.00)
0.01 0.03 (0.01) ** 0.22 **
Number of research steps performed (T2) 0.02 (0.03) 0.04 <0.01
(0.03)
<0.01 0.02 (0.03) 0.04
Usefulness of the course for a later
profession (T2)
0.10 (0.03) ** 0.24 ** 0.10 (0.04)
**
0.22 ** 0.01 (0.04) 0.03
Student autonomy (T2 lecturer survey) 0.02 (0.02) 0.04 0.01 (0.04) 0.03 0.04 (0.03) 0.09
Lecturers interest in studentswork (T2) 0.09 (0.04) * 0.16 * <0.01
(0.05)
0.01 0.03 (0.04) 0.05
Perception of conducting realresearch
(T2)
0.04 (0.04) 0.10 0.00 (0.04) 0.01 0.01 (0.03) 0.02
AIC 31,375 28,945 29,746
R
2
(SE) 0.10 (0.04) 0.05 (0.03) 0.06 (0.04)
Note: B= unstandardised coecients, SE = standard error; β= standardised coecients.
*p< .05.
**p< .01.
10 I. WESSELS ET AL.
due to similar answering patterns on the knowledge items. We used a 9-item short version of a longer
test, which might not have been sucient to identify substantial dierences between students. In
future projects, we would recommend using the 27-item test form or another objective measurement
that yields more variance in studentsanswers.
Aective-motivational research dispositions
A signicant change from the rst to the second measurement point was found for three out of the
four aective-motivational research dispositions examined.
In line with our expectations (see hypothesis 1b), uncertainty tolerance increased over the course
of RBL participation. This change in uncertainty tolerance was signicantly predicted by research self-
ecacy (see hypothesis 2c). However, self-ecacy served as a negative predictor: the higher a stu-
dents self-ecacy, the smaller the positive change in uncertainty tolerance. Students with low
levels of research self-ecacy might exhibit stronger increases in uncertainty tolerance because
these students have less research experience and thus benet more strongly from participation in
RBL. A high level of uncertainty tolerance is important for coping with the unpredictable nature of
the research process. Some claim that uncertainty tolerance is vital not only for conducting research
but also for facing an increasingly complex world in general (Brew 2010). In this sense, uncertainty
tolerance not only assists students in pursuing scientic careers but also prepares students for
other professions. How studentsuncertainty tolerance can be changed is currently a subject of
debate in several elds. In the health sciences, it has been suggested that medical studentsuncer-
tainty tolerance can be enhanced by monitoring and controlling emotional processes related to
uncertainty (Iannello et al. 2017). Translating this recommendation to research in the social sciences,
we suggest integrating guided reections on experienced emotions related to uncertainty in the
research process. One way of doing so would be to use reective learning diaries (Nevalainen, Man-
tyranta, and Pitkala 2010). However, we did not test for reective processes related to uncertainty in
our sample. We can only assume that some instructors reected on and discussed research-related
uncertainties. Further research investigating the inuence of guided reection processes on the
development of uncertainty tolerance in RBL courses would be necessary to come to a more compre-
hensive conclusion.
Interest and joy in research exhibited high mean values during both the pre- and post-test, indi-
cating that the participating students are generally very fond of research and related activities.
However, unlike uncertainty tolerance, interest and joy decreased over the course of RBL partici-
pation (see hypothesis 1b). There are several possible explanations for this. Perhaps students gain
a more realistic idea of what research is during the course. At the beginning of their studies, students
conceptions of research might be inuenced by the predominant view of research in their society: in
Germany, the public perceives research as interesting and trustworthy (Wissenschaft im Dialog 2018).
Thus, realising how small the explanatory power of a single research project is might be frustrating or
disillusioning. Gaining a more realistic understanding of the nature and practice of research might
lead to decreased interest or joy in research, while simultaneously serves as an indication of what
others have termed becoming a scientist(Hunter, Laursen, and Seymour 2007).
The regression analyses showed that certain course variables served as signicant predictors (see
hypothesis 2c): changes in studentsinterest in research were signicantly predicted by the instruc-
tors perceived interest in the studentsresearch and the perceived usefulness of the course for their
later career (both rated by the students). Perceiving that the instructor is interested in their work
might be motivating for students and increase their own interest in research. As a practical impli-
cation, this does not mean that instructors should pretend to be interested in studentswork. It
could suce for instructors to choose topics for RBL courses that are of genuine interest to them
for example, their own research topics. Bringing ones own research topics into the classroom,
thereby combining ones teaching and research, has often been recommended as a useful practice
for instructors (Vicens and Bourne 2009). One of the main arguments for this is that it saves valuable
STUDIES IN HIGHER EDUCATION 11
time for instructors involved in both teaching and research. Our results additionally suggest that com-
bining teaching and research comes with benets for students, who feel more motivated by their
instructorsinterest in the topic.
Changes in joy were signicantly predicted by the perceived usefulness of the course for students
later careers: those students who perceived the course as useful for their future career gained more
joy in research. For students who do not aspire to academic careers, it might be benecial to empha-
sise or enhance the courses usefulness for outside academia, e.g. by choosing research topics that
are of interest in non-academic careers or applying service learning (Potter, Carey, and Plante
2003). In this way, more students might perceive conducting their own research projects as useful
for careers outside academia and therefore nd greater joy in doing research.
Contrary to our expectations, the number of research steps performed and the autonomy students
were given during the RBL experience did not have an eect on changes to any of the aective-moti-
vational research dispositions (see hypothesis 2a and 2b). This indicates that even working on pre-
dened research problems or completing only a limited amount of research steps has a positive
eect on students.
Overall, the regression models used to predict changes in dierent aective-motivational variables
accounted for 5% (joy in working with scientic literature), 6% (uncertainty tolerance) and 10% (research
interest) of the latent change variables variance. While these eect sizes can be classied as small
(Cohen 1988), it is important to put these values into perspective: given that answering the question-
naires on the predictor variables took students only 12 min, the cost-value ratio of these regression
analyses can be considered very positive. From a more fundamental perspective, it must be noted
that aective-motivational dispositions are complex, multidimensional phenomena that are inuenced
by a range of external variables, such as current mood or personal life events. The variables examined in
this study (e.g. student autonomy, instructorsinterest) are not sucient to accurately predict changes in
dierent aective-motivational research dispositions over an entire course. However, they did partially
serve as signicant predictors and thus provide practical new ideas for designing RBL courses.
Limitations and implications for future research
A problem with our and other studies in the eld is the lack of a control group (cf. Lopatto 2004).
Without an adequate comparison group, it remains unclear whether the research experience itself
is eective or whether it is the type of student who participates in RBL courses (Linn et al. 2015).
Some authors claim that students who seek out RBL courses have higher academic abilities and
are more motivated than other students in the rst place (Carter et al. 2016). In our sample, partici-
pation in the RBL course was often a mandatory part of the studentsstudy programme; thus, a strong
self-selection bias in our sample can be ruled out. Nevertheless, a meaningful, matched control group
is still necessary to draw nal conclusions on the eectiveness of RBL, e.g. by examining study pro-
grammes with a waiting list for RBL courses.
Another limitation concerns the testing time point. Since the post-measurement was conducted in
the classroom towards the end of the course, our results do not reect the eect of writing nal
papers or presenting research results. However, giving a public presentation on ones research has
been described as particularly motivating by students (Cuthbert, Arunachalam, and Licina 2012)
and thus might inuence the learning outcomes associated with RBL. Future research should incor-
porate the eects of nal assignments by using later or follow-up measurements.
Our studys quantitative set-up meant that the studentspersonal perspectives on their research
projects, individual reactions to challenges in the research process and additional thoughts on their
instructorsbehaviour could not be addressed. A future project could further explore and validate the
preliminary ndings of this study and the resulting implications by incorporating studentsperspec-
tives via in-depth interviews.
The aim of this study was to examine the eectiveness of RBL courses in the social sciences for
enhancing cognitive and aective-motivational research dispositions. Based on the results, we can
12 I. WESSELS ET AL.
conclude that RBL is an eective instructional format for enhancing research knowledge and
research-related uncertainty tolerance. RBL courses proved especially eective when students
thought the RBL experience was useful for their later career.
The question of whether RBL is an eective instructional format has so far been dominated by
studies from the eld of STEM, while evidence from the social sciences remains scarce. Our study
sought to provide a systematic account of the eectiveness of RBL among students from dierent
social scientic disciplines for enhancing discipline-specic measures using a prepost design.
While the chosen procedure was suitable for extending existing evidence in the eld, a range of
open questions remain that should be addressed in further research endeavours.
Note
1. What we call RBL has dierent names elsewhere, e.g. undergraduate research experiences(URE), summer under-
graduate research experiences(SURE) or course-based undergraduate research experiences(CURE). Most of
these terms describe the context (during the summer) or the type of students (undergraduates) rather than
the instructional set-up per se. We chose the term RBLto denote a specic instructional approach independent
of the exact duration or the participating students. We do, however, use evidence from studies examining CURE
or URE. We carefully checked that the studentsresearch experiences aligned with our notion of RBL.
Acknowledgements
We would like to thank Christoph Geiger, Frederic Lenz and Luise Behm for their valuable help in conducting this study.
We acknowledge support by the Open Access Publication Fund of Humboldt-Universität zu Berlin.
Disclosure statement
No potential conict of interest was reported by the author(s).
Funding
This research was supported by the German Federal Ministry of Education and Research [grant number 01PB14004B].
ORCID
Insa Wessels http://orcid.org/0000-0002-7799-8409
References
Bauer, Karen W., and Joan S. Bennett. 2003.Alumni Perceptions Used to Assess Undergraduate Research Experience.
The Journal of Higher Education 74 (2): 21030.
Blömeke, Sigrid, Jan-Eric Eric Gustafsson, and Richard J. Shavelson. 2015.Beyond Dichotomies: Competence Viewed as a
Continuum.Zeitschrift Für Psychologie 223 (1): 313.
Brew, Angela. 2010.Imperatives and Challenges in Integrating Teaching and Research.Higher Education Research &
Development 29 (2): 13950.
British Academy. 2012.Society Counts: Quantitative Skills in the Social Sciences. London: British Academy. https://www.
thebritishacademy.ac.uk/sites/default/les/BA%20Position%20Statement%20-%20Society%20Counts.pdf.
Burisch, Matthias. 1984.Approaches to Personality Inventory Construction.American Psychologist 39 (3): 21427.
Carter, D. F., H. K. Ro, B. Alcott, and L. R. Lattuca. 2016.Co-Curricular Connections: The Role of Undergraduate Research
Experiences in Promoting Engineering Skills.Research in Higher Education 57: 363393. doi:10.1007/s11162-015-
9386-7.
Cohen, Jacob. 1988.Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: L. Erlbaum Associates.
Cuthbert, D., D. Arunachalam, and D. Licina. 2012.It Feels More Important Than Other Classes I Have Done:An
AuthenticUndergraduate Research Experience in Sociology.Studies in Higher Education 37 (2): 129142. doi:10.
1080/03075079.2010.538473.
STUDIES IN HIGHER EDUCATION 13
Deicke, Wolfgang, Christopher Gess, and Julia Rueß. 2014.Increasing StudentsResearch Interest Through Research-
based Learning at Humboldt University.Council on Undergraduate Research Quarterly 35 (1): 2733.
Gess, Christopher, Christoph Geiger, and Matthias Ziegler. 2018.Social-Scientic Research Competency: Validation of
Test Score Interpretations for Evaluative Purposes in Higher Education.European Journal of Psychological
Assessment, 35 (5): 737750.
Gess, Christopher, Insa Wessels, and Sigrid Blömeke. 2017.Domain-specicity of Research Competencies in the Social
Sciences: Evidence from Dierential Item Functioning.Journal for Educational Research Online/Journal Für
Bildungsforschung Online 9 (2): 1136.
Gilmore, Joanna, Michelle Vieyra, Briana Timmerman, David Feldon, and Michelle Maher. 2015.The Relationship
Between Undergraduate Research Participation and Subsequent Research Performance of Early Career STEM
Graduate Students.The Journal of Higher Education 86 (6): 83463.
Healey, Mick, and Alan Jenkins. 2009.Developing Undergraduate Research and Inquiry. Innovations in Education and
Teaching International. York: The Higher Education Academy.
Hernandez, Paul R., Anna Woodcock, Mica Estrada, and P. Wesley Schultz. 2018.Undergraduate Research Experiences
Broaden Diversity in the Scientic Workforce.BioScience 68 (3): 204211. doi:10.1093/biosci/bix163.
Hu, Li-tze, and Peter M. Bentler. 1999.CutoCriteria for Fit Indexes in Covariance Structure Analysis: Conventional
Criteria Versus New Alternatives.Structural Equation Modeling: A Multidisciplinary Journal 6 (1): 155.
Huber, Ludwig. 2014.Forschungsbasiertes, Forschungsorientiertes, Forschendes Lernen: Alles Dasselbe?Das
Hochschulwesen 62 (1+2): 2229.
Hunter, Anne-Barrie, Sandra L. Laursen, and Elaine Seymour. 2007.Becoming a Scientist: The Role of Undergraduate
Research in StudentsCognitive, Personal, and Professional Development.Science Education 91: 3674.
Iannello, Paola, Anna Mottini, Simone Tirelli, Silvia Riva, and Alessandro Antonietti. 2017.Ambiguity and Uncertainty
Tolerance, Need for Cognition, and Their Association with Stress. A Study among Italian Practicing Physicians.
Medical Education Online 22 (1): 110.
John, Joanna, and John Creighton. 2011.Researcher Development: The Impact of Undergraduate Research Opportunity
Programmes on Students in the UK.Studies in Higher Education 36 (7): 78197.
John, Oliver P., and Richard W. Robins. 1994.Accuracy and Bias in Self-perception: Individual Dierences in Self-enhance-
ment and the Role of Narcissism.Journal of Personality and Social Psychology 66 (1): 20619.
Kilgo, Cindy A., Jessica K. Ezell Sheets, and Ernest T. Pascarella. 2015.The Link Between High-impact Practices and
Student Learning: Some Longitudinal Evidence.Higher Education 69 (4): 50925.
Kruger, Justin, and David Dunning. 1999.Unskilled and Unaware of It: How Diculties in Recognizing Ones Own
Incompetence Lead to Inated Self-assessments.Journal of Personality and Social Psychology 77 (6): 112134.
Levant, Ronald F., David H. Barlow, Karina W. David-, Kristofer J. Hagglund, Steven D. Hollon, Josephine D. Johnson, C.
Laura, et al. 2006.Evidence-based Practice in Psychology.American Psychologist 61 (4): 27185.
Linn, M. C., E. Palmer, A. Baranger, E. Gerard, and E. Stone. 2015.Undergraduate Research Experiences: Impacts and
Opportunities.Science 347 (6222): 1261757-11261757-6.
Lloyd, Steven A., Ryan A. Shanks, and David Lopatto. 2019.Perceived Student Benets of an Undergraduate
Physiological Psychology Laboratory Course.Teaching of Psychology 46 (3): 21522.
Lopatto, David. 2004.Survey of Undergraduate Research Experiences (SURE): First Findings.Cell Biology Education 3 (4):
27077.
Lopatto, David. 2007.Undergraduate Research Experiences Support Science Career Decisions and Active Learning.CBE
Life Sciences Education 6 (4): 297306.
Lopatto, D. 2009.Science in Solution: The Impact of Undergraduate Research on Student Learning. Tucson, AZ: Research
Corporation for Science Advancement. doi:10.1187/cbe.04-07-0045.
McArdle, John J. 2009.Latent Variable Modeling of Dierences and Changes with Longitudinal Data.Annual Review of
Psychology 60 (1): 577605.
Meredith, William, and John Horn. 2001.The Role of Factorial Invariance in Modeling Growth and Change.In Decade of
Behavior. New Methods for the Analysis of Change, edited by L. M. Collins and A. G. Sayer, 20340. Washington, DC:
American Psychological Association.
Muthén, Linda K., and Bengt O. Muthén. 2017.Mplus Users Guide. 8th ed. Los Angeles, CA: Muthén & Muthén.
Nevalainen, M. K., T. Mantyranta, and K. H. Pitkala. 2010.Facing Uncertainty as a Medical Student A Qualitative Study of
Their Reective Learning Diaries and Writings on Specic Themes During the First Clinical Year.Patient Education and
Counseling 78 (2): 21823.
Postare, Liisa, and Sari Lindblom-Ylänne. 2011.Emotions and Condence Within Teaching in Higher Education.Studies
in Higher Education 36 (7): 799813.
Potter, Sharyn J., Elizabeth M. Carey, and Elizabethe G. Plante. 2003.Integrating Service Learning Into the Research
Methods Course.Teaching Sociology 31 (1): 38.
Rand, J. 2016.Researching Undergraduate Social Science Research.Teaching in Higher Education 21 (7): 77389.
Rueß, Julia, Christopher Gess, and Wolfgang Deicke. 2016.Forschendes Lernen Und Forschungsbezogene Lehre:
Empirisch Gestützte Systematisierung Des Forschungsbezugs Hochschulischer Lehre.Zeitschrift Für
Hochschulentwicklung 11 (2): 2344.
14 I. WESSELS ET AL.
Russell, Susan H., Mary P. Hancock, and James McCullough. 2007.Benets of Undergraduate Research Experiences.
Science 316 (5824): 5489.
Schmidt-Atzert, Lothar, and Manfred Amelang. 2012.Psychologische Diagnostik. 5th ed. Berlin: Springer.
Seymour, Elaine, Anne-Barrie Hunter, Sandra L. Laursen, and Tracee DeAntoni. 2004.Establishing the Benets of
Research Experiences for Undergraduates in the Sciences: First Findings from a Three-year Study.Science
Education 88 (4): 493534.
Stanford, Jennifer S., Suzanne E. Rocheleau, Kevin P.W. Smith, and Jaya Mohan. 2017.Early Undergraduate Research
Experiences Lead to Similar Learning Gains for STEM and Non-STEM Undergraduates.Studies in Higher Education
42 (1): 11529.
Taraban, Roman, and Erin Logue. 2012.Academic Factors That Aect Undergraduate Research Experiences.Journal of
Educational Psychology 104 (2): 499514.
Vicens, Quentin, and Philip E Bourne. 2009.Ten Simple Rules to Combine Teaching and Research.PLoS Computational
Biology 5 (4): 45.
Ward, Christine, Joan S Bennett, and Karen W Bauer. 2003.Content Analysis of Undergraduate Research Student Evaluation.
http://www1.udel.edu/RAIRE/Content.pdf.
Wessels, Insa, Julia Rueß, Lars Jenßen, Christopher Gess, and Wolfgang Deicke. 2018.Beyond Cognition: ExpertsViews
on Aective-motivational Research Dispositions in the Social Sciences.Frontiers in Psychology 9: 110.
Whipple, Ellen E, Anne Hughes, and Susan Bowden. 2015.Evaluation of a BSW Research Experience: Improving Student
Research Competency.Journal of Teaching in Social Work 35 (4): 397409.
Wilson, Mark. 2005.Constructing Measures: An Item Response Modeling Approach. Mahwah, NJ: Lawrence Erlbaum.
Wissenschaft im Dialog. 2018.Wissenschaftsbarometer 2018. Berlin: WiD.
Wood, W. B. 2003.Inquiry-Based Undergraduate Teaching in the Life Sciences at Large Research Universities: A
Perspective on the Boyer Commission Report.Cell Biology Education 2 (2): 11216.
STUDIES IN HIGHER EDUCATION 15
... Dabei zeigen sich nur teilweise erwartungskonforme Ergebnisse. Erwartungsgemäß haben sich die selbsteingeschätzten forschungsmethodischen Kompetenzen der Studierenden signifikant (p < 0,05) erhöht -ein Ergebnis, das in die Richtung der von Fichten (2010b) und Ophuysen et al. (2017 berichteten Selbstaussagen von Studierenden weist, aber auch durch Studien, die objektive Testinstrumente nutzen (Wessels et al. 2020), belegt werden kann. Es ist dabei darauf hinzuweisen, dass der Zugewinn nicht mit der Höhe der selbsteingeschätzten Kompetenz am Anfang zusammenhängt. ...
... Dabei ist besonders interessant, dass eine besonders hohe Abnahme bereits vom MZP1 zu MZP2 erfolgte. Dieses Ergebnis deckt sich mit Befunden aus einer deutschen Studie, in der Studierende aus dem Bereich "social sciences" befragt wurden: Auch diese zeigen am Ende von "research courses" ein geringeres Interesse an Forschung (Wessels et al. 2020). Für zukünftige Studien wäre zu überlegen, den Faktor Forschungsinteresse differenzierter zu erheben, z. ...
... Zudem muss bedacht werden, dass Forschungskompetenz über die Selbsteinschätzung der Studierenden erhoben wurde. Es ist auf Grundlage der Forschungslage stark anzunehmen, dass diese Selbsteinschätzungen von objektiven Maßen, wie sie etwa Kompetenztests ermöglichen (etwa Gess et al. 2017;Wessels et al. 2020), abweichen würden. Kompetente Personen neigen dazu, ihre Kompetenzen zu unterschätzen, während inkompetente Personen ihre Kompetenzen eher überschätzen (Böttcher-Oschmann et al. 2019, S. 502). ...
Article
Full-text available
Zusammenfassung Forschendes Lernen hat sich als hochschuldidaktisches Konzept nicht nur in Deutschland als eine Möglichkeit etabliert, um der Forderung nach mehr Forschungsorientierung in der Lehrer*innenbildung nachzukommen. Die hohen Erwartungen werden jedoch nur teilweise durch Forschungsbefunde gestützt. Die vorliegende Studie fragt daher danach, ob sich durch Forschendes Lernen Forschungskompetenz und Forschungsinteresse fördern lassen. Dazu wurden Items bzw. Skalen entwickelt und in einer quantitativen Längsschnittstudie an einer deutschen Universität im Rahmen von neun Forschungswerkstätten für Lehramtsstudierende ( n = 144) getestet. In den Forschungswerkstätten wird ein Konzept von Forschendem Lernen umgesetzt, das das Durchlaufen eines vollständigen Forschungsprozesses von ersten Forschungsfragen über Datenerhebung, Datenauswertung bis zur Präsentation an den beteiligten Schulen beinhaltet. Die Ergebnisse zeigen, dass in der Selbsteinschätzung der befragten Studierenden deren Forschungskompetenz angestiegen, gleichzeitig jedoch das Interesse an Forschung gesunken ist. Ursachen und Konsequenzen werden diskutiert.
... In turn, this scientific knowledge must be based on the foundations of scientific language -knowledge about science -and this seems to be a tremendous handicap for university students (Kerlinger and Lee, 2002;Murtonen, 2015;Wilson et al., 2016). In this regard, Wessels et al. (2021) showed that in any case, the improvement of scientific knowledge and its language often affects negatively from an attitudinal and motivational point of view. ...
... In the case of Social Education students, the need to stimulate this role and to make scientific research a clear base competence of social knowledge and of the improvement actions that are designed from this professional field seems even more evident. In this study, we followed Wessels et al. (2021) to point out that it is also necessary to attend to the negative motivational effects that seem intrinsic to the acquisition, development, and recognition of research competence, in addition to increasing theoretical knowledge about what it means to do science, build theories, and improve practices from scientific knowledge as highlighted by Salmento et al. (2021). ...
Article
Full-text available
Called as one of the competences of the 21st century, research competence should help students navigate through the complexities of a continuously shifting world. The aim of this study is to analyse the acquisition and development of the research competence in a sample of 154 undergraduate students of two bachelor’s degrees of Education Sciences (Social Education and Pedagogy) at the Universitat Autònoma de Barcelona (Spain). We conducted a three-phase study, in which 1) the learning outcomes related to research competences declared in the syllabi were identified and mapped, through content analysis; 2) students’ perceptions about the development of these learning outcomes were gathered through a questionnaire; and 3) guidelines to foster research competences among these undergraduates were explored, through Delphi panel technique. The results show that communicative skills and state-of-art reviewing skills are the least present across the different courses of both degrees. In addition, the design of research competency acquisition across courses is uneven and does not seem clearly articulated. The students’ perception is consistent with the shortcomings -or disarticulation- observed in the curriculum analysis. They consider that the most poorly acquired competencies are the state-of-the-art reviewing, content knowledge and communicative skills. It seems that more emphasis is given to reflective thinking and communicative skills; but it appears necessary to strengthen the acquisition of scientific content, the search for trustworthy information. These results were discussed in two panels of experts and certain guidelines have been defined to improve the acquisition, development, and evaluation of the research competence through university training in Education Sciences.
... While these faculty-led endeavors show promise, research regarding student-led projects shows mixed results. One pre/posttest study with 952 German students measured the outcomes of undergraduates who participated in research-based learning in the social sciences, where they had the autonomy to conduct their own research project (Wessels et al. 2020). Results show that students increased their research knowledge and tolerance for uncertainty; however, their interests and joy of conducting research decreased over the experience. ...
... Results show that students increased their research knowledge and tolerance for uncertainty; however, their interests and joy of conducting research decreased over the experience. These decreases were associated with the students' perceptions of instructors' lack of interest in the student-led projects, suggesting that teachers choose research topics of interests to themselves (per Quinten and Bourne 2009) in order to improve student outcomes and make these experiences more beneficial (Wessels et al. 2020). Another study, where social science undergraduates conducted their own research as part of a community of practice, found that reflective talk about the research process helped mitigate students' feelings of inferiority and helped move them towards having identities as researchers (Rand 2016). ...
Article
Undergraduate research experiences (UREs) can improve student skills, retention, and matriculation to postgraduate study. Traditionally, UREs are available mostly in the natural and biological sciences, which have fewer minority and women majors and thus have disproportionately excluded these groups from such experiences. One effective solution is to diversify and increase course-based undergraduate research experiences (CUREs) within the social sciences. Models are few and research is limited on the impacts of parallel social science research experiences, but both are crucial to the goals of diversifying postgraduate success. Using a survey, we retrospectively assessed, qualitatively and quantitatively, past students’ (n = 87) career and personal learning impacts and outcomes in social science CUREs. These courses led to perceived improvements in understanding research processes and ethics, self-confidence, perseverance, and collaboration. Comparing to data from natural sciences, overall impacts were similar, including the intention to complete postgraduate education among first-generation students.
... Regardless of the term, it is likely that these research experiences vary in each disciplinary context. Yet they are similar in that students investigate some kind of empirical phenomenon with the primary objective being "to provide [them] with an opportunity to experience participation in research," as Wessels et al. (2020) put it (p. 1). ...
Article
Full-text available
In this teaching experience report, we describe a research experience for undergraduates (REUs) designed to cognitively support the work of two student research assistants (RAs) from a two-year college (2YC) on a funded project that involved analyzing user-generated content for an mHealth app. First, we suggest partnerships between two- and four-year institutions as a move toward REU equity because students from 2YCs are not typically afforded these opportunities. We then review the role of research in undergraduate learning and posit the importance of scaffolding to sequence cognitive leaps. Finally, we present the cognitive scaffolding we created and connect it to our hybrid card sorting-affinity diagramming content analysis method.
... RBL is currently seen as a panacea to overcome various demands in higher education, long-term social benefits because it can encourage scientific careers, shape someone to become a future scientist, help develop the mindset of researchers in solving problems, and develop cognition and motivation. (Wessels, et al., 2020). ...
Article
Full-text available
Many new learning models in the 21st century have emerged in improving students' academic skills, one of which is research-based learning (RBL). This pedagogic and constructivist model connects research and learning in improving students' critical thinking skills (CTS). The results of the study show that studies in empowering students' CTS using RBL are still limited. Therefore, exploration and deeper measurement of CTS with the RBL model were carried out through this study. The purpose of this study was to improve students' CTS using the RBL learning model. The results of the ANCOVA test showed that there was an effect of the RBL model in improving students' CTS. Descriptive data also shows that the average value of CTS is 72.70 using RBL, while students who take part in learning using conventional models show an average value of critical thinking skills of 58.30. Thus, RBL can be recommended in increasing the CTS of elementary school students in science learning.
... Against the backdrop of the developmental trajectories of subject-interest and the influencing factors that affect it, new research questions occur that need to be addressed in the future. The question of the correlation between the development of subject-interests and research-based learning arises (Wessels et al., 2020). Furthermore, the influence the COVID-19 pandemic has on this research field as well as on the entire teaching at universities should be investigated (Ortiz, 2020). ...
Article
Full-text available
It is a well-studied phenomenon, that throughout the course of studying at university, the motivation for the study program decreases. Correlation between motivation and learners’ behaviour, for example the learning process, achievement or, in the worst case, dropout exist. So there is a need for understanding the development of motivation in detail, like that of subject-interests, and for identifying influence factors, especially for higher education. This panel study examined the development of 4,345 students in higher education. Growth mixture models for subject-interests identify two classes of trajectories: “descending interest” and “continuously high interest”. In a next step, the analysis shows that gender, university entrance score, academic field and occupational aspiration influence membership of the classes. The results are discussed with respect to their consequences for education programs, but also with respect to possible new research questions.
... Given the importance of quality in higher education, linking it to research and innovation has opened the door to changing the unidirectional teaching paradigm and focusing education on the solution of real problems (Levy & Petrulis, 2012;Robertson & Blackler, 2006;Robertson & Bond, 2010;Turner, Wuetherick, & Healey, 2008;Wessels, Rueß, Gess, Deicke, & Ziegler, 2020;Zamorski, 2010). Thus, multiple approaches for such integration have emerged. ...
Article
The purpose of this research was to explore perceptions of the research-based learning (RBL) approach as an added value in tourism education. This research used a qualitative approach with an exploratory scope through three focus groups of eight tourism students who are part of the project: “Fostering a platform for research-based education to support sustainable development through Tourism in the Cajas Massif Biosphere Area (CMBA)”, in Southern Ecuador. The study shows that a research-based learning approach can be more successful than traditional approaches to link theory and practice, using real study cases and problems in the territory.
... Some recent literature has addressed RBL in the context of education programmes. Wessels, Rueß, Gess, Deicke and Ziegler (2020) state that most of the RBL studies have been focused on STEM, and much less on social studies, and examined the effectiveness of RBL in study programmes within the social disciplines, including education programmes. The results showed a significative increase of students' research knowledge over the course of RBL participation, as well as students' uncertainty tolerance, which is important for dealing with the unpredictability of the research processes and prepares students for their professions. ...
Article
Full-text available
Research-based learning is an educational approach that aims at enhancing an active student engagement through research activities that are typical of the field. In the case of student teachers, this implies getting involved in educational research and in the development of educational innovation based on research. A valid method for this aim is design-based research, which combines theory and practice through designing different types of educational products. In this work we present a technology-enhanced learning design of a pre-service teacher training course within the frame of research-based learning and, concretely, of design-based research. The course had the focus on the use of technology in the school, as well as on the development of the digital competence, and it was organised around the preparation of an educational proposal in which the use of technology was integrated and based on a research question. As results, we share the vision of the course instructor and the student teachers. Conclusions address the innovative character of the learning design and the educational practice presented and suggest future lines of work. Keywords: initial teacher training, research-based learning, higher education, educational technology, learning design, design-based research.
Article
Full-text available
Research competence (RC) as a key ability of students in the social sciences has thus far been conceptualized as consisting primarily of cognitive dispositions. However, owing to its highly complex and demanding nature, competence in conducting research might require additional affective and motivational dispositions. To address this deficiency in the literature, first, we conducted a qualitative interview study with academic experts (N = 16) in which we asked them to identify challenging research situations and the affective-motivational research dispositions needed to cope with them. We employed a subsequent online rating (N = 27) to evaluate the situations and dispositions that had been identified. The resulting affective-motivational facet of RC encompasses six challenging situations that are often encountered and nine dispositions that are necessary to successfully conduct research in the social sciences and may be used to both inform and evaluate research-based learning. The interview-based approach may serve as an exemplary procedure to postulate affective-motivational facets of competence models.
Article
Full-text available
New data highlight the importance of undergraduate research experiences (UREs) for keeping underrepresented science students on the pathway to a scientific career. We used a large-scale, 10-year, longitudinal, multi-institutional, propensity-score-matched research design to compare the academic performance and persistence in science of students who participated in URE(s) with those of similar students who had no research experience. Our results showed that students who completed 10 or more hours of cocurricular, faculty-mentored research per week across two or more academic semesters or summers were significantly more likely to graduate with a science-related bachelor's degree, to be accepted into a science-related graduate training program, and to be training for or working in the scientific workforce 6 years after graduation. Importantly, the findings show that just having a URE was not enough to influence persistence in science; it required a commitment of 10 or more hours per week over two or more semesters of faculty-mentored research.
Article
Full-text available
Medical practice is inherently ambiguous and uncertain. The physicians’ ability to tolerate ambiguity and uncertainty has been proved to have a great impact on clinical practice. The primary aim of the present study was to test the hypothesis that higher degree of physicians’ ambiguity and uncertainty intolerance and higher need for cognitive closure will predict higher work stress. Two hundred and twelve physicians (mean age = 42.94 years; SD = 10.72) from different medical specialties with different levels of expertise were administered a set of questionnaires measuring perceived levels of work-related stress, individual ability to tolerate ambiguity, stress deriving from uncertainty, and personal need for cognitive closure. A linear regression analysis was performed to examine which variables predict the perceived level of stress. The regression model was statistically significant [R² = .32; F(10,206) = 8.78, p ≤ .001], thus showing that, after controlling for gender and medical specialty, ambiguity and uncertainty tolerance, decisiveness (a dimension included in need for closure), and the years of practice were significant predictors of perceived work-related stress. Findings from the present study have some implications for medical education. Given the great impact that the individual ability to tolerate ambiguity and uncertainty has on the physicians’ level of perceived work-related stress, it would be worth paying particular attention to such a skill in medical education settings. It would be crucial to introduce or to empower educational tools and strategies that could increase medical students’ ability to tolerate ambiguity and uncertainty. Abbreviations: JSQ: Job stress questionnaire; NFCS: Need for cognitive closure scale; PRU: Physicians’ reactions to uncertainty; TFA: Tolerance for ambiguity
Article
Undergraduate research (UR) is a high-impact practice associated with many positive student outcomes. However, extending UR opportunities to all students is not practical for all psychology programs. Classroom-based UR experiences (CUREs) provide a scalable solution to this problem, but there is a paucity of empirically validated, authentic CUREs across all programs and all areas of the undergraduate psychology curriculum. A quasi-experimental design involving a semester-long CURE in an upper-level physiological psychology laboratory course was conducted. Students participating in the lecture and the CURE reported significant perceived benefits across many key domains, more positive attitudes toward science, and favorable changes in postgraduation plans, especially toward graduate school and careers in science, when compared to students in the lecture-only group. The results support the call for UR in the psychology curriculum, demonstrate the effectiveness of an authentic CURE to meet this goal for all students, and provide a model for CURE dissemination in a physiological psychology laboratory course.
Article
Although the development of research competency is an important goal of higher education in social sciences, instruments to measure this outcome often depend on the students' self-ratings. To provide empirical evidence for the utility of a newly developed instrument for the objective measurement of social-scientific research competency, two validation studies across two independent samples were conducted. Study 1 (n = 675) provided evidence for unidimensionality, expected differences in test scores between differently advanced groups of students as well as incremental validities over and above self-perceived research self-efficacy. In Study 2 (n = 82) it was demonstrated that the competency measured indeed is social-scientific and relations to facets of fluid and crystallized intelligence were analyzed. Overall, the results indicate that the test scores reflected a trainable, social-scientific, knowledge-related construct relevant to research performance. These are promising results for the application of the instrument in the evaluation of research education courses in higher education.
Article
Das Standardwerk zur Psychologischen Diagnostik erscheint nun in der 5. Auflage – und bleibt seinem Erfolgskonzept treu: Diagnostik ist theoretisch fundiert, erfolgt stets in Hinblick auf konkrete Fragestellungen und besitzt damit weitreichende Implikationen für die Intervention. Das Buch ist gleichzeitig prüfungsrelevant und praxisnah: Didaktisch sorgfältig aufbereitete Merksätze, Definitionen und Prüfungsfragen helfen bei der Vorbereitung auf die Prüfung. Praxiskapitel geben einen Überblick über die Diagnostik in den drei größten psychologischen Anwendungsfächern (Klinische Psychologie, A&O-Psychologie, Pädagogische Psychologie) sowie weiteren Anwendungsfeldern. Diagnostische Verfahren, insbesondere wichtige Testverfahren, aber auch diagnostische Interviews und Verhaltensbeobachtung, werden vorgestellt. Auch Weiterentwicklungen gibt es in der Neuauflage viele: Alle Kapitel wurden vollständig überarbeitet und aktualisiert, die sprachliche Lesbarkeit erhöht, die Abbildungen modernisiert. Auf der Gliederungsebene wurden Hinweise zur Relevanz im Bachelor- und Masterstudium eingefügt. Auf der neuen kostenlosen Lernwebsite auf www.lehrbuch-psychologie.de stehen zahlreiche Lernmaterialien für Studierende sowie Arbeitsmaterialien für Dozenten (u.a. Vorlesungsfolien zum Download) bereit. – Eine ausgezeichnete Arbeitsgrundlage für Studium und Prüfung sowie die Psychologische Diagnostik in der Praxis.
Article
To investigate the domain-specifi city of research competencies, higher education students from the social sciences were assessed with a standardized test in four disciplines: (a) sociology, (b) political science, (c) educational studies, and (d) psychology. The measure covered declarative and procedural knowledge of research methods, methodology, and procedures. Quantitative and qualitative research traditions were represented equally by test items. The domain-specificity of the measure was examined by detecting and explaining differential item functioning (DIF) between the disciplines. It was hypothesized that due to differences in opportunities to learn (OTL), students from different disciplines responded differently to subgroups of items. As expected based on the OTL-patterns, research traditions significantly explained variance in DIF. While psychology students were more likely to correctly answer items addressing quantitative methods than students with the same overall ability level but from different disciplines, students of all other disciplines were more likely to solve items addressing qualitative methods. These differences coincided with diff erences in OTL. Overall, the findings suggest that research competencies are similar across the social sciences, but differences between disciplines exist in their focus on quantitative or qualitative methods.
Article
The experience(s) of undergraduate research students in the social sciences is under-represented in the literature in comparison to the natural sciences or science, technology, engineering and maths (STEM). The strength of STEM undergraduate research learning environments is understood to be related to an apprenticeship-mode of learning supported by more experienced (post-graduate) peers, often through ongoing research projects. Studies of undergraduate research reveal that this is not typical in the social sciences, and students report facing specific challenges to the development of their identities as researchers that include fear, intellectual confusion and emotional unsettlement. This paper examines how a social science learning environment, designed as a research study itself, fostered beginning researchers communities of practice, realised a distinct mode of apprenticing based on peers’ similarly peripheral community membership, and enabled students to reframe emotional unsettlement. It argues that, effectively mediated, talk can powerfully improve undergraduate social science research students’ experiences.