ArticlePDF Available

Once the best student always the best student? Predicting graduate study success using undergraduate academic indicators: Evidence from research masters’ programs in the Netherlands

Authors:

Abstract

In the face of increasing and diversifying graduate application numbers, evidence‐based selective admissions have become a pressing issue. By conducting multilevel regression analyses on institutional admissions data from a Dutch university, this study aims to determine the predictive value of undergraduate academic indicators for graduate study success on research masters’ programs in the life sciences. The results imply that in addition to undergraduate grade point average, undergraduate thesis grade is a valid predictor of graduate grade point average. To a small extent, the examined undergraduate academic indicators also predict graduate degree completion and time to degree. The results from this study can be used by admissions committees for evaluating and improving their current practices of graduate selective admissions. There is substantial scientific evidence that undergraduate grade point average (UGPA) is a valid predictor of certain dimensions of graduate study success. This paper adds to this evidence by showing that undergraduate thesis grade is also a valid predictor of graduate grade point average (GGPA). The predictive power of the type of prior higher education institution for the examined dimensions of graduate study success is small at best. Undergraduate academic indicators are better predictors of GGPA than of graduate degree completion or time to degree. The results of this study can be used for improving admissions decision‐making at graduate schools, especially ones with a research‐oriented curriculum. There is substantial scientific evidence that undergraduate grade point average (UGPA) is a valid predictor of certain dimensions of graduate study success. This paper adds to this evidence by showing that undergraduate thesis grade is also a valid predictor of graduate grade point average (GGPA). The predictive power of the type of prior higher education institution for the examined dimensions of graduate study success is small at best. Undergraduate academic indicators are better predictors of GGPA than of graduate degree completion or time to degree. The results of this study can be used for improving admissions decision‐making at graduate schools, especially ones with a research‐oriented curriculum.
Received: 20 May 2021
|
Revised: 7 July 2022
|
Accepted: 8 July 2022
DOI: 10.1111/ijsa.12397
RESEARCH ARTICLE
Once the best student always the best student? Predicting
graduate study success using undergraduate academic
indicators: Evidence from research mastersprograms
in the Netherlands
Anastasia Kurysheva |Nivard Koning |Christine M. Fox |
Harold V. M. van Rijen |Gönül Dilaver
Department of Biomedical Sciences, Center of
Education and Training, University Medical
Center Utrecht, Utrecht, Netherlands
Correspondence
Anastasia Kurysheva, Department of
Biomedical Sciences, Center of Education and
Training, University Medical Center Utrecht,
P.O. Box 85500, HB4.05, 3508 GA Utrecht,
Netherlands.
Email: a.kurysheva@umcutrecht.nl
Abstract
In the face of increasing and diversifying graduate application numbers, evidence
based selective admissions have become a pressing issue. By conducting multilevel
regression analyses on institutional admissions data from a Dutch university, this
study aims to determine the predictive value of undergraduate academic indicators
for graduate study success on research mastersprograms in the life sciences. The
results imply that in addition to undergraduate grade point average, undergraduate
thesis grade is a valid predictor of graduate grade point average. To a small extent,
the examined undergraduate academic indicators also predict graduate degree
completion and time to degree. The results from this study can be used by
admissions committees for evaluating and improving their current practices of
graduate selective admissions.
KEYWORDS
admissions, GPA, graduate education, master's degree, student selection, study success, thesis
Practitioner points
There is substantial scientific evidence that undergraduate grade point aver-
age (UGPA) is a valid predictor of certain dimensions of graduate study success.
This paper adds to this evidence by showing that undergraduate thesis grade is
also a valid predictor of graduate grade point average (GGPA).
The predictive power of the type of prior higher education institution for the
examined dimensions of graduate study success is small at best.
Undergraduate academic indicators are better predictors of GGPA than of
graduate degree completion or time to degree.
The results of this study can be used for improving admissions decisionmaking at
graduate schools, especially ones with a researchoriented curriculum.
Int J Sel Assess. 2022;117. wileyonlinelibrary.com/journal/ijsa
|
1
This is an open access article under the terms of the Creative Commons AttributionNonCommercialNoDerivs License, which permits use and distribution in any
medium, provided the original work is properly cited, the use is noncommercial and no modifications or adaptations are made.
© 2022 The Authors. International Journal of Selection and Assessment published by John Wiley & Sons Ltd.
1|INTRODUCTION
The goal of university admissions committees is to create a selective
admissions process that meets societal expectations of objectiveness,
fairness, and transparency. Over the last two decades, countries with
widespread instruction in English have seen a steady increase in
demand for graduate education (Association of Universities in the
Netherlands, 2021; Statista, 2020; the Higher Education Statistics
Agency, 2020). This demand has challenged admissions committees
for several reasons. First, many universities now face a disparity in
the number of graduate school places versus the number of
applicants. This has created a situation where some students, whilst
eligible, are rejected. Admissions committees, therefore, must be able
to justify their selection decisions. Second, because of the growing
number of internationally mobile students (Organisation for Eco-
nomic Cooperation and Development [OECD], 2022), application
files have become more diverse. Admissions committees are now
faced with the challenge of comparing foreign applications (from
different education systems with different evaluation processes)
against applications from national students. Third, despite efforts to
increase access to higher education for underrepresented groups
such as firstgeneration students, students with disabilities, and
students with migration backgrounds (Kurysheva et al., 2019;
Torgerson et al., 2014; Younger et al., 2018), these groups still have
less chance of accessing Higher Education Institutions (HEIs; Salmi &
Bassett, 2014). For these reasons, university admissions committees
need valid selection methods.
HEIs implement an array of selection methods for making
admissions decisions. Some HEIs use information on applicantsprior
individual characteristics (e.g., prior study success) as well as
institutional factors (i.e., factors related to studentsprior HEI). We
refer to both of them as undergraduate academic indicators. In this
study, we examine to what extent the undergraduate academic
indicators predict graduate study success.
1.1 |Graduate study success
Kyllonen et al. (2005) distinguish three subgroups of higher education
outcomes: (1) study success, or convenience measures (such as grade
point average [GPA], time to degree, attrition), (2) performance
factors (such as discipline, teamwork, leadership, management), and
(3) affective measures (such as attitudes, interest, liking). In this study,
we examine determinants of the first subgroup of higher education
outcomesstudy success. The other two subgroups are almost never
formally assessed in a consistent manner across students and
programs, while study success is easily obtainable as HEIs keep
records on various dimensions of studentsstudy success (Kyllonen
et al., 2005). Following the other studies on prediction of study
success (e.g., Schwager et al., 2015), we operationalize graduate
study success through three dimensions: (1) graduate degree comple-
tion, (2) graduate GPA (GGPA), and (3) graduate time to degree (i.e., time
taken to complete a master's degree).
The different theoretical models propose that both individual
characteristics, as well as institutional factors, determine study
success (Bean, 1980; Cabrera et al., 1993; Tinto, 1975). Below, we
justify our hypotheses on relationships between undergraduate
academic indicators and graduate study success by providing
theorybased arguments supported by relevant research findings.
1.2 |Undergraduate academic indicators
1.2.1 |Undergraduate GPA (UGPA)
Theory suggests that prior study success plays a pivotal role in
determining future study success (Galla et al., 2019; Schneider &
Preckel, 2017). Prior grades (e.g., UGPA) have especially been
shown as good determinants of subsequent study success (its
various dimensions). The proposed mechanism of this relationship
is that grades represent a composite measure of studentsIQ,
knowledge, skills, noncognitive constructs (e.g., selfregulatory
competencies), and personality traits (including conscientiousness,
perseverance, and diligence). Therefore, prior grades as composite
measures may be better predictors of future study success than
narrow measurements (such as cognitive ability tests; Borghans
et al., 2016;Tai,2020).
While certain noncognitive constructs and personality traits
(foremost, conscientiousness) are known as good predictors of study
success (Busato et al., 2000; Poropat, 2009), their usage in selective
admissions is not feasible due to various forms of applicant faking
(ranging from impression management to conscious distortions of
answers) which occurs once applicants are asked to selfreport in a
highstake situation (Niessen et al., 2017). In this regard, composite
measures such as undergraduate GPA have a substantial advantage
because they are significantly influenced by noncognitive constructs
and personality (Borghans et al., 2016), while at the same time they
do not have disadvantages of selfreports.
Existing evidence supports this proposed relationship between
UGPA and graduate study success. Higher UGPA is related to
higher graduate grade point average (GGPA; Burton et al., 2005;
Fu, 2012; Howell et al., 2014;MonetaKoehler et al., 2017;
Zimmermann et al., 2015). When it comes to other dimensions of
graduate study success, the evidence is mixed. For example, some
studies also found a positive relationship between UGPA and
graduate degree completion (e.g., MendozaSanchez et al., 2022;
MonetaKoehler et al., 2017;Schwageretal.,2015;Wollast
et al., 2018),whileothersdidnot(Coxetal.,2009;Dore,2017).
Thesameappliesforgraduatetimetodegree:Somestudiesfound
a negative relationship between UGPA and graduate time to degree
(Howell et al., 2014;MendozaSanchez et al., 2022) and others did
not (Dabney, 2012;MonetaKoehler et al., 2017). Based on
theoretical underpinnings and the findings of prior literature, we
expect that UGPA should positively relate to GGPA, but with less
certainty, positively to graduate degree completion and negatively
to graduate time to degree (Hypothesis 1).
2
|
KURYSHEVA ET AL.
1.2.2 |Undergraduate research experience
According to several theoretical frameworks, another individual
characteristic that may determine graduate study success is prior
research experience, which students gain during their undergraduate
studies. Among these theories are theories on determinants of skill
acquisition (Ericsson & Charness, 1994; Gilmore et al., 2015), Camp-
bell's model of job performance (Campbell et al., 1993; Miller
et al., 2021), and the cognitive apprenticeship model (Brown
et al., 1989; Collins et al., 1989). The theories on determinants of
skill acquisition propose that a skill develops with practice over time
and, therefore, the achieved level of skills depends on having training
of those skills (Gilmore et al., 2015). The model of job performance
distinguishes declarative knowledge, procedural knowledge, and
motivation as factors influencing study success (Campbell et al., 1993).
In this regard, prior research experience may be related to all three
factors (Miller et al., 2021): Research experience leads to gains in
declarative knowledge (i.e., knowledge about a discipline), procedural
knowledge (i.e., knowledge about relevant procedures in this
discipline), and motivation (i.e., students who have already engaged
in researchrelated tasks during their bachelor's program and are
willing to proceed with researchrelated tasks by applying to a
graduate research master's program presumably to do so because
they appreciated the prior experience and are motivated to carry on
with conducting researchrelated tasks). Finally, the cognitive
apprenticeship model (Brown et al., 1989; Collins et al., 1989)
suggests that apprentices (i.e., undergraduate students conducting
research) advance their knowledge and skills by interacting with
established researchers in their fields. This interaction creates
conditions in which supervisors model disciplinaryappropriate
thinking(Gilmore et al., 2015, p. 837), which in turn helps students
enhance their performance on various dimensions of graduate study
success in this discipline.
Considering these theoretical underpinnings of the critical
importance of undergraduate research for graduate study success
on research programs, the findings of literature may appear
surprising. The metaanalytical evidence shows that undergraduate
research experience is largely unrelated to graduate academic
performance but with less statistical certainty to degree attainment
and publication performance (Miller et al., 2021). The findings in the
metaanalysis of Miller et al. (2021) could be explained by the fact
that more than half of the included studies used the generic
dichotomous operationalization of undergraduate research experi-
ence (present or absent). When undergraduate research experience
is operationalized differently (e.g., as duration of research experi-
ence in monthsor whether or not the student wrote a thesis during
their bachelor's program), some studies find the relationship
between research experience and certain dimensions of graduate
study success (see Cox et al., 2009; Gilmore et al., 2015;
Weiner, 2014), while others do not (Hall et al., 2017).
The undergraduate level is usually the first educational level with
research training (at least in the form of a bachelor's thesis, which
is often an obligatory component of undergraduate university
curriculum). This undergraduate prior research experience such as
thesis is typically assessed in a standardized form (i.e., the duration of
work on a bachelor's thesis is regulated by the number of ECTS
assigned; the research quality is assessed by grades). A grade for
undergraduate prior research experience (labeled thesis grade)
represents a convenient variable for research models and a
convenient admissions requirement in practice, as the standardized
form provides more comparability across students. Thesis grade is
also relatively objective: It is the quantitative assessment by experts
in the field and this assessment often follows a certain rubric or at
least requires certain extent of justification. Equally important, out of
all undergraduate study activities, undergraduate thesis is the most
pertinent and the most recent indicator in relation to research
oriented graduate education.
In spite of these considerations, there have been no studies
conducted on the relationship between undergraduate thesis grade
and GGPA. We aim to fill in this gap and to explore the predictive
potential of thesis grade. Based on theoretical arguments of prior
research experience importance in determining graduate study
success and research findings of usefulness of other operationaliza-
tions of research experience rather than dichotomous present or
absent,we hypothesize that undergraduate thesis grade should
positively relate to degree completion and GGPA on researchintense
mastersprograms and negatively to graduate time to degree
(Hypothesis 2).
1.2.3 |Institutional factors
In addition to individual determinants of study success, the
theoretical models also indicate that (prior) institutional factors
represent determinants of (future) study success (Bean, 1980;
Cabrera et al., 1993; Tinto, 1975). The characteristics of prior HEI
may be associated with studentspreparedness for a certain graduate
program. For example, it is plausible to assume that the structure of
curriculum and focus on certain learning objectives at a prior HEI is
related to knowledge and skills of its graduates. If a student followed
a researchoriented curriculum during their undergraduate program,
they are more prepared and will perform better at a research
intensive graduate program than a student who followed a practice
oriented curriculum during their undergraduate studies.
Among the wide range of institutional factors, we choose to
focus on the type of HEI. The type of HEI captures basic curriculum
characteristics and learning goals of different HEIs (i.e., practice
oriented vs. researchoriented curriculums). In addition, using this
variable as a proxy for institutional factors provides good statistical
power to our analysis and generalizability to our findings, because
students may be grouped in categories based on their type of
prior HEI.
Type of prior HEI has not received much research attention in
the literature. We found one study on a German sample of business
administration and economics graduate students, which showed that
the type of HEI (categorized as university, college [Fachhochschule],
KURYSHEVA ET AL.
|
3
academy [Berufakademie], and school abroad) had a weak effect on
graduate study success (Chadi & de Pinto, 2018). We aim to address
this gap by including type of prior HEI as one of the predictors in our
research model. We expect that students from practiceoriented
undergraduate programs score lower on the three dimensions of
graduate study success compared to students from researchoriented
undergraduate programs. However, we also expect that after UGPA
has been taken into account, the relationship between type of HEI
and graduate study success would be weak in terms of substantial
significance, even though statistically significant (Hypothesis 3).
1.3 |Theoretical contributions
With this study, we hope to make the following contributions to the
field of graduate selective admissions. First, we aim to test the
predictive validity of UGPA not only for GPA, but also for degree
attainment and time to graduate degree: Testing these relationships
will clarify the mixed findings of prior literature regarding the
predictive value of UGPA for these last two dimensions. Second, the
aspects of undergraduate research work previously examined such as
undergraduate research experience present versus undergraduate
research experience absent(Cox et al., 2009; Miller et al., 2021)or
duration of research experience(Gilmore et al., 2015; Hall
et al., 2017; Weiner, 2014) do not reflect the quality of students
undergraduate research work. Our study, to the best of our
knowledge, is the first one that examines whether assessments
regarding quality of undergraduate studentsindependent research
work (i.e., a grade for undergraduate thesis) predict graduate study
success. Finally, we explore whether the data supports the expecta-
tion that students from practiceoriented HEIs perform less well than
students from HEIs with intensive researchoriented training in
research mastersprograms.
2|METHOD
Cognizant of the fact that there are multiple considerations when
it comes to determining whether an undergraduate academic
indicator is suitable for use in student selection (see Patterson
et al., 2016,2018; Posselt, 2016), this study focuses on predictive
validity. It addresses predictive validity of undergraduate academic
indicators for graduate study success on research masters
programs in the field of life sciences. We choose to focus on the
research mastersprograms in the life sciences because of the
intensive study loads in research laboratories which require
extensive and often longlasting immersion in research practice.
The goal of this study is to help provide guidance for graduate
school admissions committees regarding which undergraduate
academic indicators should be considered in student selection.
We examine the direct relationships between three undergraduate
academic indicators
1
and three operationalizations of graduate
study success. To better understand the generalizability of this
study and to set it within the context of other graduate programs, a
national and institutional context is provided below.
2.1 |National context
This study has been conducted in a large research university in the
Netherlands. The Dutch higher education system is comprised of
14 public research universities that grant academic degrees up to the
PhD level (including some university colleges which offer selective
international liberal arts and sciences bachelorsprograms), 37
universities of applied sciences (which grant professional degrees
up to master's level), and a few small specialized private institutions
(van der Wende, 2020). At research universities, researchintensive
education aims to advance understanding of the phenomena studied
within academic disciplines, to facilitate application of scientific
knowledge, and to generate new knowledge. Universities of applied
sciences offer higher professional educationtheoretical and practical
training related to professions that necessitate a higher vocational
qualification (Eurydice Network, 2020).
In this article, we focus on mastersprograms at research
universities. For comparison, the Netherlands has adapted the
Framework for Qualifications of the European Higher Education
Area (QFEHEA) which consists of three cycles (Bachelor's/Master's/
PhD). It was introduced with the Bologna Process in 2002 (Lub
et al., 2003; Witte et al., 2008) and covers levels 68 in the European
Qualifications Framework. This means that the master's phase in the
Netherlands is comparable to a master's phase in 48 countries within
the EHEA (European Higher Education Area [EHEA], n.d.). This three
cycles framework is also compatible to both the United States and
Canada with only subtle differences with the United Kingdom which
also offers an MPhil option that sits between a master's and PhD.
It is possible to enter a Dutch master's program in a research
university with an undergraduate degree either from a Dutch
research university, university college, university of applied sciences,
or the equivalent from a foreign HEI. Dutch research universities
offer not only taught but also research mastersprograms. Research
mastersprograms differentiate themselves from taught masters
with an emphasis on research, duration (2 years and 120 EC instead
of 1 year and 60 EC), and selective admissions of students
(Snijder, 2016). They aim to prepare students for researchrelated
positions both inside and outside academia (NVAO, 2016). The
curriculum of these programs is specifically focused on obtaining and
practicing research competencies and skills. For example, internships
at research laboratories typically constitute components of research
mastersprograms in the life sciences.
2.2 |Institutional context
We used data from an interdisciplinary graduate school of a major
Dutch research university with 13 RM programs in the life sciences.
At this graduate school, the demand for study placement increases
4
|
KURYSHEVA ET AL.
annually. The major research project of 9 months represents the main
component of the graduate curriculum. The remaining part of the
curriculum consists of a minor research project, different mandatory
and optional courses, and a writing assignment. The weighted grade
for these components constitutes GGPA. The research projects are
usually conducted in the university's laboratories. Students are
exposed to a variety of research processes and are expected to
conduct their own research that involves multiple stages, starting
from research design and data collection to writing a research report.
2.3 |Participants
No recruitment was needed because we used the institutional data
(i.e., data from the university administrative system). This data usage
was approved by the Netherlands Association for Medical Education
Ethical Review Board (dossier number: 2019.8.2). The data came
from six cohorts of 1792 mastersstudents. Out of these students,
1570 (88%) completed their mastersstudies and 222 (12%) dropped
out at some point during their mastersprograms.
Out of the sample of 1792 students, which is labeled Sample 1,
three additional analytical subsamples were derived (Samples 2, 3,
and 4). Samples 1 and 2 were used to predict the binary variable
graduate degree completion. Sample 1 consisted of students who
came from four different types of undergraduate HEIs (N
complete-
d_and_droppedout_from_different_HEI
= 1792). Sample 2 consisted of stu-
dents who studied their mastersat the same university as their
bachelors; therefore, their undergraduate thesis grade was available
2
(N
completed_and_droppedout_&the_same_HEI
= 1249). Samples 3 and 4 were
used to predict two metric variables (GGPA and graduate time to
degree). These study success dimensions were only available for
students who completed their studies. Sample 3 consisted of
students who came from four different types of undergraduate HEIs
(N
completed_from_different_HEI
= 1570). Sample 4 consisted of students
who studied their mastersat the same university as their bachelors;
therefore, their undergraduate thesis grade was available (N
complete-
d_and_droppedout_&the_same_HEI
= 1112). Information on sample sizes and
characteristics is presented in Table 1and the intercorrelations of
continuous study variables are presented in Table 2.
2.4 |Measures
2.4.1 |Independent variables
Percentile ranks of UGPA
UGPA refers to an average grade for all curriculum components of an
undergraduate program, weighted according to the number of credits
for each component. The UGPA of each student was transferred to
the percentile ranks due to different grading systems that are applied
at different Dutch and international education systems. Percentile
ranks allowed us to place all student grades from different grading
systems on one scale. The adequacy of usage of percentile ranks was
doublechecked via a stability check of results, using UGPA on a US
scale (from 0 till 4) instead of percentile ranks.
The percentile ranks placed each student in a relative position to
others from their own country. We used the data only from the
largest groups (n20), so that percentile ranks could be derived.
Among the Dutch students, the percentile ranks were given within
three groups: students from Dutch university colleges (UGPAs on a
scale from 1 to 4), Dutch research universities (UGPAs on a scale
from 1 to 10), and universities of applied sciences (UGPAs on a scale
from 1 to 10). The largest international student groups, who were
greater than or equal to 20 in size, came from the European Union
(EU). Namely, the international student groups included British
(UGPAs on a scale from 0 to 100), Greek (UGPAs on a scale from 1
to 10), Italian (UGPA on a scale from 0 to 30), and Spanish (UGPAs on
a scale from 1 to 10). Other EU student groups and student groups
outside of the EU were left out of the analysis due to insufficient
numbers per group.
Undergraduate thesis grade
An undergraduate thesis is a common part of undergraduate
curriculum. This variable (on the Dutch grading scale from 1 to 10)
represents a grade for an undergraduate thesis or research project.
Type of prior HEI
This variable is nominal and represents types of HEIs where students
completed their undergraduate programs. In our data, four types of
HEIs were distinguished: Dutch research universities, Dutch univer-
sity colleges, Dutch university of applied sciences, and international
HEIs (see Table 1for frequencies of each). The international HEIs
were considered as one category. This is because in this specific
sample only applications with the type of prior HEIcomparable to a
Dutch research universityare usually processed further by the
admissions committees. International students with an under-
graduate degree from a HEI that is on the level of the Dutch
universities of applied sciences are rarely ever admitted. Likewise, it
is not common to admit students with an undergraduate degree from
international colleges with liberal arts and sciences degrees (which
would be an analogue to the Dutch university colleges). Therefore, in
terms of the level of their type of prior HEI, the group of international
students can be considered comparable to the group of students
from Dutch research universities. It was then decided to keep
students from international HEIs as one group, which is in line with
other studies in the field (e.g., Chadi & de Pinto, 2018).
2.4.2 |Dependent variables
Graduate degree completion
Graduate degree completion is a binary variable wherein the category
master's degree attained(coded as 1) was defined as obtaining a
master's degree within four years after the start of the master's
program and a category master's degree was not attainedwas
defined as an actual stoppage with the master's program (coded as 0).
KURYSHEVA ET AL.
|
5
Graduate GGPA
GGPA (on the Dutch grading scale from 1 to 10) represents an
average grade for all curriculum components of a research master's
program weighted according to their credit value.
Graduate time to degree
Graduate time to degree is measured as actual duration in months of
the mastersstudies for each student. The expected duration on the
research mastersprograms at this graduate school is 24 months.
However, students are allowed to graduate earlier or later, and it is
common in this graduate school to graduate a few months later than
the nominal duration of 24 months. Graduate time to degree in our
student sample ranged from 19 to 84 months with a median of 28
months.
2.4.3 |Research model and data analysis approach
Figure 1presents our research model. It shows the examined
relationships as well as intercorrelations between the predictors.
TABLE 1 Samples' demographical and educational characteristics
Characteristics
Sample 1. Graduates
and dropouts from
different prior HEIs
(N= 1792)
Sample 2. Graduates and
dropouts who studied their
masters' at the same university
as their bachelors' (N= 1249)
Sample 3.
Graduates from
different prior HEIs
(N= 1570)
Sample 4. Graduates who
studied their masters' at the
same university as their
bachelors' (N= 1112)
Gender: males (n) 741 521 627 456
Age range in years 1749 1749 1838 1938
M
age
22.5 22.3 22.4 22.2
SD
age
2.1 2.0 1.9 1.8
Mdn
age
22.0 22.0 22.0 22.0
Citizenship (%)
The Netherlands 92 98 91 98
Other EU 8 1 8 1
Outside of EU <0.1 <1 <1 <1
Type of prior HEI (%)
Dutch research university 84 100 84 100
Dutch university college 4 4
Dutch university of
applied sciences
77
International HEI 5 5
Prior field of study (%)
Biology 34 41 34 41
Biotechnology 2 2
Biology and medical
laboratory
55
Biomedical sciences 31 36 32 36
Chemistry 5 6 5 6
Liberal arts and sciences 6 1 5
Medicine 1 1 1 1
Pharmaceutics 5 6 5 6
Psychology 5 5 5 6
Other 6 4 6 4
Missingness (values; %) 1.7 4.0 1.5 3.6
Abbreviations: EU, European Union; HEI, Higher Education Institutions.
6
|
KURYSHEVA ET AL.
Though the variables of interest are on an individual level, the data
have the multilevel structure (students nested in 68 study groups
which in turn are nested in 13 programs). To account for the
dependency of students within groups and programs, the hierarchical
linear modelling was applied. We ran the analyses on three
dimensions of graduate success separately and not on one
multivariate outcome because such a multivariate outcome would
make the interpretation of findings barely explainable and, therefore,
useless for admission practitioners. Analysis was conducted in HLM
8. Since the percentage of missingness was low (in all four samples
less than 5% of data was missing), we handled the missingness using
the ExpectationMaximization (EM) method.
3|RESULTS
Table 2shows the Pearson correlations between the study variables.
Both percentile rank of UGPA and undergraduate thesis grade are
significantly related to the three dimensions of graduate study
success: positively to degree completion and GGPA and negatively to
time to degree.
Below, we describe the results for the incremental validity of
type of prior HEI and undergraduate thesis grade above and beyond
UGPA for each of the graduate study success dimension.
Tables 3.1,4.1, and 5.1 are based on analyses of Samples 2 and 4
which included students who did their mastersat the same university
TABLE 2 Intercorrelations between study variables
Variable nMSD1234
Sample 1
1. Percentile rank of UGPA 1689 49.82 28.71 1
2. Degree completion 1792 0.88 0.33 0.053*1
Sample 2
1. Percentile rank of UGPA 1186 49.54 28.83 1
2. Undergraduate thesis grade 1011 7.76 0.72 0.52*** 1
3. Degree completion 1249 0.89 0.31 0.06 0.09** 1
Sample 3
1. Percentile rank of UGPA 1186 49.54 28.83 1
2. GGPA 1206 7.83 0.58 0.53*** 1
3. Graduate time to degree 1090 30.72 7.93 0.13*** 0.27*** 1
Sample 4
1. Percentile rank of UGPA 1059 50.09 28.84 1
2. Undergraduate thesis grade 902 7.78 0.71 0.52*** 1
3. GGPA 1099 7.87 0.55 0.58*** 0.47*** 1
4. Graduate time to degree 1099 30.72 7.93 0.13*** 0.15*** 0.27*** 1
Note: Type of prior HEI is a multinominal variable; therefore, it could not be included into the correlational table.
Abbreviations: HEI, Higher Education Institutions; GGPA, graduate grade point average; UGPA, undergraduate grade point average.
*p< .05; **p< .01; ***p< .001.
FIGURE 1 The model with undergraduate
academic indicatorspredictors of graduate
study success. HEI, Higher Education
Institutions; GGPA, graduate grade point
average; UGPA, undergraduate grade point
average.
KURYSHEVA ET AL.
|
7
as their bachelors; therefore, their undergraduate thesis grade was
available. Tables 3.2,4.2, and 5.2 are based on analyses of Samples 1
and 3 which included students from all types of HEIs.
3.1 |Graduate degree completion as a dependent
variable
Tables 3.1 and 3.2 show that the result from uncorrected correlations
UGPA as a significant predictor of degree attainmentholds even after
accounting for the dependency of students within groups and programs
by applying hierarchical modelling (Model 1), which is in line with
Hypothesis 1. Table 3.1 shows that once thesis grade is added to the
model with UGPA, the model fit increases significantly, though the
estimate for undergraduate thesis grade does not reach the chosen
alpha level of 0.05. The improvement in AIC is small. Even though the
hypothesized positive relationship between thesis grade and graduate
time to degree was detected (Hypothesis 2; see Table 2), we note that
thesis grade does not show incremental validity beyond UGPA in
predicting degree attainment.
Further models provide the results for the incremental validity of
type of HEI above and beyond UGPA. Model 2 of Table 3.2 shows
that students from Dutch research universities and from foreign HEIs
have higher odds of completing a graduate program compared to
students from Dutch universities of applied sciences and students
from Dutch university colleges. This finding is partially in line with
Hypothesis 3 (where it concerns students from Dutch universities of
applied sciences, but not where it concerns students from Dutch
university colleges). We also note that with adding each predictor,
the Akaike Information Criterion (AIC)an indicator of relative quality
of statistical modelsimproves but rather to a small extent.
3.2 |Graduate grade point average as a dependent
variable
Tables 4.1 and 4.2 show the positive relationship between percentile
rank of UGPA and GGPA (Model 1), as expected in Hypothesis 1. This
significant positive relationship holds even after including other
predictors in the model. Table 4.1 shows the incremental validity of
prior thesis grade beyond percentile rank of UGPA (Model 2). This
was expected, according to Hypothesis 2. The model with significant
predictors (Model 2) explained substantial amount of variance in
GGPA (40%).
Table 4.2 depicts the results for the incremental validity of Type
of prior HEI beyond percentile rank of UGPA. Model 2 shows that
students from Dutch universities of applied sciences attainted
significantly lower GGPAs compared to students from Dutch
research universities, Dutch university colleges, and foreign HEIs.
This finding is in line with Hypothesis 3. The addition of type of HEI
increased the explained variance in GGPA by a small amount (2%).
The model with all study variables explained almost onethird of the
total variance in GGPA.
3.3 |Graduate time to degree as a dependent
variable
Tables 5.1 and 5.2 show that the predictive validity of UGPA for
graduate time to degree holds even after accounting for the
dependency of students within hierarchical structure: the higher
the percentile rank of UGPA, the shorter graduate time to degree
(Model 1). This is in line with Hypothesis 1. Table 5.1 shows that
undergraduate thesis grade has incremental predictive validity above
and beyond UGPA: Students with higher undergraduate thesis grade
take less time to complete a research master's program (Model 2).
This finding is in line with Hypothesis 2. The total amount of
explained variance in graduate time to degree is small.
Adding type of prior HEI to the model with UGPA significantly
improves the model fit (Table 5.2, Model 2). Students, who completed
their undergraduate degree outside of the Netherlands, have
significantly shorter time to graduate degree than students from
Dutch research universities and students from Dutch university
colleges. There is no significant difference in graduate time to degree
between students from Dutch universities of applied sciences and
T A B L E 3.1 Hierarchical regression results for graduate degree
completion (Sample 2)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 8.95***
[6.79, 11.80]
9.17***
[6.92, 12.14]
9.28***
[7.00, 12.32]
Percentile rank
of UGPA
2.01*
[1.00, 1.02]
1.00
[1.00, 1.01]
Thesis grade 1.38
[1.00, 1.91]
Random effects
Variance components
Level 1 3.29 3.29 3.29
Level 2 0.24 0.25 0.24
Level 3 <0.001 <0.001 <0.01
Goodness of fit
Deviance 3153.91 3148.05 3144.29
Number of estimated
parameters
345
Model comparison test χ
2
(1) = 5.87*χ
2
(1) = 3.76*
AIC 3159.91 3156.05 3154.29
Note: The reported estimates of predictors are odds ratios. Confidence
intervals are in square brackets.
Abbreviations: AIC, Akaike Information Criterion; GGPA, graduate grade
point average; UGPA, undergraduate grade point average.
*p< .05; ***p< .001.
8
|
KURYSHEVA ET AL.
T A B L E 3.2 Hierarchical regression results for graduate degree completion (Sample 1)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 7.35*** [6.00, 9.01] 7.43*** [6.11, 9.05] 4.39*** [2.64, 7.32]
Percentile rank of UGPA 1.01*[1.00, 1.01] 1.01*[1.00, 1.01]
Dummies (Type of prior HEI)
a
Dutch research university 1.81*[1.13, 2.92]
Dutch university college 0.82 [0.40, 1.67]
Foreign HEI 3.91** [1.42, 10.78]
Random effects
Variance components
Level 1 3.29 3.29 3.29
Level 2 0.07 0.08 0.08
Level 3 0.01 <0.01 <0.01
Goodness of fit
Deviance 4633.71 4627.82 4612.15
Number of estimated parameters 3 4 7
Model comparison test χ
2
(1) = 5.89*χ
2
(3) = 15.67**
AIC 4639.71 4635.82 4626.15
Note: The reported estimates of predictors are odds ratios. Confidence intervals are in square brackets.
Abbreviations: AIC, Akaike Information Criterion; CI, confidence interval; HEI, Higher Education Institutions; UGPA, undergraduate grade point average.
a
The reference category: Dutch university of applied sciences.Rerunning analysis to test other dummies of types of prior HEI in Model 2 delivers also
other significant differences, namely for a dummy variable Dutch university college versus Dutch research university [ref],Exp(b) = 0.45**, CI = [0.26,
0.80] and for Foreign HEI versus Dutch university college [ref],Exp(b) = 4.77**, CI = [1.65, 13.76].
*p< .05; **p< .01; ***p< .001.
T A B L E 4.1 Hierarchical regression results for graduate grade point average (Sample 4)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 7.82*** [7.70, 7.94] 7.84*** [7.77, 7.93] 7.86*** [7.80, 7.91]
Percentile rank of prior average grade 0.01*** [0.01, 0.01] 0.01*** [0.01, 0.01]
Prior thesis grade 0.21*** [0.16, 0.25]
Random effects
Variance components
Level 1 0.28 0.19 0.18
Level 2 0.01 <0.01 <0.01
Level 3 0.03 0.01 0.01
Total explained variance (%) 35 40
Goodness of fit
Deviance 1765.58 1348.2282.1 1272.64
Number of estimated parameters 4 5 6
Model comparison test χ
2
(1) = 417.35*** χ
2
(1) = 75.58***
Note: Confidence intervals are in square brackets.
***p< .001.
KURYSHEVA ET AL.
|
9
T A B L E 4.2 Hierarchical regression results for graduate grade point average (Sample 3)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 7.81*** [7.69, 7.93] 7.82*** [7.74, 7.90] 7.61*** [7.49, 7.73]
Percentile rank of UGPA 0.01*** [0.01, 0.01] 0.01*** [0.01, 0.01]
Dummies (Type of prior HEI)
a
Dutch research university 0.23*** [0.13, 0.33]
Dutch university college 0.29*** [0.15, 0.43]
Foreign HEI 0.25*** [0.11, 0.39]
Random effects
Variance components
Level 1 0.27 0.20 0.20
Level 2 0.01 <0.01 <0.01
Level 3 0.04 0.02 0.01
Total explained variance (%) 30 32
Goodness of fit
Deviance 2463.46 1985.87 1960.12
Number of estimated parameters 4 5 8
Model comparison test χ
2
(1) = 477.59*** χ
2
(3) = 25.75***
Note: Confidence intervals are in square brackets.
Abbreviations: HEI, Higher Education Institutions; UGPA, undergraduate grade point average.
a
The reference category: Dutch university of applied sciences.
***p< .001
T A B L E 5.1 Hierarchical regression results for graduate time to degree (Sample 4)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 30.41*** [29.53, 31.29] 30.35*** [29.39, 31.31] 30.29*** [29.33, 31.25]
Percentile rank of prior average grade 0.04*** [0.06, 0.02] 0.01 [0.03, 0.01]
Prior thesis grade 1.98*** [1.16, 2.80]
Random effects
Variance components
Level 1 59.33 58.23 56.90
Level 2 1.06 0.88 1.13
Level 3 1.32 1.75 1.75
Total explained variance (%) 1 3
Goodness of fit
Deviance 7722.91 7702.21 7679.92
Number of estimated parameters 4 5 6
Model comparison test χ
2
(1) = 20.70*** χ
2
(1) = 22.28***
Note: Confidence intervals are in square brackets.
***p< .001.
10
|
KURYSHEVA ET AL.
students from other types of HEIs. These results do not support
Hypothesis 3. Again, the total explained variance of the model with
all study variables included is small.
4|DISCUSSION
We tested whether (and to what extent) we can predict graduate
study success using student undergraduate academic indicators in a
sample of students across several mastersprograms in the life
sciences. Our study found that the strongest predictor was percentile
rank of UGPA which showed predictive validity for all three
outcomes: The higher percentile rank of UGPA was related to highr
odds of completing a graduate program, higher GGPA, and shorter
time to degree. Undergraduate thesis grade had incremental validity
beyond UGPA in predicting GGPA and graduate time to degree: The
higher undergraduate thesis grade, the higher GGPA and shorter
graduate time to degree. Type of prior HEI was found to be predictive
of degree completion and GGPA: Students from Dutch research
universities and from foreign HEIs have higher odds of completing a
graduate program compared to students from Dutch universities of
applied sciences and students from Dutch university colleges. We
found that our models explain substantial amounts of variance in
GGPA but not in graduate time to degree. We also found that our
models predicted odds of graduate degree completion only to a small
extent.
4.1 |Predictive value of undergraduate academic
indicators for graduate degree completion
We expected that UGPA and thesis grade would be positively related
to degree attainment (Hypothesis 1 and 2, respectively). Our data
supported both expectations, even though we note that thesis grade
did not show incremental validity above UGPA. We also expected
that students from universities of applied sciences will have lower
odds of completing their research master's program because their
undergraduate training was not researchintensive (Hypothesis 3).
This hypothesis was supported by our data. In addition to that,
however, we also discovered that students from Dutch university
colleges (which officially represent part of research universities) also
have lower odds of completing their research master's program. We
explain this by the fact that students in Dutch university colleges
follow Liberal Arts and Sciences education, which teaches a wide
T A B L E 5.2 Hierarchical regression results for graduate time to degree (Sample 3)
Variable Model 0 Model 1 Model 2
Fixed effects
Intercept 29.64*** [28.78, 30.50] 29.57*** [28.63, 30.51] 28.56*** [26.91, 30.21]
Percentile rank of UGPA 0.04*** [0.06, 0.02] 0.04*** [0.06, 0.02]
Dummies (Type of prior HEI)
a
Dutch research university 1.37 [0.12, 2.86]
Dutch university college 0.62 [1.73, 2.97]
Foreign HEI 2.14 [4.28, 0.00]
Random effects
Variance components
Level 1 54.51 53.58 52.88
Level 2 1.06 0.80 0.85
Level 3 1.60 2.05 2.00
Total explained variance (%) 1 3
Goodness of fit
Deviance 10771.21 10742.23 10722.34
Number of estimated parameters 4 5 8
Model comparison test χ
2
(1) = 28.98*** χ
2
(3) = 19.89***
Note: Confidence intervals are in square brackets. The analysis of other dummies in Model 2 showed significant effects of dummy variables Foreign HEI
versus Dutch research university(b=3.51***, 95% CI [5.16, 1.86]) and Foreign HEI versus Dutch university college(b=2.76*, 95% CI
[5.19, 0.33]).
Abbreviations: CI, confidence interval; HEI, Higher Education Institutions; UGPA, undergraduate grade point average.
a
The reference category: Dutch university of applied sciences.
*p< .05; ***p< .001.
KURYSHEVA ET AL.
|
11
range of topics: this range is significantly broader than more specific
courses that students follow within regular curricula of Dutch and
foreign research universities. Considering that degree completion is a
motivationally determined outcome (Kuncel et al., 2014), it may be
that the latter two groups are more motivated to persist in their
master's education as they are more familiar with specific topics and
had chosen them more conscientiously, while students from
university colleges find it difficult to motivate themselves to study
the narrow scientific topics within one specific field.
Importantly, the examined undergraduate academic indicators
predict graduate degree completion only to a small extent. We see
two possible complementary reasons for this finding. The first reason
might be that the dropping out of students in this sample was not
related to their academic ability but to other factors during their
mastersprograms. As the empirical research shows, these could be
reasons related to psychological resources, personality, study
motivation, study conditions, study decisions, institutional guidance,
and study performance during a graduate program (Cox et al., 2009;
Heublein, 2014). The second plausible reason might be that degree
completion is determined by conscientiousness, motivation, drive,
interest, or adaptability (Kuncel et al., 2014; Schwager et al., 2015)
and, therefore, it is a hardtopredict outcome, especially using prior
academic indicators which do not directly assess these qualities. It
might be that methods that evaluate noncognitive constructs (e.g.,
conscientiousness or time management; Butter & Born, 2012)or
advanced assessment of academic work (presentations, various
operationalizations of research experience; Pacheco et al., 2015)
are better suited for prediction of degree completion.
4.2 |Predictive value of undergraduate academic
indicators for GGPA
When predicting GGPA, the strongest predictor in our analysis was
UGPA. The predictive validity of UGPA corroborates our Hypothesis
1 which was based on theoretical underpinnings of UGPA as a
complex measure that captures several influential determinants of
study success and on findings of previous studies showing a stable
relation between UGPA and GGPA (Chadi & de Pinto, 2018; Howell
et al., 2014; Park et al., 2018; Zimmermann et al., 2017). Under-
graduate thesis grade showed incremental validity above UGPA and
slightly improved the predictive power of our model. Considering that
prior studies on undergraduate research experience as a predictor of
GGPA have never operationalized it through undergraduate thesis
grade (see Miller et al., 2021; for the overview of prior operationa-
lizations), we cannot place our finding in the context of literature.
However, it does align with our Hypothesis 2 and corroborates the
metaanalytical findings which show that prior achievement (in this
case, performance on a researchrelated task such as undergraduate
thesis) is one of the best predictors of future achievement
(Richardson et al., 2012; Schneider & Preckel, 2017). We consider
that it could be beneficial to explore this operationalization further,
especially as it allows us to place students on one metric, at least
those who come from the same prior HEI. In doing so, it is important
to keep the possible effects of unintentional internal grading culture
in mind.
Our next finding regarding prediction of GGPA is that students
from universities of applied sciences obtain significantly lower
GGPAs than students from other types of HEIs, in line with
Hypothesis 3. This can be explained by a more practiceoriented
curriculum of universities of applied sciences versus a research
oriented curriculum of research universities. It makes sense that the
lack of preparation for the theoretical aspects of research places
these students at a disadvantage compared with students from
researchoriented HEIs and leads at the end of a research master's
program to lower GGPA. It is important to note, however, that
despite incremental validity beyond and above UGPA, the gain in
explained variance from type of HEI is small. This means that GGPA is
not heavily determined by type of prior HEI in presence of UGPA, as
we expected.
4.3 |Predictive value of undergraduate academic
indicators for graduate time to degree
With caution, we hypothesized that UGPA and thesis grade would be
negatively related to graduate time to degree (Hypothesis 1 and 2).
Even though we found these hypothesized relationships to be
statistically significant, the extent to which we can explain variance in
graduate time to degree is small (around 3%). We also hypothesized
that students from universities of applied sciences may take longer
time to complete their research master's program because their prior
practiceoriented training may not provide them with all the
knowledge and skills needed to complete the researchintense
internships and assignments in the required time (Hypothesis 3).
Our results do not support this hypothesis. It might be that the
students from universities of applied sciences manage well with the
timeline of their researchintense curriculum because they were
trained in practical aspects such as working with biological material,
keeping a logbook of experiments, and so forth. While these students
from universities of applied sciences may struggle with designing a
research proposal, formulating theorydriven hypotheses, and so
forth, their peers with undergraduate degrees from research
universities might struggle with the practical aspects, which students
from universities of applied sciences are good at. All together, these
strengths and weaknesses of students from two different types of
universities balance each other, which leads to the absence of
significant difference in their graduate time to degree.
What we did not hypothesize in our Hypothesis 3, but what we
found is that international students have significantly shorter time to
degree than Dutch students from research universities and university
colleges. We think that this finding is striking. Although the international
students may experience a cultural shock (Zhou et al., 2008), both within
and outside of studies (housing, teaching methods, and adjustment to
new culture), they still take less time to complete a master's degree than
local students who had researchintensive undergraduate education in
12
|
KURYSHEVA ET AL.
their own country. Some of this can be explained by the fact that all
international students in our analyses were from the EU, therefore, their
academic and social integration scores are comparable to domestic
students (Rienties et al., 2014). This finding can also be explained by the
fact that many international students receive grants or loans as a part of
an international exchange program, and this funding is usually provided
for the official duration of their master's program (e.g., for 2 years in the
case of mastersprograms addressed in this study). Therefore, finishing
on time could be a strong motivator for international students because it
prevents them from taking out further loans or having to apply for
additional grant money.
Despite the relationships that we discussed above, it is important
that all three examined undergraduate academic indicators predict
graduate time to degree only to a small extent. This result is in line
with findings on undergraduate level where it was shown that
precollege characteristics account for a small amount of variance in
time to degree (Yue & Fu, 2017), and we have two possible
explanations. The first explanation is that among the undergraduate
academic indicators we examined, there were none that measured
motivation of students. There are, however, some indications that
intrinsic motivation exerts positive influence on study progress
(Slijper et al., 2016). We could not use assessments of motivation due
to a practical reason (they were not available in our institutional data).
We would also like to note that the existing selection methods based
on motivation such as personal statements have not been shown as
valid instruments (Murphy et al., 2009). Thus, we do not expect that
having assessments of motivation available would deliver a substan-
tial gain in explained variance in graduate time to degree.
Our second explanation is that what occurs during a graduate
program plays a more important role in graduate study delays than
undergraduate academic indicators. The factors during a graduate
program that are influential for study delays are individual (e.g.,
student sense of belonging), supervisory (e.g., clarity of supervisor's
communication with their student), and departmental/institutional
(e.g., graduate policies and practices, workload during a program; de
Valero, 2001; Ruete et al., 2021; van de Schoot et al., 2013; van Rooij
et al., 2021). In additon to these three factors, we think that research
mastersstudents might feel pressure to produce earlycareer
publications (Crane & Pearson, 2011) because publishing academic
work makes a difference when applying to a researchoriented
position in the future (Stoilescu & McDougall, 2010). This pressure
impacts studentsdecisions to produce a publication at the cost of
longer time to degree. Overall, it appears that time to degree
represents a variable that is hard to predict using information
available upon admissions to a graduate program.
4.4 |Theoretical contributions
Our study adds to the existing literature on valid selection methods in
the following regards. First, we clarified the mixed findings on
whether UGPA is predictive of graduate time to degree and degree
attainment. In line with a number of prior studies (Howell et al., 2014;
MendozaSanchez et al., 2022; MonetaKoehler et al., 2017;
Schwager et al., 2015; Wollast et al., 2018), we found that UGPA is
a statistically significant predictor of graduate time to degree and
degree attainment. However, it explains little variance in time to
degree and increases the odds of completing a graduate degree to a
small extent. This may be a reason why a number of other studies
(Cox et al., 2009; Dabney, 2012; Dore, 2017; MonetaKoehler
et al., 2017) failed to detect this relationship using their data. So even
though UGPA seems to be weakly related to graduate time to degree
and degree attainment, researchers may want to focus on exploring
whether other information, available upon admissions, can add or
even outperform UGPA in prediction of these two dimensions of
graduate study success.
The second theoretical contribution is that this study found
support for the theoretical underpinnings regarding research experi-
ence as one of the determinants of graduate study success (Gilmore
et al., 2015; Miller et al., 2021). Our operationalization of
undergraduate research experience in the form of thesis grade
appeared as a valid predictor and delivered incremental validity
above UGPA. Such an operationalization has not been tested before,
and the replication studies would be valuable: More evidence of the
usefulness of this operationalization of undergraduate research
experience could justify the inclusion of thesis grade as an admissions
requirement for researchoriented mastersprograms.
The third theoretical contribution is that our assumption that
prior HEI may represent one of the determinants of graduate study
success found empirical support. Along with that, we showed that
this relationship is weak. The practitioners, therefore, will possibly be
facing a dilemma whether to use type of HEI as a selection method or
not. We discuss these and other practical aspects of our findings in
the section below.
4.5 |Practical contributions
The application of undergraduate thesis grade as a selection method
could be considered in practice, especially in programs with a similar
researchoriented focus and where admission committees regard
GGPA as an important dimension of graduate study success of their
students. However, we call for a conscious choice in doing so. If we
select students, who are already good in what they are supposed to
do during their graduate program, what is the added value of the
program in the learning process? Do we not exclude students who
come from nonresearch undergraduate schools? Or should programs
select and teach those who will gain the most (e.g., students who
were less successful in researchrelated tasks such as an under-
graduate thesis or simply never had a chance to work on a thesis
during their undergraduate studies). We suggest that universities and
graduate programs make this decision of using undergraduate thesis
grade for selection purposes, accounting for their mission statements
and vision of their student body.
Another practical consideration regarding the implementation
of grades is that although this study showed their predictive validity
KURYSHEVA ET AL.
|
13
(i.e., of undergraduate thesis grade and UGPA), it is important to
account for the context in which these grades were obtained. While
the traditional meritocratic equality of opportunity model of fair
access implies that study places go to the most highly capable,
irrespective of their socialeconomic background, an alternative
model is gaining recognition and states that indicators of merit,
including grades, need to be assessed contextually in light of an
applicant's socioeconomic circumstances (Boliver & Powell, 2021). To
make this possible, the individual HEIs should be allowed to gather
and use data on socioeconomic status for conducting research on this
topic which is almost never the case in certain European countries
(partly a consequence of the recently adopted European Data
Protection Regulation).
As for the type of prior HEI, when applying it as a selection
criterion, admissions committees should consider the indications that
certain types of prior HEI are associated with lower socioeconomic
status (e.g., on average, students from Dutch universities of applied
sciences tend to have lower socioeconomic status than their peers in
research universities; The Netherlands Association of Universities of
Applied Sciences, 2012). Therefore, the application of type of prior
HEI as a selection criterion could mask student selection based on
socioeconomic characteristics which would be morally and legally
inappropriate. Instead, it might be practical to provide these students
with additional guidance during their graduate studies to ensure
graduate study success.
4.6 |Limitations
This study does not come without limitations. First, we used data of
already selected students and did not have information on how
students, who were not selected, would have performed. However,
since we were interested in detecting relationships between
undergraduate academic indicators and graduate study success and
not establishing the means or cutoff scores, there is no reason to
assume that these relationships would be fundamentally different in a
wider sample of all applicants.
The next limitation is that student admissions data registered in
the administrative system at this graduate school are limited to
variables from official transcripts. The scores on other documents
that require additional assessment of admissions committees
(recommendation letters, interviews, personal statements, etc.) were
not standardized across programs at this graduate school, therefore,
could not be included into the statistical analysis. However, the fact
that our data came from official transcripts basically excluded the
possibility of unreliable data. Moreover, the undergraduate academic
indicators, which were the focus of this paper, are usually present in
most similar graduate schoolsdata sets which allows considerable
generalizability of our findings.
Another limitation is that this study is conducted within one
graduate school of one university. However, students from 13
different research mastersprograms were included from relatively
diverse field of studies which provides an opportunity for a certain
generalizability of our findings for other researchoriented graduate
programs.
5|CONCLUSIONS
In this study, we aimed to validate certain widely used undergraduate
indicators to help create a more objective, efficient, and inclusive
master's admissions process. What we found is that undergraduate
thesis grade is a valid predictor of GGPA in addition to UGPA.
Therefore, these indicators should be considered for selection purposes
for researchoriented graduate programs in the life sciences and
possibly for programs with a similar focus. We also showed that type of
prior HEI does not add much to the prediction of graduate study
success after the prior grades have been taken into consideration. All
examined undergraduate academic indicators did not contribute much
to prediction of graduate degree completion and time to degree. While
this study took place in a Dutch HEI, our findings, especially those on
UGPA and undergraduate thesis grade are generalizable to research
intensive programs across EHEA. The graduate programs outside EHEA
can consider them as well, accounting for the differences in structure of
graduate programs. Likewise, our models, which combined different
international student groups by using percentile ranks, can be applied
across different HEIs.
ACKNOWLEDGMENTS
The authors would like to thank prof. dr. Marijk van der Wende for
her supervision of this study and the overarching PhD project
focused on graduate selective admissions. The authors are also
grateful to educational data specialist Guus Dikker. His professional
advice on data processing was very valuable. This study received
internal institutional funding.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
DATA AVAILABILITY STATEMENT
Raw data were generated at the Utrecht University Graduate School
of Life Sciences. Derived data supporting the findings of this study
are available from the corresponding author A.K. on request.
ORCID
Anastasia Kurysheva http://orcid.org/0000-0001-7425-1345
Gönül Dilaver http://orcid.org/0000-0002-6227-2197
ENDNOTES
1
We have also conducted an analysis where we included prior field of
study (biology, biomedical sciences, medicine, psychology, chemistry,
liberal arts and sciences, pharmaceutics, biotechnology, biology and
medical laboratory) as a predictor in the last step of our model. This
analysis delivered negligible increment in explained variance. It is
available for interested parties by request.
2
Ideally, we would have wanted to use undergraduate thesis grade as a
predictor in all our analyses. Unfortunately, these grades were not
14
|
KURYSHEVA ET AL.
registered in the administrative system for students who had come
from different universities. They were registered only for students who
studied their bachelor's program at the same university as their
master's program. For this reason, we had to analyze four samples
instead of two.
REFERENCES
Association of Universities in the Netherlands. (2021). Instroom nieuwe
masterstudenten [Influx of new mastersstudents]. https://vsnu.nl/
instroom-master.html
Bean, J. P. (1980). Dropouts and turnover: The synthesis and test of a
causal model of student attrition. Research in Higher Education,12(2),
155187. https://doi.org/10.1007/BF00976194
Boliver, V., & Powell, M. (2021). Fair admission to universities in England:
Improving policy and practice. Nuffield Foundation.https://www.
nuffieldfoundation.org/wp-content/uploads/2021/01/Fair-
admission-to-universities-in-England.pdf
Borghans, L., Golsteyn, B., Heckman, J., & Humphries, J. (2016). What
grades and achievement tests measure. Proceedings of the National
Academy of Sciences of the United States of America,113(47),
1335413359. https://doi.org/10.1073/pnas.1601135113
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the
culture of learning. Educational Researcher,18(1), 3542. https://doi.
org/10.3102/0013189X018001032
Burton, N. W., & Wang, M. (2005). Predicting longterm success in graduate
school: A collaborative validity study (Report No. 9914R. ETS RR05
03). Educational Testing Service. http://grad.uga.edu/wp-content/
uploads/2017/09/GRE_Research_Report.pdf
Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual
ability, learning style, personality, achievement motivation and
academic success of psychology students in higher education.
Personality and Individual Differences,29(6), 10571068. https://
doi.org/10.1016/S0191-8869(99)00253-6
Butter, R., & Born, M. P. (2012). Enhancing criterionrelated validity
through bottomup contextualization of personality inventories: The
construction of an ecological conscientiousness scale for Ph.D.
candidates. Human Performance,25(4), 303317. https://doi.org/10.
1080/08959285.2012.703730
Cabrera, A., Nora, A., & Castaneda, M. (1993). College persistence:
Structural equations modeling test of an integrated model of student
retention. The Journal of Higher Education,64(2), 123139. https://
www.jstor.org/stable/2960026
Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory
of performance. Personnel Selection in Organizations,3570,3570.
Chadi, A., & de Pinto, M. (2018). Selecting successful students? Under-
graduate grades as an admission criterion. Applied Economics,50(28),
30893105. https://doi.org/10.1080/00036846.2017.1418072
Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship:
Teaching the crafts of reading, writing, and mathematics. In L. B.
Resnick (Ed.), Knowing, learning and instruction (pp. 453494). Essays
in honor of Robert Glaser.
Cox, G. W., Hughes, W. E., Jr., Etzkorn, L. H., & Weisskopf, M. E. (2009).
Predicting computer science Ph.D. completion: A case study. IEEE
Transactions on Education,52(1), 137143. https://doi.org/10.1109/
TE.2008.921458
Crane, N. J., & Pearson, Z. (2011). Can we get a pub from this? Reflections
on competition and the pressure to publish while in graduate school.
The Geographical Bulletin,52,7780. https://gammathetaupsilon.
org/the-geographical-bulletin/2010s/volume52-2/article4.pdf
Dabney, K. P. T. (2012). Differences within: A comparative analysis of
women in the physical sciencesmotivation and background factors
(Publication No: AAT 3507074) [Doctoral dissertation, ProQuest
Dissertations and Theses]. University of Virginia https://doi.org/10.
18130/V3XZ9R
de Valero, Y. F. (2001). Departmental factors affecting timetodegree and
completion rates of doctoral students at One LandGrant Research
Institution. The Journal of Higher Education,72(3), 341367. https://
doi.org/10.2307/2649335
Dore, M. L. (2017). Factors in the admissions process influencing persistence
in a Master's of Science program in marine science [Doctoral
dissertation]. Nova Southeastern University. https://nsuworks.
nova.edu/fse_etd/110/
Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure
and acquisition. American Psychologist,49(8), 725747. https://doi.
org/10.1037/0003-066X.49.8.725
European Higher Education Area. (n.d.). Threecycle system. https://
www.ehea.info/page-three-cycle-system
Eurydice Network. (2020). Key features of the Dutch education system.
https://eacea.ec.europa.eu/national-policies/eurydice/content/
netherlands_en
Fu, Y. (2012). The effectiveness of traditional admissions criteria in predicting
college and graduate success for American and international students
[Doctoral dissertation]. University of Arizona. ProQuest LLC. https://
eric.ed.gov/?id=ED536665
Galla, B. M., Shulman, E. P., Plummer, B. D., Gardner, M., Hutt, S. J.,
Goyer, J. P., & Duckworth, A. L. (2019). Why high school grades are
better predictors of ontime college graduation than are admissions
test scores: The roles of selfregulation and cognitive ability.
American Educational Research Journal,56(6), 20772115. https://
doi.org/10.3102/0002831219843292
Gilmore, J., Vieyra, M., Timmerman, B., Feldon, D., & Maher, M. (2015).
The relationship between undergraduate research participation and
subsequent research performance of early career STEM graduate
students. The Journal of Higher Education,86(6), 834863. https://
doi.org/10.1353/jhe.2015.0031
Hall, J. D., O'Connell, A. B., & Cook, J. G. (2017). Predictors of student
productivity in biomedical graduate school applications. PLOS ONE,
12(1), e0169121. https://doi.org/10.1371/journal.pone.0169121
Heublein, U. (2014). Student dropout from German higher education
institutions. European Journal of Education,49(4), 497513. https://
doi.org/10.1111/ejed.12097
Howell, L. L., Sorenson, C. D., & Jones, M. R. (2014). Are undergraduate
GPA and general GRE percentiles valid predictors of student
performance in an engineering graduate program? International
Journal of Engineering Education,30(5), 11451165. https://
scholarsarchive.byu.edu/facpub/1343/
Kuncel, N. R., Kochevar, R. J., & Ones, D. S. (2014). A metaanalysis of
letters of recommendation in college and graduate admissions:
Reasons for hope. International Journal of Selection and Assessment,
22(1), 101107. https://doi.org/10.1111/ijsa.12060
Kurysheva, A., van Rijen, H. V. M., & Dilaver, G. (2019). How do admission
committees select? Do applicants know how they select? Selection
criteria and transparency at a Dutch University. Tertiary Education
and Management,25(4), 367388. https://doi.org/10.1007/s11233-
019-09050-z
Kyllonen, P., Walters, A. M., & Kaufman, J. C. (2005). Noncognitive
constructs and their assessment in graduate education: A review.
Educational Assessment,10(3), 153184. https://doi.org/10.1207/
s15326977ea1003_2
Lub, A., van der Wende, M., & Witte, J. (2003). Bachelormaster
programmes in the Netherlands and Germany. Tertiary Education and
Management,9(4), 249266. https://doi.org/10.1080/1358 3883.
2003.9967108
MendozaSanchez, I., deGruyter, J. N., Savage, N. T., & Polymenis, M.
(2022). Undergraduate GPA predicts biochemistry PhD completion
and is associated with time to degree. CBELife Sciences Education,
21. https://doi.org/10.1187/cbe.21-07-0189
Miller, A., Crede, M., & Sotola, L. K. (2021). Should research experience
be used for selection into graduate school: A discussion and
KURYSHEVA ET AL.
|
15
metaanalytic synthesis of the available evidence. International
Journal of Selection and Assessment,29(1), 1928. https://doi.org/
10.1111/ijsa.12312
MonetaKoehler, L., Brown, A. M., Petrie, K. A., Evans, B. J., & Chalkley, R.
(2017). The limitations of the GRE in predicting success in
biomedical graduate school. PLOS ONE,12(1), e0166742. https://
doi.org/10.1371/journal.pone.0166742
Murphy, S. C., Klieger, D. M., Borneman, M. J., & Kuncel, N. R. (2009). The
predictive power of personal statements in admissions: A meta
analysis and cautionary tale. College & University,84(4), 8388.
Niessen, A. S. M., Meijer, R. R., & Tendeiro, J. N. (2017). Measuring non
cognitive predictors in highstakes contexts: The effect of self
presentation on selfreport instruments used in admission to higher
education. Personality and Individual Differences,106, 183189.
https://doi.org/10.1016/j.paid.2016.11.014
NVAO. (2016). Specification of additional criteria for research master's
programmes. https://www.nvao.net/files/attachments/.1255/Nadere_
uitwerking_aanvullende_criteria_onderzoeksmasters_30_mei_2016_
EN_def_nieuwe_huisstijl.pdf
OECD. (2022). International student mobility (indicator). https://doi.org/
10.1787/4bcf6fc3-en
Pacheco, W. I., Noel, R. J., Porter, J. T., & Appleyard, C. B. (2015). Beyond
the GRE: Using a composite score to predict the success of Puerto
Rican students in a biomedical PhD program. CBE Life Sciences
Education,14(2), 17. https://doi.org/10.1187/cbe.14-11-0216
Park, H.Y., Berkowitz, O., Symes, K., & Dasgupta, S. (2018). The art
and science of selecting graduate students in the biomedical
sciences: Performance in doctoral study of the foundational
sciences. PLOS One,13(4), e0193901. https://doi.org/10.1371/
journal.pone.0193901
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J.
(2016). How effective are selection methods in medical education? A
systematic review. Medical Education,50(1), 3660. https://doi.org/
10.1111/medu.12817
Patterson, F., Roberts, C., Hanson, M. D., Hampe, W., Eva, K.,
Ponnamperuma, G., Magzoub, M., Tekian, A., & Cleland, J. (2018).
2018 Ottawa consensus statement: Selection and recruitment to the
healthcare professions. Medical Teacher,40(11), 19. https://doi.
org/10.1080/0142159X.2018.1498589
Poropat, A. E. (2009). A metaanalysis of the fivefactor model of
personality and academic performance. Psychological Bulletin,135(2),
322338. https://doi.org/10.1037/a0014996
Posselt, J. R. (2016). Inside graduate admissions: Merit, diversity, and
faculty gatekeeping. Harvard University Press. https://doi.org/10.
1080/00220671.2016.1184524
Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates
of university studentsacademic performance: A systematic review
and metaanalysis. Psychological Bulletin,138(2), 353387. https://
doi.org/10.1037/a0026838
Rienties, B., Luchoomun, D., & Tempelaar, D. (2014). Academic and social
integration of master students: A crossinstitutional comparison
between Dutch and international students. Innovations in Education
and Teaching International,51(2), 130141. https://doi.org/10.
1080/14703297.2013.771973
Ruete, D., Zavala, G., Leal, D., Lopez Lira, P., Medina, L. P. S. M.,
Aravena, M. E., & Costa, G. (2021, July). Early detection of delayed
graduation in master's students [Paper presentation]. American
Society for Engineering Education Annual Conference, Virtual
Meeting. https://strategy.asee.org/36999
Salmi, J., & Bassett, R. M. (2014). The equity imperative in tertiary
education: Promoting fairness and efficiency. International Review of
Education,60(3), 361377. https://doi.org/10.1007/s11159-013-
9391-z
Schneider, M., & Preckel, F. (2017). Variables associated with achievement
in higher education: A systematic review of metaanalyses.
Psychological Bulletin,143(6), 565600. https://doi.org/10.1037/
bul0000098
Schwager, I. T. L., Hülsheger, U. R., Bridgeman, B., & Lang, J. W. B. (2015).
Graduate student selection: Graduate record examination, socio-
economic status, and undergraduate grade point average as
predictors of study success in a western European university.
International Journal of Selection and Assessment,23(1), 7179.
https://doi.org/10.1111/ijsa.12096
Slijper, J., Kunnen, E. S., Onstenk, J., & van Geert, P. (2016). The role of
time perspective, motivation, attitude, and preparation in educa-
tional choice and study progress. Education Research International,
2016,115. https://doi.org/10.1155/2016/1382678
Snijder, J. K. (2016). Mastering research: A study about the effectiveness
of research master programmes in the Netherlands [Doctoral
dissertation]. University of Twente. https://doi.org/10.3990/1.
9789036541466
Statista. (2020). Number of master's degrees earned in the United States
from 1949/50 to 2029/30, by gender. https://www.statista.com/
statistics/185160/number-of-masters-degrees-by-gender-
since-1950/
Stoilescu, D., & McDougall, D. (2010). Starting to publish academic
research as a doctoral student. International Journal of
Doctoral Studies,5,7992. http://ijds.org/Volume5/IJDSv5p079-
092Stoilescu299.pdf
Tai, J. (2020). Do College Grades Predict Future Success? https://www.
forbes.com/sites/theyec/2020/10/19/do-college-grades-predict-
future-success/?sh=60cb5a735af6
the Higher Education Statistics Agency. (2020). Higher education student
statistics: UK, 2018/19qualifications achieved. https://www.hesa.
ac.uk/news/16-01-2020/sb255-higher-education-student-
statistics/qualifications
The Netherlands Association of Universities of Applied Sciences. (2012).
Achtergrondnotitie. Toegankelijkheid Hoger Onderwijs en de
Rol Studiefinanciering [Background note. Access to higher
education and the role of Student Finance]. https://www.
vereniginghogescholen.nl/system/knowledge_base/attachments/
files/000/000/390/original/Toegankelijkheid_hoger_onderwijs_rol_
studiefinanciering_-_achtergrondnotitie_HBO-raad_over_
leenstelsel.pdf?1443427985
Tinto, V. (1975). Dropout from higher education: A theoretical
synthesis of recent research. Review of Educational Research,45(1),
89125.
Torgerson, P. C., Gascoine, L., Heaps, C., Menzies, V., & Younger, K.
(2014). Higher education access: Evidence of effectiveness of
university access strategies and approaches. A report to The Sutton
Trust. Durham University. https://www.suttontrust.com/wp-
content/uploads/2019/12/Higher-Education-Access-Report-1.pdf
van de Schoot, R., Yerkes, M. A., Mouw, J. M., & Sonneveld, H. (2013).
What took them so long? Explaining PhD delays among doctoral
candidates. PLoS One,8(7), e68839. https://doi.org/10.1371/
journal.pone.0068839
van der Wende, M. (2020). Netherlands, higher education. In M. David, &
M. Amey (Eds.), The SAGE encyclopedia of higher education (pp.
10791080). SAGE Publications, Inc. https://doi.org/10.4135/
9781529714395.n403
van Rooij, E., FokkensBruinsma, M., & Jansen, E. (2021). Factors that
influence PhD candidatessuccess: The importance of PhD project
characteristics. Studies in Continuing Education,43(1), 4867.
https://doi.org/10.1080/0158037X.2019.1652158
Weiner, O. D. (2014). How should we be selecting our graduate students.
Molecular Biology of the Cell,25(4), 429430. https://doi.org/10.
1091/mbc.e13-11-0646
Witte, J., van der Wende, M., & Huisman, J. (2008). Blurring boundaries:
How the Bologna process changes the relationship between
university and nonuniversity higher education in Germany, the
16
|
KURYSHEVA ET AL.
Netherlands and France. Studies in Higher Education,33(3), 217231.
https://doi.org/10.1080/03075070802049129
Wollast, R., Boudrenghien, G., Van der Linden, N., Galand, B., Roland, N.,
Devos, C., De Clercq, M., Klein, O., Azzi, A., & Frenay, M. (2018).
Who are the doctoral students who drop out? Factors associated
with the rate of doctoral degree completion in universities.
International Journal of Higher Education,7(4), 143156. https://
doi.org/10.5430/ijhe.v7n4p143
Younger, K., Gascoine, L., Menzies, V., & Torgerson, C. (2018). A
systematic review of evidence on the effectiveness of interventions
and strategies for widening participation in higher education. Journal
of Further and Higher Education,43(6), 132. https://doi.org/10.
1080/0309877X.2017.1404558
Yue, H., & Fu, X. (2017). Rethinking graduation and time to degree: A fresh
perspective. Research in Higher Education,58(2), 184213. https://
doi.org/10.1007/s11162-016-9420-4
Zhou, Y., JundalSnape, D., Topping, K., & Todman, J. (2008). Theoretical
models of culture shock and adaptation in international students in
higher education. Studies in Higher Education,33(1), 6375. https://
doi.org/10.1080/03075070701794833
Zimmermann, J., Brodersen, K. H., Heinimann, H. R., & Buhmann, J. M.
(2015). A modelbased approach to predicting graduatelevel
performance using indicators of undergraduatelevel performance.
Journal of Educational Data Mining,7(3), 151176. https://doi.org/
10.5281/ZENODO.3554733
Zimmermann, J., von Davier, A. A., Buhmann, J. M., & Heinimann, H. R.
(2017). Validity of GRE general test scores and TOEFL scores for
graduate admission to a technical university in Western Europe.
European Journal of Engineering Education,43(1), 144165. https://
doi.org/10.1080/03043797.2017.1343277
How to cite this article: Kurysheva, A., Koning, N., Fox, C. M.,
van Rijen, H. V. M., & Dilaver, G. (2022). Once the best
student always the best student? Predicting graduate study
success using undergraduate academic indicators: Evidence
from research mastersprograms in the Netherlands.
International Journal of Selection and Assessment,117.
https://doi.org/10.1111/ijsa.12397
KURYSHEVA ET AL.
|
17
... Note that this paper's purpose is to demonstrate the use of SDT in decision making in higher education, with student selection as an example. The reader is referred to the literature on admission and selection for details on selection instruments in specific contexts (Steenman et al., 2016;Wouters et al., 2016;Niessen et al., 2018;Kurysheva et al., 2019Kurysheva et al., , 2022. Three variables of interest in this demonstration, bachelor's average grade, bachelor's thesis grade and master's average grade are given on a Dutch scale from 1 to 10. ...
Article
Full-text available
Signal Detection Theory (SDT) is rarely used in higher education, yet has much potential in informing decision-making. In this methodological paper, we describe the potential of SDT for different higher education contexts and demonstrate its practical application. Both the commonly used regression analyses and SDT analyses provide information on the accuracy of a predictor, and thus which instrument(s) to use. SDT analyses, in addition, provide information on the effects of setting specific cut-off scores on outcomes of interest. SDT provides the sensitivity and specificity information for the chosen instrument(s) at specific cut-off scores (criteria in SDT). This allows for evidence-informed, deliberate choice of cut-off scores to steer toward desired outcomes. Depending on how undesirable false positives and false negatives are considered in a specific situation, a lower or higher cut-off score can be deemed adequate. Using SDT analyses in our example, we demonstrate how to use the results to optimize “real-life” student selection. However, selection is only one of many decision-making practices where SDT is applicable and valuable. We outline some of the areas within higher education decision-making and quality assurance, where SDT can be applied to answer specific questions and optimize decision-making.
Article
Full-text available
Prior research experience is widely considered by graduate school admissions committees in the United States of America. Here, we use meta‐analytic methods and data from 18 unique samples and a total sample size of 3,525 students to shed light on the validity of prior research experience as a predictor of graduate school performance. Prior research experience was largely unrelated to academic performance (ρ = .01, k = 8, N = 1,419), degree attainment (ρ = .05, k = 3, N = 140), professional/practice performance (ρ = .06, k = 4, N = 1,120), and publication performance (ρ = .11, k = 7, N = 1,094). We also discuss whether consideration of prior research experience may unfairly disadvantage the students with lower levels of SES, students with childcare or eldercare responsibilities, and students from institutions at which research opportunities are limited.
Article
Full-text available
In this study, we investigated the currently applied selective admission criteria and tools of the two-year research master’s programs of both the Graduate Schools of Life Sciences and Natural Sciences of Utrecht University (the Netherlands). In addition, we evaluated their transparency to applicants. Both admissions staff members and applicants participated. To determine admission criteria that are important for admission decisions, we ranked 51 admission criteria and, on their basis, combined into six domains: academic background, grades, cognitive ability, research background, personality and personal competencies, motivation factors. To evaluate transparency, we contrasted the perceptions of applicants with the actual importance of admission criteria, as reported by admission staff members. We found that admissions criteria related to personality and personal competencies are less important in admission decisions than criteria related to grades, academic background and motivation. The applicants find the admissions decisions transparent to a moderate degree. This study also revealed that selectors use criteria and tools both with and without predictive value for later graduate performance. Moreover, some of the currently applied admission instruments might be prone to admission biases. We advocate selectors to use admission criteria and tools that are evidence-based, resistant to admission biases, and transparent to the applicants.
Article
Full-text available
High dropout rates, delay, and dissatisfaction among PhD students are common problems in doctoral education. Research shows that many different factors are associated with doctoral success, but these factors have not often been studied simultaneously. Moreover, characteristics of the PhD project are mostly neglected. In this study, we investigate which supervision, psychosocial, and project characteristics are related to satisfaction, progress, and quit intentions in a sample of 839 PhD candidates at a university in the Netherlands. Results of regression analyses show that experienced workload was negatively related to satisfaction and progress and positively to quit intentions. The quality of the supervisor-PhD candidate relationship, the PhD candidate's sense of belonging, the amount of freedom in the project, and working on a project closely related to the supervisor's research were positively related to satisfaction and negatively to quit intentions. The high workload of PhD candidates should be a major point of attention for universities who wish to increase their rates of PhD completion and PhD candidates' satisfaction. In addition, the 'match' between PhD candidate and supervisor is crucial, both personally-a good relationship-and academically, i.e. that the PhD candidate works on a topic closely related to the supervisor's research.
Article
Full-text available
The graduate admissions process is crucial for controlling the quality of higher education, yet, rules-of-thumb and domain-specific experiences often dominate evidence-based approaches. The goal of the present study is to dissect the predictive power of undergraduate performance indicators and their aggregates. We analyze 81 variables in 171 student records from a Bachelor's and a Master's program in Computer Science and employ state-of-the-art methods suitable for high-dimensional data-settings. We consider regression models in combination with variable selection and variable aggregation embedded in a double-layered cross-validation loop. Moreover, bootstrapping is employed to identify the importance of explanatory variables. Critically, the data is not confounded by an admission-induced selection bias, which allows us to obtain an unbiased estimate of the predictive value of undergraduate-level indicators for subsequent performance at the graduate level. Our results show that undergraduate-level performance can explain 54% of the variance in graduate-level performance. Significantly, we unexpectedly identified the third-year grade point average as the most significant explanatory variable, whose influence exceeds the one of grades earned in challenging first-year courses. Analyzing the structure of the undergraduate program shows that it primarily assesses a single set of student abilities. Finally, our results provide a methodological basis for deriving principled guidelines for admissions committees.
Article
Full-text available
Selection and recruitment into healthcare education and practice is a key area of interest for educators with significant developments in research, policy, and practice in recent years. This updated consensus statement, developed through a multi-stage process, examines future opportunities and challenges in selection and recruitment. There is both a gap in the literature around and a compelling case for further theoretical and empirical literature to underpin the development of overall selection philosophes and policies and their enactment. More consistent evidence has emerged regarding the quality of different selection methods. Approaches to selection are context-dependent, requiring the consideration of an institution’s philosophy regarding what they are trying to achieve, the communities it purports to serve, along with the system within which they are used. Diversity and globalization issues continue to be critically important topics. Further research is required to explore differential attainment and explain why there are substantial differences in culturally acceptable ways of approaching diversity and widening access. More sophisticated evaluation approaches using multi-disciplinary theoretical frameworks are required to address the issues. Following a discussion of these areas, 10 recommendations are presented to guide future research and practice and to encourage debate between colleagues across the globe.
Article
Full-text available
The issue of considerable dropout rate in doctoral programs is well documented across a large number of countries. However, few studies address the factors associated with doctoral completion among Non-U.S. countries, multiple universities and fields of research. Nor do they investigate the interactions between these factors. The present paper aimed to overcome these limitations and analyzed the population of doctoral students in all disciplines of the two largest universities of the French-speaking Community of Belgium (N = 1509). Specifically, we focused on several factors: gender, nationality, marital status, master grade, whether students continued at the same university when transitioning to the doctoral degree, whether they continued in the same field, age at registration, research field and funding (i.e., type of funding and associated job requirements). Findings indicate that four factors (marital status, master grade, research field and funding) are directly associated with dropout rate when all factors are considered jointly in the same model. Furthermore, results indicate that some of these factors, such as the marital status and gender, interact. In addition, we found that an accumulation of risk factors leads to a massive increase in dropout rates. Finally, a time course analysis revealed that the highest dropout rate occurs during the first two years and is related to the absence of funding or scholarship. The results, limits and futures perspectives are discussed.
Article
There is interest in admission criteria that predict future success in biomedical graduate school programs, but identifying predictors of PhD attainment is inherently complex. In particular, high noncompletion rates of PhD programs have long been recognized as a major crisis. Here, we present a quantitative analysis of the PhD students enrolled in the Department of Biochemistry and Biophysics at Texas A&M University between 1980 and 2010. The input variables included sex, country of citizenship, undergraduate grade point average (GPA), and Graduate Record Examination (GRE) scores (Verbal and Quantitative Reasoning). Only GPA was a significant predictor of PhD completion based on logistic regression. We also examined associations involving nonbinary measures of success (PhD duration, first author, and total number of publications) among