Content uploaded by Martin Ebner
Author content
All content in this area was uploaded by Martin Ebner on Jan 15, 2018
Content may be subject to copyright.
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
Confidence in and beliefs about first-year engineering student success
Case study from KU Leuven, TU Delft, and TU Graz
T. De Laet
Head Tutorial Services Engineering Science
Leuven Engineering and Science Education Center, KU Leuven
Leuven, Belgium
E-mail: tinne.delaet@kuleuven.be
T. Broos
PhD student
Leuven Engineering and Science Education Center, KU Leuven
Leuven, Belgium
E-mail: wim.schepers@kuleuven.be
J.P. van Staalduinen
Project Manager Research & Online Labs
TU Delft Online Learning
Delft, Netherlands
E-mail: J.P.vanStaalduinen@tudelft.nl
M. Ebner
Head of Lehr und Lerntechnologien
Technische Universität Graz
Graz, Austria
E-mail: martin.ebner@tugraz.at
G. Langie
Vicedean Engineering Technology
Leuven Engineering and Science Education Center, KU Leuven
Leuven, Belgium
E-mail: greet.langie@.kuleuven.be
C. Van Soom
Head Tutorial Services Science
Leuven Engineering and Science Education Center, KU Leuven
Leuven, Belgium
E-mail: carrolien.vansoom@kuleuven.be
W. Schepers
Study career counsellor, project collaborator
Tutorial Services Engineering Science, KU Leuven
Leuven, Belgium
E-mail: wim.schepers@kuleuven.be
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
ABSTRACT
This paper explores the confidence freshman engineering students have in being
successful in the first study year and which study-related behaviour they believe to be
important to this end. Additionally, this paper studies which feedback these students
would like to receive and compares it with the experiences of second-year students
regarding feedback. To this end, two questionnaires were administered: one with
freshman engineering students to measure their expectations regarding study success
and expected feedback and one with second-year engineering students to evaluate
their first year feedback experience.
The results show that starting first-year engineering students are confident regarding
their study success. This confidence is however higher than the observed first-year
students success. Not surprisingly, first-year students have good intentions and
believe that most academic activities are important for student success. When second-
year students look back on their first year, their beliefs in the importance of these
activities have strongly decreased, especially regarding the importance of preparing
classes and following communication through email and the virtual learning
environment. First-year students expect feedback regarding their academic
performance and engagement. They expect that this feedback primarily focuses on the
impact on their future study pathway rather than on comparison to peer students.
Second-year students indicate that the amount of feedback they receive could be
improved, but agree with the first-year students that comparative feedback is less
important.
Conference Key Areas: Engineering Education Research, Attractiveness of
Engineering Education, Gender and Diversity
Keywords: academic self-confidence, feedback, reasons for students success, student
beliefs
1 GENERAL
The transition from secondary to higher education is challenging both from the
academic and social perspective [1]. Students have to adapt their study and learning
strategies to the new context of higher education, but , a priori it is often not clear for
students how and to what extend they have to adapt. The social-comparison theory [2]
states that people evaluate their abilities through comparison to others when they are
lacking objective means of comparison. As students enter a new social group when
starting in higher education, they lack a comparison framework, which induces
uncertainty about their abilities.
This paper explores the self-reported confidence of starting first-year engineering
students in being successful in the first study year of higher education. Additionally,
the paper explores what students belief to be important study-related behaviours to
obtain study success. Finally, as feedback is considered a powerful tool for improving
student achievement [3], the paper explores what feedback students would like to
receive during their first year in higher education.
Self-efficacy, or the expectation to be successful for a specific task, can be considered
as a situation-specific self-confidence [4], as such academic self-confidence can be
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
viewed as self-efficacy in an academic context [5]. A person’s self-confidence in the
context of academic achievement is different from general self-confidence.
Earlier research showed that self-efficacy is significantly related to academic
performance [6] but that the effect depends on the timing of the self-efficacy
measurement [7] and might only be important for students with high intelligence [7].
The impact of students’ academic self-confidence on student performance also exists
for engineering students [5].
Feedback has been a proven powerful tool for improving student achievement, but its
effectiveness depends on the type of feedback and the circumstances under which
feedback is given [3]. During the transition from secondary to higher education this
feedback is considered pivotal regarding student motivation, confidence, retention, and
success [8], [9].
2 DATA COLLECTION AND METHODOLOGY
2.1 Data collection
In the academic year 2016-2017 data was collected from first-year engineering
science students at three higher education institutes: KU Leuven, Delft University of
Technology (TU Delft) and Graz (TU Graz). The survey contained 16 questions
regarding confidence in and beliefs about first-year student success, expectations
regarding feedback, and experiences regarding feedback in the transition from
secondary to higher education. At KU Leuven a paper-and-pencil questionnaire (in
Dutch) was administered in the first weeks of the academic year. TU Delft performed
an online questionnaire (in Dutch) in the first weeks of the academic year, in which four
of the 16 questions of the survey were included. TU Graz organized a paper-and-pencil
questionnaire (in German) during the welcome days within the first study week.
2,127 students completed the questionnaire (KU Leuven n=409 from Engineering
Science bachelor, TU Delft n=777 from 12 Bachelor of Science programs, all with a
technological focus (e.g. Computer Science, Mathematics, etc.,) TU Graz n=941 from
all Engineering Science bachelors).
In the academic year of 2016-2017 data was collected from second-year engineering
science students at the KU Leuven using a Dutch paper-and-pencil questionnaire
during a lecture in the first week of the academic year. The survey contained 41
questions regarding the activities and behaviours students believed to be important for
first-year student success, and their experiences regarding feedback in the first year.
271 students completed the questionnaire.
The two questionnaires used a five-point Likert scale ranging from not at all typical,
over not typical, somewhat typical, typical, to very typical.
2.2 Methodology
This paper uses descriptive statistics visualized using diverging stacked bar charts to
present the survey results [10]. To assess the significance of the differences between
the different institutes a two-step analysis was performed. First, a Kruskal-Wallis rank
sum test was used to test the overall significance of differences between groups.
Second, a pairwise Wilcoxon test was used to test the significance between different
groups. For the pairwise Wilcoxon test a correction was used to accommodate for
multiple testing [11].
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
3 RESULTS
3.1 Confidence in and beliefs about first-year student success
On the question "I feel very confident to successfully complete the first year.", 71% of
the first-year students reply this is typical or very typical for them, while 7% of the
students indicate this is not typical or not at all typical for them.
Fig. 1 shows the responses for the different institutes. The Kruskal-Wallis test indicated
statistically significant differences between the responses on the survey question by
the students in the different institutes (chi-squared=477.81, df=2, p-value < 2.2e-16).
The pairwise Wilcoxon-test showed significant differences between all three institutes.
The survey questions regarding the reasons for success was formulated as a main
question "To be successful in the first year it is important that I will" with four
subquestions: "study hard, attend classes, prepare classes, and meticulously follow
the communication of teachers and faculty staff through email and the virtual learning
environment.". Overall 93% of the students believe that studying hard is important for
study success. For attending classes this is 91%, for preparing classes this is 80%,
and for meticulously following the communication of teachers and faculty staff through
email and the virtual learning environment this drops to 50%.
Fig. 1. Stacked boxplot for first-year student responses on "I feel very confident to
successfully complete the first year." for different institutes.
The Kruskal-Wallis test indicated statistically significant differences between the
responses in the different institutes on all above subquestions (Table 1).
Table 1. Table showing the statistically significant differences between institutes for
on the first-year student responses regarding the question for reasons for study
success. Institutes sharing a letter are not significantly different (Wilcoxon test, p-
value threshold 0.05).
To be successful in the first year it is important
that I will …
KU Leuven
TU Delft
TU Graz
Kruskal-Wallis
study hard
a
a
b
𝜒2=17.9, df=2,
p=1.31e-4
attend classes
a
b
c
𝜒2=264, df=2,
p < 2.2e-16
prepare classes
a
b
a
𝜒2=331, df=2,
p < 2.2e-16
meticulously follow the communication of teachers and
faculty staff through email and the virtual learning
environment.
a
-
b
𝜒2=81.6, df=1,
p < 2.2e-16
For all questioned reasons regarding activities underlying first-year success the
differences between the institutes are significant. KU Leuven and TU Delft students
believe "studying hard" is more essential than TU Graz students. KU Leuven students
agree most that attending class is important for study success, with the TU Delft
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
students on the other end. KU Leuven and TU Graz students believe preparing classes
is more important for study success than TU Delft students. Finally, KU Leuven
students believe that following the communication through email and the virtual
learning environment is more important for study success than TU Graz students (TU
Delft students did not receive this question).
The survey questions regarding the activities and behaviours second-year students
believe to be important for first-year student success was formulated as a main
question "To be successful in the first year it was important in my opinion to " with
seven subquestions: “use good study techniques, study hard, attend classes, prepare
classes, meticulously follow student email, regularly checking the virtual learning
environment for new messages and content, and feel well in my studies (academic
well-being)”. Overall students agreed most that using good study techniques (82%),
feeling well in the studies (76%) and studying hard (76%) was important (Fig. 2).
Attending classes (45%) and regularly checking the virtual learning environment (38%)
was considered important by less than half of the students. Preparing classes (22%)
and meticulously following student email (15%) were considered least important.
Fig. 2. Stacked boxplot for second-year student responses on possible reasons for
first-year student success.
Discussion
Overall starting first-year engineering students are confident regarding their first-year
student success. However, large differences exist between KU Leuven, TU Delft, and
TU Graz. While cultural differences definitely impacts students’ answers, the different
university contexts including admission requirements also impact student confidence.
Surprisingly this confidence does not correlate with the percentage of successful
students in the first year (drop-out bachelor around: KU Leuven: 42%, TU Delft 24%,
TU Graz 60%). A small elaboration is needed to explain the higher confidence of TU
Graz students, despite the high drop-out rate. 40% of the beginners at TU Graz are
coming from dedicated colleges providing them with a prior knowledge in the specific
field of studies. While these beginners have strong domain knowledge, experience
shows that they are often not successful in higher education as the difference between
secondary and higher education is quite big and is often underestimated by those
beginners. Another effect is that a lot of Austrian students are recruited by companies
before obtaining a bachelor degree, so these students are dropping out, while they
might have been successful.
Not surprisingly, first-year students initially have good intentions and believe that most
academic activities are important for first-year student success (while some initial
doubt is already observed concerning the importance of preparing classes and
meticulously following communication). When student look back on their first year their
beliefs in the importance of these activities and behaviours have strongly decreased.
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
While studying hard is still considered important by most students, attending and
preparing classes and meticulously following communications are not considered
essential.
3.2 Feedback
When asked which feedback first-year students would like to receive during the first
year 87% agrees they want to receive feedback on their academic performance and
69% agrees with feedback concerning their academic engagement and activities (Fig.
3). Very few students respond negative on these questions (3% and 7%). 79% of the
first-year students want to receive feedback showing the expected impact of their
current academic performance and behaviour on their future study pathway or study
success (Fig. 4). The demand for comparative feedback (position with respect to fellow
students) is lower (58%) and more student do not agree they want to receive such
comparative feedback (19%).
Fig. 3. Stacked boxplot for first-year student responses on expectations regarding
topics of first-year feedback.
Fig. 4. Stacked boxplot for first-year student responses on expectations regarding
content of first-year feedback.
When second-year students are asked whether they received sufficient feedback in
their first year, 61% agrees for feedback regarding academic achievement, while only
38% agrees for feedback on study efforts, and even less 20% regarding feedback on
academic well-being (Fig. 5).
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
Fig. 5. Stacked boxplot for second-year student responses on experiences regarding
content of first-year feedback.
When asked about the format of feedback 55% of second-year students agrees it
showed the possible impact of their activities and achievements on their future study
pathway way, few students (23%) agree it helped them to position themselves with
respect to peer students (Fig. 6). On a positive note, students find the feedback useful
(57%) and indicate that it made clear what was expected from them (54%), that it
helped to make study-related decisions (44%), or even made them reflect on their
studies (38%) or adapt their study behaviour (31%).
Fig. 6. Stacked boxplot for second-year student responses on experiences regarding
format and effect of first-year feedback.
Discussion
A big difference exists regarding first-year student expectations on feedback and
experiences of second-year students regarding the feedback they received during the
first year. While 69% of the first-year students explicitly agreed they would like to
receive feedback on their academic engagement only 38% of the second-year
students agreed they received sufficient feedback on this topic. Similarly 79% of first-
year students expect feedback on the impact on their future study pathway while 54%
of second-year students agrees to have received such feedback sufficiently. Only 23%
of the second-year students indicate that they could use the feedback to compare to
peers (23%), but this was also believed to be less important by first-year students
(58%).
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
4 CONCLUSION AND FUTURE WORK
The results show that starting first-year engineering students are confident regarding
their success. This confidence is however higher than the observed first-year students
success. Not surprisingly, the students have good intentions and believe that most
academic activities are important for student success. When second-year student look
back on their first year their beliefs in the importance of these activities and behaviours
have strongly decreased, especially concerning the importance of preparing classes
and following communication through email and the virtual learning environment. First-
year students are expecting feedback regarding their academic performance and
engagement. They expect that this feedback primarily focuses on the impact on their
future study pathway rather than on comparison to peer students. Second-year
students indicate that the amount of feedback could be improved, but agree with the
first-year students that comparative feedback is less important.
With regards to future work, first of all, the results will be used to improve and shape
feedback towards first-year students. Future feedback should rather focus on the
impact on study pathway than on comparison to peers. Second-year students indicate
the importance of academic well-being for student success. Therefore, this is another
point of focus of feedback during the first-year. The results show that big differences
exist between institutes, therefore feedback should always take into account the local
context. As future work, the influence of gender and other background variables on the
first-year experience will be studied. Finally, the obtained results will be used as a
baseline measurement for studying the impact of newly designed feedback.
5 ACKNOWLEDGMENTS
This research is co-funded by the Erasmus+ program of the European Union
(562167-EPP-1-2015-1-BE-EPPKA3-PI-FORWARD).
REFERENCES
1. Tinto V. Leaving College : Rethinking the Causes and Cures of Student Attrition.
2. Festinger L. A Theory of Social Comparison Processes. Hum Relations.
1954;7(2):117-140. doi:10.1177/001872675400700202.
3. Hattie J, Timperley H. The Power of Feedback. Rev Educ Res. 2007;77(1):81-
112. doi:10.3102/003465430298487.
4. Bandura A. Social Foundations of Thought and Action: A Social Cognitive
Theory. Vol 1.; 1986.
5. Alias M, Aini N, Hafir HM. Alias and Mohd Hafir Academic self-confidence and
cognitive performance The relationship between academic self-confidence and
cognitive performance among engineering students. In: Proceedings of the
Research in Engineering Education Symposium. ; 2009.
http://rees2009.pbworks.com/f/rees2009_submission_82.pdf. Accessed April
20, 2017.
6. Wood RE, Locke EA. The Relation of Self-Efficacy and Grade Goals to
Academic Performance. Educ Psychol Meas. 1987;47(4):1013-1024.
doi:10.1177/0013164487474017.
7. Vrugt AJ, Langereis MP, Hoogstraten J. Academic Self-Efficacy and Malleability
45th SEFI Conference, 18-21 September 2017, Azores, Portugal
of Relevant Capabilities as Predictors of Exam Performance. J Exp Educ.
1997;66(1):61-72. doi:10.1080/00220979709601395.
8. Nicol D. Assessment for learner self‐regulation: enhancing achievement in the
first year using learning technologies. Assess Eval High Educ. 2009;34(3):335-
352. doi:10.1080/02602930802255139.
9. Lisa O, Morag M, Margaret P, et al. Technology- Enabled Feedback in the First
Year: A Synthesis of the Literature.; 2016.
http://www.teachingandlearning.ie/wp-
content/uploads/2016/10/SynthesisoftheLiterature2016.pdf. Accessed April 20,
2017.
10. Heiberger RM, Robbins NB. Design of Diverging Stacked Bar Charts for Likert
Scales and Other Applications. J Stat Softw. 2014;57(5):1-32.
doi:10.18637/jss.v057.i05.
11. Benjamini Y, Hochberg Y. Controlling the False Discovery Rate: A Practical and
Powerful Approach to Multiple Testing. J R Stat Soc Ser B. 1995;57(1):289-300.
http://www.jstor.org/stable/2346101.
