Conference PaperPDF Available

Abstract and Figures

Many universities and vocational training institutions conduct laboratories as simulated experiments. This is due to the costs and supervision needs to conduct hands-on labs safely. Numerous studies have presented mixed opinions on whether hands-on laboratory work is more conducive to learning than a simulated laboratory. Most of the studies put students from experimental and control groups in significantly different conditions. Therefore, it is hard to reach any definite conclusion regarding the influence of the learning mode onto the learning achievements. PURPOSE This study compares learning outcomes of student laboratory work in an energy storages course conducted in two different modes: first as a practical hands-on exercise and second using computer-based simulations. APPROACH In order to provide reliable insights, this study implements optimized research methodology to avoid any other effect (e.g. learning synchronicity/distance learning/instructions) on the learning outcome rather than the effect of the learning mode itself. The student laboratory experiments were created in a manner that they could be conducted in both modes in the same way and using a single set of instructions. To ensure a comparable group environment for the individual student, the students were arranged into two similar groups based on the student's practical experience. In this crossover study, the groups were taught the same topics by means of interchanging learning modes. RESULTS To evaluate the influence of each mode on student learning, short written tests regarding the previous experiment were conducted at the beginning of the subsequent laboratory session. 102 students have taken part in the study in two years. Overall learning results of hands-on experiments were slightly better than those of simulated laboratories (Cohen's d=0.25), the difference in performance was statistically significant (p<0.02). Through solicited feedback on each laboratory session, in hands-on mode more students expressed they have acquired new insights/comprehensions (76% vs. 66%, Cohen's d=0.23, small effect, p<0.07). CONCLUSIONS Following the strategy not to optimize the lessons individually to the learning mode, other influences on the learning outcome, which were usually mixed, were excluded. The students' subjective opinions show advantages of the hands-on mode. Based on the objective data, a weak, but significant outcome to better knowledge acquisition with hands-on laboratory experiments was achieved. This observation is against the trend of the literature in the last years towards better or equal learning with nontraditional labs. Some of the excluded factors might have a stronger influence on student learning than estimated previously. To get a clear view, the authors recommend isolated research.
Content may be subject to copyright.
AAEE2017 CONFERENCE
Manly, Sydney, Australia
This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit
http://creativecommons.org/licenses/by/4.0/
1
Laboratory Learning:
Hands-on versus Simulated Experiments
Fabian Stegera,b, Alexander Nitscheb, Cayler Mileyc,
Hans-Georg Schweigerb, and Iouri Belskia.
School of Engineering, RMIT, Melbourne, Australiaa,
Faculty of Electrical Engineering and Computer Science, THI, Ingolstadt, Germanyb
Faculty of Computer Science and Engineering, UNR, Reno, USAc
Corresponding Author Email: fabian.steger@thi.de
SESSION C1: Integration of theory and practice in the learning and teaching process
CONTEXT Many universities and vocational training institutions conduct laboratories as
simulated experiments. This is due to the costs and supervision needs to conduct hands-on
labs safely. Numerous studies have presented mixed opinions on whether hands-on
laboratory work is more conducive to learning than a simulated laboratory. Most of the
studies put students from experimental and control groups in significantly different conditions.
Therefore, it is hard to reach any definite conclusion regarding the influence of the learning
mode onto the learning achievements.
PURPOSE This study compares learning outcomes of student laboratory work in an energy
storages course conducted in two different modes: first as a practical hands-on exercise and
second using computer-based simulations.
APPROACH In order to provide reliable insights, this study implements optimized research
methodology to avoid any other effect (e.g. learning synchronicity/distance
learning/instructions) on the learning outcome rather than the effect of the learning mode
itself. The student laboratory experiments were created in a manner that they could be
conducted in both modes in the same way and using a single set of instructions. To ensure a
comparable group environment for the individual student, the students were arranged into
two similar groups based on the student's practical experience. In this crossover study, the
groups were taught the same topics by means of interchanging learning modes.
RESULTS To evaluate the influence of each mode on student learning, short written tests
regarding the previous experiment were conducted at the beginning of the subsequent
laboratory session. 102 students have taken part in the study in two years. Overall learning
results of hands-on experiments were slightly better than those of simulated laboratories
(Cohen's d=0.25), the difference in performance was statistically significant (p<0.02).
Through solicited feedback on each laboratory session, in hands-on mode more students
expressed they have acquired new insights/comprehensions (76% vs. 66%, Cohen's d=0.23,
small effect, p<0.07).
CONCLUSIONS Following the strategy not to optimize the lessons individually to the
learning mode, other influences on the learning outcome, which were usually mixed, were
excluded. The students' subjective opinions show advantages of the hands-on mode. Based
on the objective data, a weak, but significant outcome to better knowledge acquisition with
hands-on laboratory experiments was achieved. This observation is against the trend of the
literature in the last years towards better or equal learning with nontraditional labs. Some of
the excluded factors might have a stronger influence on student learning than estimated
previously. To get a clear view, the authors recommend isolated research.
KEYWORDS Hands-on vs. simulated experiment, battery experiment, learning-mode
comparison
AAEE-2017_Full_Paper_Submission_174 940
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 2
Introduction
Many universities and vocational training institutions conduct laboratories as simulated
experiments instead of using the traditional hands-on way (Ma 2006, Brinson 2015, Heradio
2016). One reason is the cost of required lab equipment to conduct hands-on experiments. In
case of potentially dangerous learning objects like lithium-ion cells (Dahn 2011)
additional supervision is necessary to conduct traditional labs safely.
This study compares learning outcomes of student laboratory work conducted in two different
modes: first as a practical hands-on exercise and second using computer-based simulations.
Previous studies have presented conflicting outcomes (Ma 2006, Brinson 2015, Heradio
2016, e.g. Mathiowetz 2016, Sarabando 2016). After comparing the results of such research,
Ma and Nickerson (2006) concluded that many studies did not allow to reach universally
applicable conclusions.
Most of the studies put students from experimental and control groups in significantly
different conditions. This is a result of separately developing and optimizing the teaching
experiment in each mode, as the teachers/researchers do not see the modes as directly
competing solutions for exactly the same objective, they prefer to accommodate different
circumstances with both modes. For example, students conduct hands-on experiments in
groups whilst on campus, but as off-shore students they are engaged in simulated exercises
over the web and individually. Therefore, it is hard to reach a definite conclusion regarding
the influence of the learning mode onto the learning achievements based on this kind of
studies. As the change of modes is linked with other influences on the learning outcome (like
changing learning objectives in the two modes, amount/type of supervision, cooperative
learning effects, distance learning vs. learning in the university, and/or differing instructional
papers) such studies always compare the combination of aspects. If these interfering aspects
have stronger influences, the studies are unable to specifically identify the difference in
learning effectiveness of one aspect (Lindsay 2005). This may explain the inconsistent
outcome of present research (Ma 2006, Brinson 2015). Most of these other influences are
difficult to describe clearly in written form, or just not mentioned in the publications. In order
to provide more reliable insights, the present study follows an optimized research
methodology to avoid many other influences on the learning outcome while comparing the
modes. The student experiments were strictly developed in such a manner that the learning
objectives and experimental procedure were matching in both modes.
Approach
In a crossover study, both groups were taught the same objectives, clustered into four
content areas. All of the content areas were related to lithium-ion cells or battery systems.
The first group learned the first and third area with traditional hands-on experiments while the
second group was taught the same areas by computer-based simulations. The second and
fourth areas were taught using the opposite learning mode in both groups.
In the beginning of the next laboratory session 10-minute long tests were conducted to
evaluate the influence of the mode on student’s knowledge on the content areas of the
previous experiment. To compare which mode has been more successful, the mean results
for each group were evaluated. Moreover, the students were asked to reflect on their
learning during each session in an online survey.
The experiment was conducted three times. Forty students of the study program "Electrical
Engineering and Electric Mobility" at Technische Hochschule Ingolstadt (THI) were enrolled
in the mandatory laboratory subject in summer-semester 2016. Thirty THI students were
enrolled in summer-semester 2017. All students were asked to join the full anonymous study.
Additionally an international summer school was included in the study to collect a broader
database.
AAEE-2017_Full_Paper_Submission_174 941
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 3
Learning objectives of the laboratory
It was decided to teach the most relevant transferable skills and knowledge that may be
beneficial for the students' future careers. Thus, students should get familiar with the
characteristic behavior and the most important parameters of battery cells. They should also
gain knowledge on how to determine the parameters of battery cells self-sufficiently by
means of appropriate experimental setups.
The recognized learning objectives where clustered into four main content areas: (A) contact
and isolation resistance, (B) open-circuit voltage, (C) internal resistance and power, and (D)
energy of cells. Grouped in these areas, seven laboratory experiments were conducted in
both modes in the same manner:
Low Resistance Measurements (A1): Students discover that a multimeter is an
inaccurate tool for low ohmic measurements (milliohm range), and why such
measurement is a misuse of this tool. They learn how to use alternative procedures
for low ohmic measurements, including the four-wire measurement in AC and DC.
Contact Resistance (A2): Students conduct experiments with a variety of typical
electrical connections in battery systems to determine the contact resistance values.
Isolation Resistance (A3): Students learn to estimate the influence of moisture on the
isolation resistance.
Voltage Curve (B): Students investigate the voltage of a cell depending on the state of
charge. They use two different types of lithium-ion cells.
Internal Resistance (C1): Students learn to use AC- and DC-methods to measure
internal resistances, being aware of the temperature dependency of battery cells.
Students learn to approximate temperature changes caused by power loss inside a
cell.
Power (C2): Students investigate the maximum discharge rate of battery cells.
Students discover the dependency of maximum discharge power from state of
charge, pulse duration, and temperature.
Energy and Capacity (D): Students determine the capacity of a lithium-ion cell and
learn about the factors influencing it. They learn to calculate the energy efficiency of
charge and discharge cycles.
Each experiment was developed in parallel for both modes. Only a single set of instructions
was created and then used in both laboratory modes, as instructions affect the learning
outcome of an experiment, e.g. Chamberlain 2014 found, that the guidance level can
influence student exploration.
Creating two comparable groups for the crossover study
In a crossover study, differences of the compared groups' average performances may be
equalized by statistics. Nevertheless, with the goal of isolating the influence of the learning
mode in the experiment, the authors had to consider the in-group interaction in laboratories.
The same student may have different experiences in different groups, with consequent
effects on his or her learning (van der Laan Smith 2007, Webb 1989). It was assumed that
students with more practical experience might perform differently than their peers with a
lesser practical background. Therefore the authors tried to create each of the two laboratory
groups from students with a similar mix of practical skills. For the two study semesters of
2016 and 2017, an introductory questionnaire that asked students of their practical
experience was developed. Its results were used for the group allocations. While completing
the questionnaire, each student created a code-word that was used in the study to allocate
the responses and test results to the appropriate individual, while keeping all participants
anonymous. These code words were also used to assign students into laboratory groups.
(Steger 2016)
AAEE-2017_Full_Paper_Submission_174 942
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 4
Thirty-two students from the international summer school were allocated to groups to ensure
similar distribution of the field of studies, number of study semester, and the nationality of the
students in each group.
Conducting laboratories in content areas A to D
For each of the four main content areas a session with practical hands-on experiments and a
session with computer-based simulations were established. Planned as a crossover study,
each group followed the same sequence of contents, while alternating the learning mode
between sessions. The first group handled the first content area with a computer-based
simulation while the second group conducted it as a hands-on experiment. For the upcoming
session both groups switched between learning modes. For the first content area, a specially
created web tool was used for simulations. For the other areas a black box simulation of the
battery cell and the hands-on equipment (Steger 2017a) was used. The battery simulation
imitates all effects observed in the student experiments and was parameterized to match the
battery cells which are used in the hands-on mode. To avoid any influence from user’s
interface, in both modes the simulation model was controlled by the same graphical user
interface, a program to create the sequences for testing batteries. In both groups students
selected companions to work in small teams of three to five students on the experiments. All
groups and teams remained constant to avoid changing cooperative learning while the
conduction of the semester lasting educational experiment. The laboratories were developed
in a way to be conducted autonomously by the students in a supervised environment. All the
learning targets were addressed through the experiment procedure/instructions, without
essential explanations from the instructor. To pass the subject all learning teams had to
prepare a written laboratory report for each content area.
During international summer school two trimmed content areas were taught on the same
day; the same crossover principle was used as during ordinary semesters in 2016 and 2017.
Online survey after conducting the experiments
All student participants excluding that of the international summer school were asked to
express opinions on their laboratory learning in a short online survey. It was conducted fully
anonymously using the university’s digital learning environment. Responses to the following
survey questions were compared in order to evaluate the laboratory learning in both learning
modes:
(a) "By conducting the experiment I gained new insights/comprehensions today." (German:
"Ich habe heute durch den Versuch neue Erkenntnisse gewonnen.") This was a yes/no
answer and coded 1 or 0.
(b) "At which point in the experiment did you have the biggest problem proceeding with the
experiment?" (German: "An welcher Stelle im Versuch hatten Sie am meisten Probleme
voranzukommen?") This was a free text question, which was not compulsory. For data
evaluation, the information was coded to 1 if any problem was mentioned or to 0 if students
wrote nothing or were expressing they had no problems.
(c) "The procedure of the experiment is quite difficult (1) / feasible (0.5) / easy (0)" (German:
"Die Versuchsdurchführung ist recht schwer / machbar / leicht") The answers were coded in
a three step Likert scale.
(d) "The content of the experiment is also relevant for me outside the university; I can
imagine that it will be beneficial for my future professional life." (German: "Der Inhalt des
Versuchs hat auch außerhalb der TH Relevanz für mich; ich kann mir vorstellen, im
Berufsleben Gewinn aus dieser Versuchsdurchführung zu ziehen.") The answers were coded
in a five step Likert scale: fully agree (1) / somewhat agree (0.75) / maybe (0.50) / somewhat
disagree (0.25) / disagree (0)
AAEE-2017_Full_Paper_Submission_174 943
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 5
Testing the learning outcome
Three written knowledge tests - each lasting 10 minutes - were used to determine the
influence of the learning mode on student learning outcomes. Knowledge gained by students
in a laboratory was tested just prior to the next laboratory session. A mix of descriptive and
multiple-choice questions, free response, and drawings on the learning objectives of the
previous experiment were used. The results were evaluated using a positive point system.
The achievable number and the distribution of the points were set fixed beforehand.
As the tests were conducted anonymously, the students were distinguished by their self-
created code words that were created by individual students during the Questionnaire on
practical experience.
To exclude time influence on memorizing knowledge the authors aimed for equal time lapses
between two sessions and thereby between experiments and its corresponding tests for both
compared groups. Unfortunately this goal was not achieved regarding content area A in 2016
(Steger 2016). By conducting the sessions for both groups on the same day of the week it
was easier to keep the time gap between the experiments and tests equal for both learning
modes in 2017.
Independent of the learning mode, the same environment was established for both groups
during tests. All students were able to sit at a desk in the same computer laboratory while
working on the knowledge tests.
To find out which mode provides better learning outcome, average results for the test for
each group were compared between hands-on and simulated modes.
Assessment of the influence of the study mode on learning outcomes of students that
participated in the international summer school was similar to that of the students of peers
that conducted laboratories during study semester. Due to logistical constrains, though, tests
on the laboratory knowledge were administered to students as an examination one week
after the experiments.
Results
Learning outcomes based on written tests
In year 2016 with 40 students for three content areas (A to C) a weak effect towards benefits
of the hands-on mode was discovered. Hands-on laboratory sessions led to a better
knowledge acquisition compared to simulated experiments. Content area D showed no
difference between the modes. Overall learning results of hands-on experiments were
slightly better than those of simulated laboratories (weak effect, Cohen's d=0.22), but the
difference in performance was not statistically significant. (Steger 2016)
The second experimental run was conducted with 30 students in summer semester 2017.
The range of individual scores was from 20% to 79% for hands-on, and from 15% to 77% for
the simulated mode. For content areas A (d=0.50) and D (d=0.80, p<0.02) an effect towards
benefits of the hands-on mode was measured. Content area C resulted in a weak effect
(d=0.14) in the same direction, and content area B showed no difference between the
modes. This run was demonstrating hands-on laboratory sessions led to a significant better
knowledge acquisition compared to simulated experiments (d=0.34, p<0.05).
Additionally the learning outcome of 32 participants of the international summer school 2017
in Ingolstadt was evaluated. A short version of content areas B and C1 was taught.
Independent samples T-test showed that content area B had no effect (d=-0.09), and content
area C1 demonstrated a small to medium effect towards better learning in hands-on mode
(d=0.44).
Overall learning results of hands-on experiments were slightly better than those of simulated
laboratories (small effect, Cohen's d=0.25), the difference in performance was statistically
AAEE-2017_Full_Paper_Submission_174 944
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 6
significant (p<0.02). The correlation between an individual student’s overall performance in
the knowledge tests and learning better with simulations is very weak and insignificant
(Pearson correlation 0.12, p=0.37, N=53).
Table 1: Learning Results Comparison
Research run
Number of
Students
Std. Deviation
of Percentage
Points
Sample
size N
Mean
Value of
Points
Sample
size N
Mean
Value of
effect size /
Cohens d
(pos = adv. of
hands-on)
Student's t
distribution /
p-value one-
tailed
Both modes
Hands-on
Simulated
THI 2016
40
17%
73
47%
75
0.22
0.09
THI 2017
30
20%
58
56%
57
0.34
0.03
THI SS 2017
32
24%
28
60%
28
0.15
0.28
ALL
102
20%
159
53%
160
0.25
0.014
In the 2017 research the pre-questionnaire was extended to ask for more detailed
information. There was no correlation found between average mark in the study program and
individual more effective learning mode (Spearman rho=0.03, N=24). Students who had not
studied directly after school, but made a German VET (Vocational Education and Training,
BMBF 2016), which is a dual (company & school) apprenticeship training and studied later
(1st) have more hands-on experience (Pearson r=0.50, p=0.008, N=27) and (2nd) learn better
with hands-on experiments compared to simulations (Pearson r=0.53, p=0.007, N=25) as
they (3rd) have comparatively bad results with simulated experiments (Pearson r=0.47,
p=0.017, N=25).
Online survey for student feedback
The student feedback results are based on all results of the first and second summer
semester iteration. Student feedback was not collected during the summer school due to
time constraints. Because fewer students responded to the questionnaire in 2016 (37%), an
incentive was offered for completing the questionnaire in 2017 (Steger 2017b). The minimum
passing score of 50% was reduced to 45% for participation in the online survey. As a result
the return rate increased to 70% in 2017.
(a) In hands-on mode more students expressed they have acquired new
insights/comprehensions (76% vs. 66%, Cohen's d=0.23, small effect, p<0.07).
(b) Slightly more students mentioned problems while conducting the hands-on equivalents
(45% vs. 39%, Cohen's d=0.12, not significant, very weak effect).
(c) The engagement in the simulated experiments was stated to be a very small amount (5%
of scale) more difficult. Cohen's d=-0.16 demonstrates a very weak effect, which was not
significant.
(d) In the feedback form on the experiments, students who conducted the experiments in the
hands-on mode rated the execution of the experiments a little more beneficial for their future
professional life (58% hands-on vs. 54% simulation, Cohen's d = 0.20, small effect).
A significant correlation between (a) and (d) was found. It shows that students who claimed
that they gained new insights also tend to believe that the execution of the experiment will
help them in their future professional life (Pearson's r=0.36, p<0.001, Spearman's rho=0.35).
Looking at both modes separately, this correlation in the simulation mode was little stronger
(Pearson's r=0.39, p<0.001; Spearman's rho=0.40; N=86) compared to the hands-on mode
(Pearson's r=0.30, p=0.115; Spearman's rho=0.27; N=92).
AAEE-2017_Full_Paper_Submission_174 945
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 7
Interesting is a significant weak correlation that exists only in hands-on mode: Students who
(c) stated a hands-on experiment difficult, tend to not (d) consider it beneficial for their future
professional life (Pearson's r=-0.23, p=0.017; Spearman's rho=-0.22; N=86). Regarding
simulated experiments Pearson's r and Spearman's rho is 0.04.
The actual methodology does not allow establishing correlations between individual learning
outcome and student's feedback, as the standard online feedback form did not ask for the
self-created code word that was used in the tests. The authors plan to request this missing
information by updating a questionnaire and use the information for the next iteration.
Conclusions
Learning outcomes based on written tests
Following the strategy not to optimize the lessons individually to the learning mode, other
influences on the learning outcome, which were usually mixed, were excluded. Based on the
existing data, a weak, but significant outcome of better knowledge acquisition with hands-on
laboratory experiments was achieved. This is against the trend of the recent literature that
reported on better or equal learning with nontraditional (virtual/simulated) labs (Heradio 2016,
Brinson 2015). Some of the excluded factors might have a stronger influence on student
learning than estimated previously. To get a clear view, the authors recommend isolated
research. The study on the mode will be continued through 2018 at THI and at more
universities and training institutions with different types of students (e.g. international
students, students enrolled in summer schools).
The average performance in the knowledge tests of a single student is independent from his
better performing learning mode. It is important to note that this is the result of the created
group environment, and may differ if one creates groups of high and low performers.
The difference between students that completed German Vocational Education and Training
before studies and those who enrolled at university directly after school was interesting.
Obviously, the VET-participants have more hands-on experience. This fact was confirmed by
the study. The group creation questionnaire distributed the VET-participants equally to both
groups based on the hands-on experience without using the VET-info. In 2017, 59% of the
participating students in the study were VET-participants. Looking at the correlational data,
this group learns better with hands-on experiments compared to simulations, as they have
significant disadvantages while taught by simulations. Checking the 2017 results of the
knowledge test separately, confirmed this point of view: VET-participants had a significant
better learning outcome with hands-on experiments (Pearson r=0.56, p<0.015, N=60 tests),
while non-VET-participants had no significant difference (N=37 tests).
Online survey for student feedback
The students' opinions show advantages of the hands-on mode. The effect between both
modes in the student's subjective opinion about gained knowledge (a) is very similar to the
objective results tests, even when student’s expressions are less significant. Regarding
mentioned problems (b) and stated difficulty (c) the experiments are considered equivalent.
The slight, insignificant differences may be a result of statistical effects. To clarify this, more
data collection is necessary. Asking the students to describe the laboratory difficulty in a free
text answer may help to gain a deeper insight. Independent from the learning mode, more
than forty percent of the students stated the opinion that they do not benefit in their future
professional life from the experiments (d). There are slight advantages in the hands-on mode
and it is planned to ask in future experiments for missing content students estimate as more
important for their future profession.
The correlation between (a) and (d) suggests that students who believed that the execution
of the experiment will help them in their future professional life also expressed that they
gained new comprehension. The slightly stronger correlation between (a) and (d) in the
AAEE-2017_Full_Paper_Submission_174 946
Proceedings, AAEE2017 Conference
Manly, Sydney, Australia 8
simulations mode suggests that teachers should pay attention to explain the relevance of the
experiment in simulated labs more carefully. Comparing the amount of positive answers in
(a) and (d), one can conclude that the students do not consider all of the gained insights
relevant for their profession. Identifying these insights may be beneficial for the improvement
of the experiments. Future surveys will ask for the greater or lesser beneficial insights gained
and whether these are considered useful outside the university.
References
BMBF, German Federal Ministry of Education and Research (2016). Report on Vocational Education
and Training 2016.
Brinson, J. R. (2015). Learning outcome achievement in non-traditional (virtual and remote) versus
traditional (hands-on) laboratories. A review of the empirical research. Computers & Education 87,
pp. 218237.
Chamberlain, J. M., Lancaster, K., Parson, R., Perkins K.K. (2014). How guidance affects student
engagement with an interactive simulation. Chemistry Edu. Res. & Practice 15(4), pp. 628-638.
Dahn, J. & Ehrlich, G. M. (2011). Lithium-Ion Batteries, Linden's Handbook of Batteries 4th Edition,
McGraw-Hill, pp. 26.1-26.79.
Heradio, R., de la Torre, L., Galan, D., Cabrerizo, F. J., Herrara-Viedma, E., Dormido, S. (2016).
Virtual and remote labs in education. A bibliometric analysis. Computers & Education 98, pp. 14
38.
Lindsay, E. D (2005). The Impact of Remote and Virtual Access to Hardware upon the Learning
Outcomes of Undergraduate Engineering Laboratory Classes. The University of Melbourne
Ma, J., Nickerson J. V. (2006). Hands-On, Simulated, and Remote Laboratories: A Comparative
Literature Review. ACM Computing Surveys 38(3), Article 7
Mathiowetz, V., Yu, C.-H., Quake-Rapp, C. (2016): Comparison of a gross anatomy laboratory to
online anatomy software for teaching anatomy. Anatomical sciences education 9(1), pp. 5259.
Sarabando, C., Cravino, J. P., Soares A. (2016): Improving Student Understanding of the Concepts of
Weight and Mass with a Computer Simulation. Journal of Baltic Science Education 15(1).
Steger, F., Nitsche, A., Schweiger, H.-G., Belski, I. (2016). Teaching Battery Basics in Laboratories:
Comparing Learning Outcomes of Hands-on Experiments and Computer-based Simulations, Paper
presented at the 27th Australasian Association for Engineering Education (AAEE) Annual
Conference, Coffs Harbour.
Steger, F., Nitsche, A., Brade, K., Belski, I., Schweiger, H.-G. (2017a). Teaching Energy Storages by
means of a Student Battery Cell Test System, Paper presented at the 45th European Society for
Engineering Education (SEFI) Annual Conference, Angra do Heroísmo.
Steger, F., Nitsche, A., Schweiger, H.-G., Belski, I. (2017b). Hands-on Experiments vs. Computer-
based Simulations in Energy Storage Laboratories, Paper presented at the 45th European Society
for Engineering Education (SEFI) Annual Conference, Angra do Heroísmo.
Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal of Educational
Research (13)1, pp. 21-39.
van der Laan Smith, J., Spindle, R. M. (2007): The Impact of Group Formation in a Cooperative
Learning Environment. In Journal of Accounting Education 25(4), pp. 153167.
Acknowledgements
This research was approved by the Faculty of Electrical Engineering and Computer Science
of TH Ingolstadt and the College Human Ethics Advisory Network of RMIT, Melbourne.
The project "Academic Education Initiative for E-Mobility Bavaria - Saxony" and the Faculty
of Electrical Engineering and Computer Science at THI provided funding which was used for
the development and the production of the laboratory devices.
This research was not possible without students, generously ready to take part in the study
to improve the learning outcome of future groups.
AAEE-2017_Full_Paper_Submission_174 947
... A positive correlation between the two variables was found [R2; N = 27; Spearman's ρ(25) = 0.50, p < 0.008]. Students who completed VET reported a significantly higher amount of practical experience than their colleagues [39]. ...
Article
Full-text available
Contribution: Prior studies comparing the effectiveness of different laboratory learning modes do not allow one to draw a universally valid conclusion, as other influences are mixed with the learning modes. In order to contribute to the existing body of work and to add another piece to the puzzle, this article demonstrates an improved methodology to evaluate the effectiveness of computer-simulated laboratories in comparison to hands-on exercises using a battery basics practical course as a case study. Background: Computer-simulated experiments are becoming increasingly popular for conducting laboratory exercises in higher education and vocational training institutions. To ensure the consistent quality of laboratory learning, an accurate comparison between the results of simulated experiments and practical hands-on experiments is required. Intended Outcomes: In this article, the achievement of the following learning objectives were compared between the two laboratory modes: 1) comprehension of the most important parameters of battery cells and 2) knowledge on how these parameters can be determined using adequate experimental procedures. Application Design: To avoid interference of factors other than laboratory mode on the learning, laboratory instructions and experimental interfaces ensured identical execution of the experiments in the compared modes. Using a counterbalanced methodology, the two laboratory modes alternated by the session, while the experimental procedures remained constant regardless of the respective modes. Findings: Tests taken by the participants after conducting the laboratory experiments revealed that hands-on laboratories resulted in statistically significantly better student performance than simulated laboratories. This difference was even more pronounced for the participants that finished a vocational education and training program before the university studies.
Thesis
Full-text available
45 Prozent - So hoch ist die Studienabbrecherquote an deutschen Universitäten in den Studiengängen der Elektrotechnik und Informatik. Die meisten Studierenden brechen dabei aufgrund von Leistungsproblemen und mangelnder Studienmotivation ab. In dieser Dissertation wird evaluiert, inwieweit eine Reduktion von Einflussfaktoren auf den Studienabbruch durch eine gezielte Aktivierung und Berücksichtigung individuell vorhandener Kompetenzen, Motivationen sowie Lernstrategien der Studierenden möglich ist. Die Aktivierung großer Kohorten stellt unter Berücksichtigung der diversen Bildungsbiographien und dem Aufbau der Hörsäle eine besondere Herausforderung da. Aus diesem Grund wird im Rahmen dieser Dissertation untersucht, wie sich eine große inhomogene Kohorte aktivieren lässt. Hierzu wird ein fachdidaktisches Konzept mit einem Design-Based Research Ansatz konzeptioniert, realisiert, erprobt sowie anhand von Gütekriterien evaluiert und optimiert. Den Ausgangspunkt einer Lern-Lehr-Situation bildet nach dem fachdidaktischen Konzept eine komplexe, realitätsnahe Problemstellung, welche sich auf ein Problemprodukt bezieht. Dieses Problemprodukt ist ein viabler Lerngegenstand und stammt aus dem späteren Professionsgebiet von Ingenieurinnen und Ingenieuren sowie Informatikerinnen und Informatikern. In einem mehrphasigen Problemlöseprozess informieren sich die Studierenden zu Hause über die Problemstellung, planen Lösungsoptionen und entscheiden sich zwischen diesen. In der Präsenzveranstaltung im Hörsaal werden die Lösungsoptionen diskutiert, kollaborativ implementiert und exemplarisch anhand des Problemproduktes kontrolliert sowie reflektiert. Aufgrund der großen Kohorte können nicht alle Problemlösungen im Hörsaal kontrolliert werden. Aus diesem Grund sind die Problemprodukte Bestandteil eines Remote Labs, sodass eine Kontrolle von zu Hause aus möglich ist. Bei erfolgreicher Teilnahme an Testaten zu den Remote Lab-Versuchen haben Studierende 2020 Bonuspunkte für die abschließende Prüfung erhalten. Das fachdidaktische Konzept ist im Jahr 2019 und unter Einschränkungen durch die Coronapandemie im Jahr 2020 in der Lehrveranstaltung Industrielle Steuerungstechnik und Echtzeitsysteme mit über 100 Studierenden erprobt sowie quantitativ als auch qualitativ evaluiert worden. Während die Mehrheit der Studierenden sich 2019 nicht intrinsisch motivieren ließen, die problemorientierten Aufgabenstellungen zu lösen, führten die Bonuspunkte 2020 zu einer persistenten Aktivierung der Studierenden. Das Interesse an lehrveranstaltungsnahen Professionen konnte in beiden und die Bewertung der eingesetzten Lehrmethode im letzten Feldversuch signifikant verbessert werden. Der entstandene Mehraufwand sowie der Leistungsdruck wurden von den Studierenden als sehr kritisch angesehen, rentierte sich jedoch durch signifikant bessere Prüfungsergebnisse und einemKompetenzzuwachs. Die Evaluationsergebnisse zeigen insgesamt, dass lernendenzenterierte aktivierende Lehre mit großen Kohorten möglich ist
Chapter
Vocational high school is a secondary education whose practice portion is larger than its theoretical portion. This allows students to do more hands-on practice in the laboratory, as skill competency is very important in vocational education. Through practice, students have the skills to become competent and skilled technicians in the future. When students practice in a hands-on laboratory, errors may occur that can injure students, equipment, and components. In addition, short circuits can also endanger student safety. Therefore, to improve practical skills in the laboratory, teachers must find innovative ways to incorporate these methods into the learning process. One of the things that can be done to improve students’ practical skills is to use simulation software before doing direct practice in the laboratory. In-depth interviews were conducted with three electrical engineering teachers to verify the perspective of the proposed model. The results suggest that the proposed design is likely to improve problem-solving skills when an error occurs during the simulation, and it will improve practical skills when using hands-on laboratories so that students learn more about hands-on lab practice in vocational high school.KeywordsHands-on laboratorySimulation softwarePractical skillsVocational students
Thesis
Full-text available
Background: Engineering courses often complement lectures with laboratory classes to optimise student learning outcomes and further develop valuable skills for future employment. Computer simulated experiments for conducting laboratory exercises have become increasingly popular in higher education and vocational training institutions to replace traditional hands-on laboratories. Reasons for this include for example, cost efficiency and repeatability. Research question: There has been a wide array of discussion on the efficacy of the two laboratory modes in teaching, both in general and for students in engineering fields (for example, chemical engineering or electrical engineering). However, many previous studies on this question did not reach a universally valid conclusion. The used methodologies mixed other influences with the impact of the investigated learning modes. These influences include for example accompanying lectures, experimental instructions, teachers, learning objectives, tests, working teams, and many more. Thus, the differences in results of these studies cannot be attributed to the laboratory mode only. The study conducted for this thesis investigated differences in learning outcomes of students in higher education when comparing two laboratory modes in the local domain: 1. In-person hands-on laboratories allow students to directly interact with the subject at hand, although this interaction might be mediated through technology or a user interface. 2. In-person simulated laboratories moderate all student interactions through a user interface. The properties of the investigated effect are simulated by computer software. The students work in a classroom equipped with computers on which the simulations are running. Since this study was focused solely on comparing different learning modes, all other aspects were held as constant as possible. Improvements that were theoretically possible in only one of the teaching methods (e.g. time-lapse in simulations) were not implemented in order to keep the surrounding conditions as equal as possible. Thus, the aim of the research was not to determine which of the investigated laboratory modes would be best for teaching a specific topic, but rather to investigate whether or not there are discernible differences in teaching success when conducting the same experiment in hands-on and simulated laboratories. The ultimate goal was to establish more reliable and generalisable insights into the influence of a particular laboratory mode on learning. The study did not include a remote laboratory condition; the comparison was only made between in-person laboratory teaching with proper laboratory equipment and simulations conducted in the local domain. An important note on demographics: a third of students at universities of applied sciences have completed apprenticeships in the German vocational education and training programs (VET) before enrolment. These VET programs mainly consist of practical on-the-job learning and aim to directly prepare apprentices for entering the job market. Due to the large size of this demographic and their previous experiences mostly with hands-on learning, it was of additional interest to see if VET-participants’ results differed significantly from those of their peers when confronted with the two laboratory modes. It was also of interest to see if the perception of the learning modes influenced the outcome. Methodology: This study was conducted in two consecutive phases on the example of a practical course teaching the basics of batteries (not related to physical manipulation of the batteries). A counterbalanced within-subject methodology was employed with German and international participants in nine study runs. The laboratory modes alternated, while the learning objectives and the experimental approach of laboratory exercises remained practically identical. In the first phase, the objective was to compare students’ learning success when working with hands-on laboratories and with overt computer simulations, respectively. The second phase was conceptualised to give insight into possible subjective influences of students’ perception of the two laboratory modes. In this phase, the simulation condition was hidden. Participants used hands-on equipment in both conditions. In the first condition, real measurements were shown; in the second condition, hands-on devices displayed simulated battery behaviour to investigate the influence of students’ perception. The participants were not aware of the differences in data sources. Besides the comparison of knowledge test results, questionnaires were employed to correlate prior, specifically technical, practical experience and previous apprenticeship training with the success of the knowledge transfer in both of the compared modes. Well-known personality tests were also employed in order to provide further insight into the subjects. The study collected subjective opinions regarding the laboratory modes in two ways: 1. Participants of the main study were asked to provide feedback after conducting a laboratory experiment. This method allowed for the indirect gathering of information about the difference in perception towards the two modes. 2. Persons who had either not yet started the laboratory or weren’t participating in the laboratory were asked to fill out a general questionnaire distributed amongst different universities in different countries. This method asked directly for subjective opinions regarding the learning modes. Finally, the THI university database was analysed to extract objective information about students with and without vocational training degree to gain broad background information about the compared groups. Outcomes: In the first phase, it was found that there were statistically significant differences in learning outcomes favouring the hands-on mode. When the simulation condition was overt, students with a background in vocational training before enrolment showed statistically significant trends towards better learning with hands-on experiments. Students in the international runs and Germans without a VET background performed similarly in both modes. In the second phase, when students were not aware that they were using simulations, both modes showed similar student learning across all student groups. Generally, simulations were reported as less relevant and their authenticity was called into question. A VET background seems to determine whether or not students had different levels of success in hands-on and simulated laboratories. As hidden differences in the simulations could be excluded from having been the reason for inferior learning results, psychological effects needed to be considered to comprehend the different laboratory modes’ effectiveness. The study outcomes lead to the conclusion that students’ personal perception of the laboratory modes, particular simulations, can have a significant impact on laboratory learning.
Conference Paper
Full-text available
Energy storage systems are vital for success of electric mobility. Thus, knowledge of energy storage systems behaviours is essential for the development of electric vehicles. Universities and vocational training institutions are urgently required to educate specialists in electrical storage systems. A safe and easy-manageable battery test system was developed to improve the outcomes of student learning of battery storage systems and to allow practice for future employment through hands-on laboratory training. This test system supports temperature-dependent experiments with different cell types including lithium-ion cells, and incorporates a redundant safety shut-off module that protects students from being injured. Based on this system, a new energy storage laboratory course was developed. This course covers four main content areas of battery energy storage: (1) contact & isolation resistance, (2) open-circuit voltage, (3) internal resistance & power, and (4) energy of cells. This new laboratory course was introduced in summer 2016 and has already been conducted with several full-time and part-time student groups.
Conference Paper
Full-text available
pp121-128 This cross-over study compares student laboratory work conducted in two different learning modes: first as a practical hands-on exercise and second using computer-based simulations. The research methodology was optimized to avoid other effects on the learning outcome. To evaluate the influence of the mode, short tests on knowledge gained during the previous experiment were conducted at the beginning of the next laboratory session. In 2016, forty students have taken part. Overall learning results of hands-on experiments were slightly better than those of simulated laboratories, but the difference in performance was not statistically significant. The study is continuing in 2017 with 30 participants. In addition to the knowledge tests, after each laboratory session the students were asked for their opinion in an online survey. A similar percentage of the students stated the execution of the experiments is beneficial for their future professional life. In the hands-on learning mode more students expressed they have acquired new knowledge. Although more students assessed the simulated laboratories as more challenging compared to hands-on experiments, more students mentioned obstacles while conducting the hands-on equivalents.
Conference Paper
Full-text available
BACKGROUND Understanding the characteristics of rechargeable batteries is essential for a successful career in the field of research and development of hybrid and electric cars. It has been shown that hands-on laboratory work can significantly influence the outcomes of student learning. However, universities and vocational training institutions need proper laboratory equipment to engage students in effective learning of batteries' behaviour. Increased amount of supervision to conduct hand-on labs safely as well as costs of specialised laboratory equipment make hands-on laboratories expensive. Therefore, many universities conduct such laboratories as simulated experiments. PURPOSE The aim of this study was to compare the learning outcomes of laboratory work on lithium-ion battery cells and components of battery systems conducted in two different modes: as a practical hands-on exercise and by means of computer-based simulation. The research had a strong focus on the learning mode of the laboratory experiment, the method was designed to avoid other effects on the result. DESIGN/METHOD The students were split into two comparable groups based on their prior practical experience to ensure a similar background level of the two groups. Each group was taught four content areas: two as practical hands-on experiments and two as computer-based simulations. One group completed the even laboratory sessions as hands-on experiments and the odd ones as computer-based simulations. The other group completed the odd laboratory sessions as hands-on experiments and the even as computer-based simulations. To evaluate the influence of the learning mode onto the student learning, anonymous 10-minute tests on knowledge gained during the previous experiment were conducted at the beginning of the next laboratory session. The average group results between hands-on and simulated mode were compared, to answer the question, which mode was more successful to transfer the knowledge. The method excludes learning synchronicity/distance learning/supervision effects, and is focused on the mode. RESULTS Forty students took part in the study. Three of four content areas showed weak to moderate effect: hands-on laboratory sessions led to a better knowledge acquisition compared to simulated experiments. One content area did not show any effect of study mode. Overall learning results of hands-on experiments were slightly better than that of simulated laboratories (weak effect, Cohen's d = 0.22), but the difference in performance was not statistically significant. CONCLUSIONS This study showed that the described methodology is applicable to focus on the comparison of two learning modes. The slightly better learning results in hands-on mode are not significant. To get statistically significant results, more data collection is necessary. KEYWORDS Hands-on experiment, simulated experiment, student experiment, battery experiment, comparing learning-modes
Article
Full-text available
Although weight and mass are considered fundamental concepts in physics, they are still not well understood by students. A computer simulation was designed to improve students’ learning of these concepts and compared it with other teaching strategies. The research was carried out with 142 students (7th grade; 12-13 years old), from three schools. There is a significant change in conceptual understanding of the concepts weight and mass for all groups. Nevertheless, total gains were higher for students who used the computer simulation. The implication is that using a computer simulation, carefully designed to address specific conceptual difficulties, may help the students understand the concepts of weight and mass. We also interviewed teachers to understand their role in the classroom. It was found that the features most likely to contribute to improve students’ learning are related to the balance between support and autonomy given to students during the use of the computer simulation.
Article
Full-text available
Laboratory experimentation plays an essential role in engineering and scientific education. Virtual and remote labs reduce the costs associated with conventional hands-on labs due to their required equipment, space, and maintenance staff. Furthermore, they provide additional benefits such as supporting distance learning, improving lab accessibility to handicapped people, and increasing safety for dangerous experimentation. This paper analyzes the literature on virtual and remote labs from its beginnings to 2015, identifying the most influential publications, the most researched topics, and how the interest in those topics has evolved along the way. To do so, bibliographical data gathered from ISI Web of Science, Scopus and GRC2014 have been examined using two prominent bibliometric approaches: science mapping and performance analysis.
Article
Abstract This review presents the first attempt to synthesize recent (post-2005) empirical studies that focus on directly comparing learning outcome achievement using traditional lab (TL; hands-on) and non-traditional lab (NTL; virtual and remote) participants as experimental groups. Findings suggest that most studies reviewed (n = 50, 89%) demonstrate student learning outcome achievement is equal or higher in NTL versus TL across all learning outcome categories (knowledge and understanding, inquiry skills, practical skills, perception, analytical skills, and social and scientific communication), though the majority of studies (n = 53, 95%) focused on outcomes related to content knowledge, with most studies (n = 40, 71%) employing quizzes and tests as the assessment instrument. Scientific inquiry skills was the least assessed learning objective (n = 4, 7%), and lab reports/written assignments (n = 5, 9%) and practical exams (n = 5, 9%) were the least common assessment instrument. The results of this review raise several important concerns and questions to be addressed by future research.
Article
This study was designed to assess the grades, self-perceived learning, and satisfaction between occupational therapy students who used a gross anatomy laboratory versus online anatomy software (AnatomyTV) as tools to learn anatomy at a large public university and a satellite campus in the mid-western United States. The goal was to determine if equivalent learning outcomes could be achieved regardless of learning tool used. In addition, it was important to determine why students chose the gross anatomy laboratory over online AnatomyTV. A two group, post-test only design was used with data gathered at the end of the course. Primary outcomes were students' grades, self-perceived learning, and satisfaction. In addition, a survey was used to collect descriptive data. One cadaver prosection was available for every four students in the gross anatomy laboratory. AnatomyTV was available online through the university library. At the conclusion of the course, the gross anatomy laboratory group had significantly higher grade percentage, self-perceived learning, and satisfaction than the AnatomyTV group. However, the practical significance of the difference is debatable. The significantly greater time spent in gross anatomy laboratory during the laboratory portion of the course may have affected the study outcomes. In addition, some students may find the difference in (B+) versus (A-) grade as not practically significant. Further research needs to be conducted to identify what specific anatomy teaching resources are most effective beyond prosection for students without access to a gross anatomy laboratory. Anat Sci Educ. © 2015 American Association of Anatomists. © 2015 American Association of Anatomists.
Article
We studied how students engaged with an interactive simulation in a classroom setting and how that engagement was affected by the design of a guiding activity. Students (n = 210) completed a written activity using an interactive simulation in second semester undergraduate general chemistry recitations. The same simulation – PhET Interactive Simulations' Acid–Base Solutions – was used with three written activities, designated as Heavy Guidance (HG), Moderate Guidance (MG), or Light Guidance (LG). We collected mouse click data and classroom field notes to assess student engagement with each type of activity. Simulation features were characterized as “prompted” or “exploratory” based on the presence or absence of explicit guidance in the written activity to use that feature. While students in every condition were engaged with the simulation and their activity, student interaction with “exploratory” features decreased significantly when more guidance was provided (LG = 85%, MG = 68%, HG = 9%, p < 0.0005). Lighter guidance groups explored more and attended to their simulation interactions, indicated by a redraw task in the week after use. These results indicate that activity design – in terms of guidance level – can strongly influence student exploration with an interactive simulation. We discuss the implications of these results for the design of activities to accompany simulations, including how to increase student practice in scientific inquiry.
Article
Research indicates that cooperative learning (CL) has the potential to increase accounting student satisfaction without impairing the learning of technical material (Lancaster, K., & Strand, C. (2001). Using the team-learning model in a managerial accounting class: An experiment in cooperative learning. Issues in Accounting Education, 16(4), 549–567). This study investigates whether instructor-formed heterogeneous groups produce a more effective CL environment than student self-selected groups by measuring individual academic performance and perceptions. Results indicate the presence of a treatment interaction, implying that the best group composition may not be the same for all students. In some circumstances, higher ability students had statistically higher performance in more homogeneous groups. Lower ability students did better (not significant) in heterogeneous groupings. The majority of students gave high ratings to the impact of CL on learning and development of team skills. Students were only mildly interested in increasing the time spent in groups, indicating that they value both traditional teaching methods and CL.