Content uploaded by Christin Kleinsorgen
Author content
All content in this area was uploaded by Christin Kleinsorgen on Sep 27, 2017
Content may be subject to copyright.
AHEAD OF PRINT ARTICLE
Impact of Virtual Patients as Optional
Learning Material in Veterinary
Biochemistry Education
Christin Kleinsorgen nMaren von Ko
¨ckritz-Blickwede nHassan Y. Naim n
Katja Branitzki-Heinemann nMarta Kankofer nMı
´ra Ma
´ndoki nMartin Adler n
Andrea Tipold nJan P. Ehlers
ABSTRACT
Biochemistry and physiology teachers from veterinary faculties in Hannover, Budapest, and Lublin prepared inno-
vative, computer-based, integrative clinical case scenarios as optional learning materials for teaching and learning
in basic sciences. These learning materials were designed to enhance attention and increase interest and intrinsic
motivation for learning, thus strengthening autonomous, active, and self-directed learning. We investigated learn-
ing progress and success by administering a pre-test before exposure to the virtual patients (vetVIP) cases, offered
vetVIP cases alongside regular biochemistry courses, and then administered a complementary post-test. We
analyzed improvement in cohort performance and level of confidence in rating questions. Results of the perfor-
mance in biochemistry examinations in 2014, 2015, and 2016 were correlated with the use of and performance in
vetVIP cases throughout biochemistry courses in Hannover. Surveys of students reflected that interactive cases
helped them understand the relevance of basic sciences in veterinary education. Differences between identical
pre- and post-tests revealed knowledge improvement (correct answers: þ28% in Hannover, þ9% in Lublin) and
enhanced confidence in decision making (‘‘I don’t know’’ answers: 20% in Hannover, 7.5% in Lublin). High case
usage and voluntary participation (use of vetVIP cases in Hannover and Lublin >70%, Budapest <1%; response
rates in pre-test 72% and post-test 48%) indicated a good increase in motivation for the subject of biochemistry.
Despite increased motivation, there was only a weak correlation between performance in final exams and perfor-
mance in the vetVIP cases. Case-based e-learning could be extended and generated cases should be shared across
veterinary faculties.
Key words: veterinary education, biochemistry education, educational activities, virtual systems, CASUS
software, virtual cases, virtual patients, e-learning, case-based learning
INTRODUCTION
To reflect and optimally support students’ individual de-
velopment, learning processes, and learning potentials,
the detection of their progress according to learning
outcomes and success regarding confidence levels are
crucial.
1
As Van der Vleuten has described, regular prog-
ress testing can help to emphasize functional longitudinal
knowledge and has positive effects on the level of usual
anxiety about examinations. Innovations in digital tech-
nologies provide great access to learning resources,
which promote individualized and self-directed learning.
2
For teachers it is important to stimulate students’ intrinsic
motivation to enhance independent and self-regulated
learning.
3
A pre-test/post-test study design is a widely
used tool to measure changes resulting from experimen-
tal treatments or, as described in this study, the introduc-
tion of educational services.
4
Over the past years, many universities have integrated
problem-based and case-based learning into their veteri-
nary curricula.
5–8
However, at the participating veterinary
faculties in this study (i.e., University of Life Sciences in
Lublin, University of Veterinary Medicine in Hannover,
and Szent Istvan University in Budapest) the teaching of
biochemistry is mainly lecture-based, including practical
hands-on classes. Problem- or case-based learning has
not been well established in this subject. The use of vir-
tual patients has steadily increased in medical
9–13
and
veterinary medical education.
14–16
Virtual patients are
‘‘an interactive computer simulation of real-life clinical
scenarios for the purpose of medical training, education,
or assessment.’’
17(p.2)
There are various types of virtual
patients described in the literature, ranging from case
presentation using multimedia systems to high-fidelity
simulations presented in virtual worlds.
18
According to
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
Kononowicz’s recently published framework, the virtual
patients described in the present study are in the category
of case presentation and interactive patient scenarios, pre-
dominantly imparting knowledge and clinical reasoning
using multimedia systems as technology. Problem- and
case-based models and the use of virtual patients are
known as student-centered learning formats, which en-
courage self-directed learning and problem-solving
skills.
19,20
It is well known that providing the context for learning
and understanding brings intellectual satisfaction and
seems to play an essential role in establishing memory.
21
The ability to recall or reconstruct correlation between
basic science mechanisms and clinical signs and symp-
toms helps students to acquire critical and diagnostic
thinking skills and to understand the context as a conse-
quence of the learning process.
22,23
In accordance with
the Ebbinghaus curve of forgetting,
24
regularly recalled
and applied knowledge in meaningful context leads to
better retention. To support this learning process toward
understanding and longer lasting retention, the Veterinary
Virtual Patients (vetVIP) Consortium (see Acknowledg-
ments) decided to implement formative tests and exercises
that provide immediate feedback and foster the testing
effect.
25–28
Roediger and Karpicke review the phenomenon
of the testing effect and state that ‘‘testing not only mea-
sures knowledge, but also changes it, often greatly im-
proving retention of the tested knowledge. Taking a test
on material can have a greater positive effect on future
retention of that material than spending an equivalent
amount of time restudying the material, even when per-
formance on the test is far from perfect and no feedback
is given on missed information.’’
29(p.181)
As feedback also
enhances the described benefits of testing, each question
within the created vetVIP cases was complemented by a
detailed explanation of right and wrong answers. On the
one hand, this introduction of formative testing should
drive engagement in the subject of biochemistry and, on
the other hand, the rapid feedback should help the stu-
dents to better understand and reflect their own learning
progress.
The subject of biochemistry is not known as a favorite
among veterinary students.
30
Retention of basic science
has been reviewed by Custers.
31
His recommendations
for instructional strategies included frequent testing and
supplementing important material with additional inde-
pendent learning sessions.
Biochemistry and physiology teachers from the veteri-
nary faculties in Hannover, Budapest, and Lublin prepared
innovative, computer-based, integrative clinical case sce-
narios as optional learning material for teaching and
learning in basic sciences. The goal of these learning
materials was to enhance attention to the relevance of
biochemistry as a subject and to increase interest and in-
trinsic motivation for learning, thus strengthening auton-
omous, active, and self-directed learning.
32,33
Throughout the vetVIP project, 30 virtual patients were
created using CASUS (Instruct AG, Munich), a case-
based, multimedia learning and authoring system. The
virtual patients were designed to enhance diagnostic and
problem-solving skills. The teaching of complex mecha-
nisms and problems was supplemented by audiovisual
media to document authentic situations and clinical signs.
Illustration in this format is intended to help students to
better remember the content and to link theoretical back-
ground with practical clinical examples.
In Hannover, the authoring and case-based e-learning
tool CASUS had already been implemented and is used
on a regular basis. In 2010, Borchers described wide
acceptance of the system in Hannover among students
and case authors, which was taken as an opportunity to
expand the case offerings and conduct further research
in this field.
14
In addition to the veterinary faculty in Hannover, those
in Budapest and Lublin participated in the present study.
Throughout the vetVIP project, CASUS was introduced as
an e-learning tool to the subject of biochemistry at all three
participating veterinary faculties.
The following research questions were tested in this
study:
eCan optional learning material in addition to regular
biochemistry courses increase student motivation for
the subject of biochemistry and thus their learning
success?
eIs there a correlation between the use of optional
learning material in addition to regular biochemistry
courses and student performance in their final
biochemistry examinations?
MATERIAL AND METHODS
The vetVIP Cases
In total, 30 veterinary virtual patients were created in
close collaboration with the three participating veterinary
faculties, the Instruct AG, and the E-Learning Depart-
ment in Hannover (www.vetvip.eu). To create uniform
cases at three different locations, common standards
were defined and guidelines for case creation were pro-
vided. The first 15 cases present diseases (e.g., rickets,
scurvy, diarrhea, heart failure by ischemia, and oxygen
shortage in myocardial cells) or discuss biochemical pro-
cesses (hemostasis, cell death, glucose metabolism, lac-
tase deficiency, thyroid hormone regulation) in different
patients (horse, dog, cat, monkey, calf, guinea pig, and
carp; see Figure 1). Before publication, all cases were
mutually reviewed for content as well as technical and
didactical aspects.
Study Design
Education in the subject of biochemistry at the participat-
ing veterinary faculties lasts for 1 year (i.e., two semesters).
In Hannover, for example, there are 84 hours of lectures
and 28 hours of practical classes. These lectures and prac-
tical classes are mainly run by biochemists and not by
veterinarians.
At the beginning of the second semester, an identical
set of questions regarding the content of 15 vetVIP cases
was administered as a pre-test in the form of a voluntary
online survey. The tests were available in the following
2JVME 2017; ahead of print article doi: 10.3138/jvme.1016-155r1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
languages: English, German, Polish, and Hungarian. The
questions were mutually checked by the vetVIP Consor-
tium for correctness and terms of content. Before publica-
tion of the first 15 virtual cases, the pre-test was active for
one month (November 2013) and the link was sent by
email to veterinary undergraduate students in their
second year of the degree and their second semester of
biochemistry education. No feedback or solutions were
given after pre- or post-tests. In total, 795 students were
invited (Budapest n¼311, Hannover n¼268, Lublin
n¼216) to participate in the pre-test and were informed
about the implementation of the vetVIP course in CASUS
as optional learning material in addition to their regular
biochemistry classes. Participation in the pre-test had no
influence on the invitation to the CASUS course, so 795
students were invited to register. Due to the low case
usage in Budapest (>1%), the corresponding post-test
was only sent to 484 students from Hannover (n¼268)
and Lublin (n¼216).
The pre- and post-tests consisted of two parts: the first
part included four questions regarding student profile
(university, semester, gender, and age), and the second
part contained three single-best-answer questions cover-
ing the educational objectives of the 15 virtual cases (45
questions). After the pre-test, students had access to the
cases (five cases from each university). All 15 cases could
be processed at any time or pace. During the examination
period for biochemistry in 2014, the corresponding post-
test and an evaluation questionnaire were administered
(Hannover: February 2014; Lublin: March 2014). The
time of the post-test corresponded with the time of the
examination period at each faculty. The post-test was
open for one month, about 2 weeks prior and 2 weeks
after the exam.
In the vetVIP cases, various formats of questions and
answers were used (single best answer, multiple choice,
free text, underline, sorting, mapping, etc.). For the pre-
and post-tests, only one format was chosen: the four
answer options for each of the 45 questions in the pre-
and post-tests were defined as one correct answer, two
distractors (Wrong I and Wrong II), and a fourth option
of ‘‘I don’t know.’’ The correct answer and the two dis-
tractors were randomly arranged, whereas the ‘‘I don’t
know’’ option was always displayed fourth.
For the analyses of the overall performance of each stu-
dent in the pre- and post-tests, success was established in
three ways:
1. Success in deciding: the student makes a choice. The
important factor is the student’s assertiveness in mak-
ing a choice. Whether the answer is right or wrong is
irrelevant.
2. Success in not choosing a distractor (Wrong I or
Wrong 2): the student chooses the correct answers or
is undecided and chooses ‘‘I don’t know.’’
3. Success in choosing the correct answer: the student
makes the right choice.
These three meanings of success were defined to evaluate
not only the improvement in knowledge but also students’
confidence and decision-making competence. Further-
more, the level of known or unknown incompetence
should be addressed by using and comparing the three
different measures.
34
In addition, the performance in final biochemistry exams
in 2014, 2015, and 2016 among students from Hannover
were correlated with performance in CASUS. The bio-
chemistry exams were conducted electronically using
Q-Exam, and 60 single-best-answer questions with one
attractor and two distractors were tested. Several ques-
tions tested in final exams were similar to questions used
in the virtual cases or the pre- and post-tests.
Potential Sources of Methodological Bias
One limitation is that CASUS user log files only store
data of the last session of one user, thus export and
analysis of files are limited to the very last session. Any
Figure 1: Screenshot of CASUS in player mode (one card of the case ‘‘Identifying the killer’’)
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 3
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
analysis of progress throughout repeated case sessions on
individual levels were not measurable. Therefore, for the
performance within the cases, only students’ final perfor-
mance could be analyzed. Throughout the vetVIP cases,
immediate feedback was given for each question or task.
Students were asked to complete the pre- and post-
tests only once to not invalidate the results. No scores or
solutions were given after the pre- or post-test. As both
tests consisted of 45 questions, each took an average of
1 hour, and so we estimate that hardly any student
completed either test more than once.
The correlation of performance in final biochemistry
exams with vetVIP case usage was only performed with
results from Hannover.
Due to data protection, the pre- and post-test surveys
were completed anonymously, so no comparison of
performance on individual levels could be measured.
Therefore analysis had to be concluded using mean data
and aggregated percentages.
Data Collection
The design, distribution, and collection of the pre- and
post-test surveys were done through the web-based survey
tool SurveyMonkey (www.surveymonkey.com) and its
integrated statistic software.
Students’ actions in the vetVIP course were registered
in the software package CASUS with its integrated auto-
matic report system. Quantitative data, such as time
spent per session or success rate, are reported for each
registered user and can be exported as user log files.
Assessment and analysis of the final exams using Q-
Exam were managed in cooperation with the Institute
for Quality Management in Teaching and Training (IQuL
GmbH, Bergisch Gladbach, Germany).
Statistical Analysis
For statistical analyses, data from CASUS and Survey-
Monkey were analyzed using SAS Enterprise Guide, ver-
sion 5.1 (SAS Institute, Cary, NC).
The answers given by students from Hannover and
Lublin in pre- and post-tests were checked for normal
distribution. To determine changes between the pre- and
post-test surveys, the nonparametric Wilcoxon matched-
pair test was used. Furthermore a Pearson Chi-square
test for homogeneity of distribution and the one-sided
Fisher’s exact test were performed.
Spearman’s rank non-parametric correlations were con-
ducted to evaluate the correlation between the overall
performance in final exams and overall performance in
case sessions.
Ethical Considerations
The study was conducted according to the ethical rules of
the University of Veterinary Medicine in Hannover. The
data protection officer and the doctoral committee of the
university gave their consent to the proposed project
before students completed the surveys. All data obtained
were processed and evaluated anonymously and in ac-
cordance with EU Directive 95/46/EC.
For registration in the CASUS system, students had to
accept the data privacy statement actively, which included
storage and collection of personal data (case usage, time
spent per case, answers given, success rate, etc.).
In the pre- and post-test surveys, the first question
asked participants to accept the data protection statement
to continue, or to be disqualified by refusal.
Table 1: Number of participants in vetVIP cases and pre- and post-test surveys
Participants in vetVIP cases Participants in pre-test Participants in post-test
Invitations
sent
Registration
rate
Completed
vetVIP cases
Invitations
sent
Started
surveys
Completed
surveys
Invitations
sent
Started
surveys
Completed
surveys
Budapest 311 12 3 311 215 121 – – –
(3.9%) (1.0%) (69.1%) (38.9%)
Hannover 268 202 199 268 171 137 268 118 74
(75.4%) (74.3%) (63.8%) (51.1%) (44%) (27.6%)
Lublin 216 177 164 216 189 162 216 116 81
(81.9%) (75.9%) (87.5%) (75%) (53.7%) (37.5%)
Total 795 391 366 795 575 420 484 234 155
(49.2%) (46.0%) (72.3%) (52.8%) (48.3%) (32%)
Table 2: Pre- and post-test mean values of observed
responses (% of students from Hannover and Lublin)
Pre-test Post-test Difference
Mean SD Mean SD t-test
Hannover
Correct 41.49 27.98 69.06 24.80 <.001
Wrong I 12.04 11.23 8.53 11.10 .010
Wrong II 13.38 16.04 9.50 13.96 .056
I don’t know 33.09 22.04 12.90 13.81 <.001
Lublin
Correct 61.76 27.87 70.63 23.70 <.001
Wrong I 12.49 14.16 11.62 14.59 .217
Wrong II 14.49 17.25 14.00 16.46 .596
I don’t know 11.26 12.03 3.75 4.63 <.001
4JVME 2017; ahead of print article doi: 10.3138/jvme.1016-155r1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
RESULTS
Registration rate for the virtual course provided in the
CASUS system was 3.9% in Budapest (12/311), 75.4% in
Hannover (202/268), and 81.9% in Lublin (177/216).
Analyses of the Pre- and Post-Test Surveys
In total, 575 of 795 invited participants (72%) started the
pre-test survey and 420 (53%) completed it. The post-test
was only sent to students from Hannover and Lublin,
and 234 of 484 invited students (48%) started the post-
test, of which 155 (32%) completed it (see Table 1).
For the comparison of results from the pre- and post-
test surveys, only completed surveys were used.
The mean values of observed responses for the pre-
and post-test surveys of students from Hannover and
Lublin are illustrated in Table 2.
In Hannover, the mean differences between mean
values of detected answers given in the pre- and post-
test surveys reveal an increase of 27.57% for correct
answers. Selection of the Wrong I option decreased by
3.5% and of the Wrong II option by 3.87%. In total, dis-
tractor answers were given 7.37% less often in the post-
test than in the pre-test. The ‘‘I don’t know’’ option was
chosen 20.19% less often.
In Lublin, the mean differences between pre- and post-
test surveys reflect an increase in correct answers of
8.87%, a decrease in Wrong I choices of 0.87%, and a
decrease in Wrong II choices of 0.49%. In total, students
chose distractor answers 1.36% less often in the post-test.
The ‘‘I don’t know’’ option was chosen 7.52% less often
(see Figure 2).
The one-sample t-test yielded significant differences
between mean differences in percentage of given correct
answers (p<.001), Wrong I choices (p¼.001), and ‘‘I
don’t know’’ choices (p<.001) for the pre- and post-test
surveys of students from Hannover. Differences in Wrong
II had a pvalue of .056.
In Lublin, significant differences between pre- and post-
test results were measured for given correct answers
(p<.001) and the ‘‘I don’t know’’ option (p<.001). Re-
sults for Wrong I are p¼.217 and for Wrong II are
p¼.596.
Analyses of Success of Decision Making
Pre- and post-test results were compared for all 45 ques-
tions using three contingency tables with regards to
success in three different categories:
1. Success in deciding: comparison of decided (correct,
Wrong I, Wrong II) and undecided (‘‘I don’t know’’)
answer options in pre- and post-test surveys.
2. Success in not choosing a distractor: comparison of
correct and distractor (Wrong I, Wrong II) answers
in pre- and post-test surveys.
3. Success in choosing the correct answer: comparison
of correct and not correct (Wrong I, Wrong II, ‘‘I
don’t know’’) answers in pre- and post-test surveys.
Out of 45 questions, the one-sided Fisher’s exact test
shows statistically significant differences for 35 questions
between decided and undecided answers in pre- and
post-test surveys in Hannover (see Figure 3a). Further-
more, differences between correct and distractor answer
choices in pre- and post-test surveys significantly differed
Figure 2: Results (%) of the pre- and post-tests in Hannover and Lublin
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 5
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
for 26 out of 45 questions (see Figure 3b). For the success
in choosing the correct answer, differences between results
in pre- and post-test surveys were analyzed as statistically
significant in 36 tested questions (see Figure 3c).
In Figures 4a–c, results of the pre- and post-test surveys
in Lublin are illustrated. Statistically significant differences
between decided and undecided answers were yielded for
19 questions, between correct and distractor answers for 9
questions, and between correct and not correct answers
for 15 questions.
Correlation of Exam and CASUS Performance
In this study, performance on the biochemistry exam is
defined as the percentage of given correct answers in
Figures 3a–c: Results of the pre- and post-test surveys in
Hannover
Figures 4a–c: Results of the pre- and post-test surveys in
Lublin
6JVME 2017; ahead of print article doi: 10.3138/jvme.1016-155r1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
the final test out of the maximum score (60 questions: 60
correct answers M100%). Performance in CASUS is de-
fined as percentage of correct answers given throughout
the vetVIP cases. The number of tested questions in
CASUS differs as students worked through different
numbers of cases. Table 3 shows the number of students
who took the final exams in biochemistry in Hannover
and additionally used the vetVIP cases throughout their
biochemistry course, and the number of students who
attended the final exam but did not use CASUS.
There was no statistical difference in exam performance
between students who completed vetVIP cases in CASUS
and those who did not use the CASUS system.
Results of the correlation of performance in final exams
and performance in CASUS from students in Hannover
are illustrated in Table 4.
The scattergrams for the three biochemistry courses are
illustrated in Figures 5a–c.
The regression line indicates a positive correlation
between performance in final biochemistry exam and
performance in the CASUS system, though only weak
correlations were measured by Spearman’s correlation
coefficient.
DISCUSSION
When implementing innovative e-learning tools, efficiency
and effectiveness are of particular importance. Previous
studies have shown that in the mind of medical students,
e-learning does not replace but rather complements tradi-
tional or instructional teaching.
2
This view is in accor-
dance with the outcomes of a previous study performed
during the vetVIP project.
35
Students from Hannover and
Lublin confirmed that the content of the virtual patients/
problems and the corresponding teaching events comple-
mented each other well. The authors of the vetVIP cases
from Budapest, Hannover, and Lublin agreed that cases
should be used as supplementation to selected lectures
or practical classes. Students felt that the questions they
were asked while working through the cases were help-
ful in enhancing diagnostic reasoning and, furthermore,
that the combination of virtual patients/problems and
corresponding teaching events enhanced their clinical
Table 3: Exam and CASUS performance for the biochemistry courses of 2012–2014, 2013–2015, and 2014–2016
Exam performance
(%)
CASUS performance
(%)
Time spent in CASUS
(min)
Min Max Mean Min Max Mean Min Max Mean
Students who used
CASUS and attended
exam
2012–2014 (n¼192) 35.59 96.61 71.85 0 97 57.37 1 1,648 352.85
2013–2015 (n¼82) 35.00 93.33 68.92 0 90.00 60.26 1 783 149.44
2014–2016 (n¼136) 45.00 98.33 75.10 0 92.00 60.38 1 741 107.66
Students who attended
exam without using
CASUS
2012–2014 (n¼43) 42.37 94.92 68.51
2013–2015 (n¼151) 36.67 88.33 63.78
2014–2016 (n¼92) 25.00 90.00 67.41
Mean performance in exam Mpercentage of reached score from maximum score
Maximum score of biochemistry exam in 2012–2014 ¼59, in 2013–2015 ¼60, and in 2014 –2016 ¼60
Mean performance in CASUS Mmean average of success rate of case sessions expressed as a percentage
Table 4: Correlation of exam performance with CASUS performance
Biochemistry course 2012–2014 Biochemistry course 2013–2015 Biochemistry course 2014–2016
Spearman’s correlation coefficient,
n¼194 prob >|r| under H0: rho ¼0
Spearman’s correlation coefficient,
n¼133 prob >|r| under H0: rho ¼0
Spearman’s correlation coefficient,
n¼194 prob >|r| under H0: rho ¼0
Exam
performance
Exam
performance
Exam
performance
CASUS
performance
.209 CASUS
performance
.298 CASUS
performance
.151
.004*.007*.080
Time spent in
CASUS
.019 Time spent
in CASUS
.044 Time spent
in CASUS
.051
.795 .696 .555
Successful case
sessions
.144 Successful
case sessions
.037 Successful
case sessions
.182
.046*.7388 .034*
Spearman’s correlation coefficient: .00–.19 ¼very weak, .20–.39 ¼weak, .40–.59 ¼moderate, .60–.79 ¼strong, .80– 1.0 ¼very strong
*p<.05 is statistically significant
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 7
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
reasoning skills. However, students from Budapest did
not choose to use the learning material. A possible reason
for this was the less developed state of CASUS imple-
mentation. The e-learning tool CASUS was newly intro-
duced in Budapest within the vetVIP project, whereas it
had already been used in Hannover since 2005
14
and in
Lublin since 2011.
33
In addition, the intensity of advertise-
ment varied at each participating university.
Also, biochemistry examinations differ at each univer-
sity. In Hannover, there are two written tests using single-
best-answer format. In Lublin and Budapest, students have
to pass an oral examination. Students from Hannover
already know the format of single-best-answer tests,
although they do not have an ‘‘I don’t know’’ option in
summative examinations. This could help explain why
students from Hannover used the ‘‘I don’t know’’ option
less often than students from Lublin, as they are not used
to having this answer option. Fortunately students from
Hannover and Lublin both acquired knowledge and
made fewer undecided or wrong (i.e., distractor) answer
choices (Figures 3 and 4). In addition, students made use
of the ‘‘I don’t know’’ option more often at the beginning
of the semester than shortly before their final exams.
Having fewer Wrong 1 or Wrong 2 answers and fewer
‘‘I don’t know’’ answers proved that the students reflected
their knowledge and did not suffer from ‘‘unknown in-
competence.’’
34
Furthermore, students did decide more often in the
post-test (i.e., they did not choose ‘‘I don’t know’’), and
they chose the correct answer option more often. How-
ever, this improvement cannot be correlated only with
the optional e-learning support but also with the ongoing
traditional biochemistry teaching and students’ self-study
period before the exam. Through intensive preparation
during the semester with various types of learning, and
especially before the exam, students might have compen-
sated for differences in their knowledge and decision-
making skills.
Previous studies have confirmed that case-based learn-
ing in veterinary curricula enhances the development of
clinical reasoning skills, but it is difficult to assess the
effect size and the reason why students improve.
7
In our
study, students did gain confidence in decision making.
Significantly fewer students used the ‘‘I don’t know’’ re-
sponse option and more correct answers were chosen.
Unfortunately the factors why students were less un-
decided in the post-test survey were not elaborated in
this study. The case usage might have assisted the stu-
dents to become more decisive, as they were trained in
cases that require a decision-making approach.
A limitation of the study due to data protection regula-
tions was that pre- and post-test surveys were completed
anonymously and no paired comparison of performance
was possible. To be able to perform a direct comparison
on a personal level, names or IP-addresses would have
been needed and stored. Unfortunately, consent for col-
lection of these data was not approved within this study.
Because the voluntarily participation rate in the pre-test
and the corresponding post-test exceeded 30% of the
cohort, we feel that the results are representative. The
pre- and post-test surveys were only conducted for
the biochemistry course in 2012–2014. We cannot verify
if there was a positive or negative selection among stu-
dents. It may be that only highly motivated students
completed the pre- and post-test surveys and also the
vetVIP cases. Another interesting question within the
vetVIP project that has yet to be investigated is whether
there are there differences in the learning outcomes
Figures 5a–c: Scattergrams of the correlation of exam
performance with CASUS performance for the biochemistry
courses of 2012–2014, 2013–2015, and 2014–2016
8JVME 2017; ahead of print article doi: 10.3138/jvme.1016-155r1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
when case-based learning material is compulsory instead
of voluntary.
We were pleased with the quite high number of volun-
tarily participating students in the pre- and post-tests and
the time spent processing the virtual cases. It is important
to note that special advertisements were only directed
at students throughout the first biochemistry course in
2012–2014, when the virtual cases were newly introduced.
During the next two courses, fewer students participated
(2012–2014 >75%, 2013–2015 >35%, and 2014–2016 >
55%). Despite this, the rate of case use as optional learning
material was quite high. A previous concern that students
might just click through formative tests and virtual cases
rather than working through them conscientiously was
disproved. Cases were designed to take on average 20–45
minutes to complete. The time students spent per case
(mean average 34 minutes) revealed that students did not
just click through, as this would probably have taken only
5 minutes. An increase in motivation to deal with the
subject of biochemistry alongside the standard instruction
can be estimated due to the high voluntary participation.
Motivating effects of problem- and case-based learning
are consistent in the published literature.
23
Furthermore,
success was documented in all three of the defined cate-
gories of success (see Figures 3 and 4). In Hannover and
Lublin, a large learning effect occurred when students
started their biochemistry course with little knowledge
and then improved through the combination of lectures,
practical classes, and use of virtual patients. Whether an
increase in knowledge and establishment of long-lasting
retention occurred cannot be answered. Unfortunately,
only weak or very weak correlations between performance
in final exams and performance in the CASUS system
were statistically detected. The linear regression line added
in scattergrams (Figure 5) illustrated a positive trend.
As already foreseen in the first study conducted within
the vetVIP project, a noteworthy positive side effect of
the development of vetVIP cases is that teachers reflect
on their teaching and shift from instructor-centered
toward more student-centered models of instruction.
35
A
special feature of this project was that three universities
from three different countries with different curricula
participated. The case creation and review process included
teachers from biochemistry and physiology together with
educationalists, technical staff, and clinicians and it led
to a remarkable interactive and interdisciplinary net-
working experience.
Horton stated more than a decade ago, ‘‘E-technologies
do not change how human beings learn. What technology
does is to remove constraints on the kinds of learning ex-
periences we can economically and practically create.’’
36(p.3)
The implementation of e-learning tools in biochemistry
teaching in veterinary medical education may not change
student learning, but it enriches the spectrum of supportive
supply. It gives students the opportunity for self-regulated
learning. It offers great potential for the integration of
multimedia features and therefore the possibility to ac-
commodate different learning styles. As Cook et al. have
already recapitulated, ‘‘Virtual patients are associated
with large positive effects compared with no interven-
tion. Effects in comparison with noncomputer instruction
are on average small. Further research clarifying how to
effectively implement virtual patients is needed.’’
20(p.1589)
CONCLUSION
In general, student surveys revealed that the interactive
cases helped them to understand the relevance of basic
sciences in veterinary education and furthermore en-
hanced their diagnostic and clinical reasoning skills. A
relatively high case usage and voluntary participation
rate suggested an increase in motivation for the subject
of biochemistry. Growth in success was observed as
students gained knowledge and experiences in decision
making. Whether this effect took place through tradi-
tional instruction, the use of virtual patients, or the com-
bination of teaching and learning events has not been
proven. No causality of the relationship between perfor-
mance in final exams and performance in CASUS was
measured. However, it is important to note that partici-
pation in vetVIP cases was not a negative distraction to
students’ performance on content examinations. Overall,
all participants saw the vetVIP project as a success. All
groups appreciated the changes and improvements in
the learning and teaching activities at their respective
universities.
The vetVIP Consortium aims to establish opportunities
for intra- and extramural cooperation in courses and
content for more transparency of education and oppor-
tunities for university-independent collaborative work
among students and teachers. The vetVIP cases have
already been used at additional veterinary schools for
teaching and learning. This exchange of experiences and
sharing of innovative learning materials set an important
foundation for the sustainable improvement of teaching
and learning in veterinary medicine.
ACKNOWLEDGMENTS
The vetVIP project (use of virtual problems/virtual patients
in veterinary basic sciences) was supported by an EU
grant (526137-LLP-1-2012-1-PL-ERASMUS-FEXI, EU Life-
long Learning Programme).
The scientific work of the Polish partners was co-
financed by funds from the Polish Ministry of Science
and Higher Education (years 2012–2014), which were
assigned for the realization of he international vetVIP
project.
Special thanks to the students, veterinarians, and edu-
cationalists who volunteered to participate in the project
and helped with project activities.
The authors also thank Prof. Duncan Ferguson for
English proofreading.
The members of the vetVIP Consortium are as follows:
University of Life Sciences in Lublin: Marta Kankofer,
Zbigniew Gradzki, Witold Kedzierski, Jacek Wawrzykowski,
Marta Wojcik, Marta Giergiel, Michal Danielak, Marek
Szczubial, Wojciech Lopuszynski, Ewa Sobieraj
Szent Istvan University in Budapest: Bartha Tibor,
Ma
´ndoki Mira, To
´th Istva
´n, Somogyi Vira
´g, Jo
´csa
´kGergely,
Kiss Da
´vid Sa
´ndor
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 9
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
University of Veterinary Medicine in Hannover:
Hassan Y. Naim, Maren von Ko
¨ckritz-Blickwede,
Graham Brogden, Katja Branitzki-Heinemann,
Sucheera Chotikatum, Lena Diekmann, Eva-Maria Ku
¨ch,
Helene Mo
¨llerherm, Christin Kleinsorgen, Jan P. Ehlers
Instruct AG Munich: Martin Adler
REFERENCES
1 Van Der Vleuten CPM, Verwijnen GM, Wijnen WHFW.
Fifteen years of experience with progress testing in a
problem-based learning curriculum. Med Teach.
1996;18(2):103–9. https://doi.org/10.3109/
01421599609034142.
2 Ruiz JG, Mintzer MJ, Leipzig RM. The impact of
e-learning in medical education. Acad Med.
2006;81(3):207–12. https://doi.org/10.1097/00001888-
200603000-00002. Medline:16501260
3 Ten Cate TJ, Kusurkar RA, Williams GC. How self-
determination theory can assist our understanding of
the teaching and learning processes in medical
education. AMEE guide No. 59. Med Teach.
2011;33(12):961–73. https://doi.org/10.3109/
0142159X.2011.595435. Medline:22225433
4 Dimitrov DM, Rumrill PD Jr. Pretest–posttest designs
and measurement of change. Work. 2003;20(2):159–65.
Medline:12671209
5 Allenspach K, Bell J, Whittlestone KD. Interactive
clinical cases in veterinary education used to promote
independent study. J Vet Med Educ. 2008;35(4):589–94.
https://doi.org/10.3138/jvme.35.4.589.
Medline:19228913
6 Crowther E, Baillie S. A method of developing and
introducing case-based learning to a preclinical
veterinary curriculum. Anat Sci Educ. 2016;9(1):80–9.
https://doi.org/10.1002/ase.1530. Medline:25952276
7 Farnsworth CC. Measuring the effects of problem-based
learning on the development of veterinary students’
clinical expertise. Acad Med. 1997;72(6):552–4. https://
doi.org/10.1097/00001888-199706000-00024.
Medline:9200592
8 Patterson JS. Increased student self-confidence in clinical
reasoning skills associated with case-based learning
(CBL). J Vet Med Educ. 2006;33(3):426–31. https://
doi.org/10.3138/jvme.33.3.426. Medline:17035220
9 Cook DA, Triola MM. Virtual patients: a critical
literature review and proposed next steps. Med Educ.
2009;43(4):303–11. https://doi.org/10.1111/j.1365-
2923.2008.03286.x. Medline:19335571
10 Huang G, Reynolds R, Candler C. Virtual patient
simulation at US and Canadian medical schools. Acad
Med. 2007;82(5):446–51. https://doi.org/10.1097/
ACM.0b013e31803e8a0a. Medline:17457063
11 Huwendiek S, Ko
¨pf S, Ho
¨cker B, et al. Fu
¨nf Jahre
Erfahrung mit dem curricularen Einsatz des fall-und
webbasierten Lernsystems’’ CAMPUS-Pa
¨diatrie’’ an der
Medizinischen Fakulta
¨t Heidelberg [Five years of
experience with the curricular use of the case- and web-
based ‘‘CAMPUS-Pediatrics’’ learning system at the
Medical Faculty of Heidelberg]. GMS Z Med Ausbild.
2006;23(1):10. Available from: http://www.egms.de/
en/journals/zma/2006-23/zma000229.shtml.
12 Radon K, Kolb S, Reichert J, et al. Case-based e-learning
in occupational medicine—the NetWoRM project in
Germany. Ann Agric Environ Med. 2006;13(1):93–8.
Medline:16841879
13 Smothers V, Ellaway R, Balasubramaniam C. eViP:
sharing virtual patients across Europe. J AMIA Annu
Symp Proc. 2008:1140. Medline:18998985.
14 Bo
¨rchers M, Tipold A, Pfarrer Ch, et al. Akzeptanz von
fallbasiertem, interaktivem eLearning in der Tiermedizin
am Beispiel des CASUS-Systems [Acceptance of case-
based, interactive e-learning in veterinary medicine on
the example of the CASUS system]. Tierarztl Prax Ausg
K Kleintiere Heimtiere. 2010;38(6):379–88.
Medline:22212751
15 Byron JK, Johnson SE, Allen LCV, et al. Development
and pilot of Case Manager: a virtual-patient experience
for veterinary students. J Vet Med Educ. 2014;41(3):225–
32. https://doi.org/10.3138/jvme.1113-151R1.
Medline:24947678
16 Trace C, Baillie S, Short N. Development and
preliminary evaluation of student-authored electronic
cases. J Vet Med Educ. 2012;39(4):368–74. https://
doi.org/10.3138/jvme.0212-017R. Medline:23187029
17 Ellaway R, Candler C, Greene P, et al. An architectural
model for MedBiquitous virtual patients. Baltimore,
MD: MedBiquitous; 2006.
18 Kononowicz AA, Zary N, Edelbring S, et al. Virtual
patients—what are we talking about? A framework to
classify the meanings of the term in healthcare
education. BMC Med Educ. 2015;15(1):11. https://
doi.org/10.1186/s12909-015-0296-3. Medline:25638167
19 Harden RM, Davis MH. The continuum of problem-
based learning. Med Teach. 1998;20(4):317–22. https://
doi.org/10.1080/01421599880733.
20 Cook DA, Erwin PJ, Triola MM. Computerized virtual
patients in health professions education: a systematic
review and meta-analysis. Acad Med. 2010;85(10):1589–
602. https://doi.org/10.1097/ACM.0b013e3181edfe13.
Medline:20703150
21 Pintrich PR. A motivational science perspective on the
role of student motivation in learning and teaching
contexts. J Educ Psychol. 2003;95(4):667–86. https://
doi.org/10.1037/0022-0663.95.4.667.
22 Woods NN, Brooks LR, Norman GR. The value of basic
science in clinical diagnosis: creating coherence among
signs and symptoms. Med Educ. 2005;39(1):107–12.
https://doi.org/10.1111/j.1365-2929.2004.02036.x.
Medline:15612907
23 Patel VL, Groen GJ, Norman GR. Effects of conventional
and problem-based medical curricula on problem
solving. Acad Med. 1991;66(7):380–9. https://doi.org/
10.1097/00001888-199107000-00002. Medline:2059263
24 Ebbinghaus H. U
¨ber das Geda
¨chtnis: Untersuchungen
zur experimentellen Psychologie [On memory: studies
on experimental psychology]. Leipzig: Duncker and
Humblot; 1885.
25 Larsen DP, Butler AC, Roediger HL III. Test-enhanced
learning in medical education. Med Educ.
2008;42(10):959–66. https://doi.org/10.1111/j.1365-
2923.2008.03124.x. Medline:18823514
26 Roediger HL III, Karpicke JD. Test-enhanced learning:
taking memory tests improves long-term retention.
10 JVME 2017; ahead of print article doi: 10.3138/jvme.1016-155r1
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
Psychol Sci. 2006;17(3):249–55. https://doi.org/10.1111/
j.1467-9280.2006.01693.x. Medline:16507066
27 Brame CJ, Biel R. Test-enhanced learning: the potential
for testing to promote greater learning in undergraduate
science courses. CBE Life Sci Educ. 2015;14(2):4. https://
doi.org/10.1187/cbe.14-11-0208. Medline:25999314
28 Nicol DJ, Macfarlane-Dick D. Formative assessment and
self-regulated learning: a model and seven principles of
good feedback practice. Stud High Educ. 2006;31(2):199–
218. https://doi.org/10.1080/03075070600572090.
29 Roediger HL III, Karpicke JD. The power of testing
memory: basic research and implications for educational
practice. Perspect Psychol Sci. 2006;1(3):181–210.
https://doi.org/10.1111/j.1745-6916.2006.00012.x.
Medline:26151629
30 Rivarola VA, Bergesse JR, Garcia MB. A different
approach to the teaching of biological chemistry to
veterinary medicine students. Biochem Educ.
1996;24(2):96–7. https://doi.org/10.1016/0307-
4412(95)00139-5.
31 Custers EJFM. Long-term retention of basic science
knowledge: a review study. Adv Health Sci Educ
Theory Pract. 2010;15(1):109–28. https://doi.org/
10.1007/s10459-008-9101-y. Medline:18274876
32 Kankofer M. Learning theoretical knowledge doesn’t
have to be boring. Vet Rec. 2014;175(21):i–ii. https://
doi.org/10.1136/vr.g7173. Medline:25431386
33 Kankofer M, Kedzierski W, Wawrzykowski J, et al. Use
of virtual problems in teaching veterinary chemistry in
Lublin (Poland). Wien Tierarztl Monatsschr. 2016;103(5–
6):125–31.
34 Caleon IS, Subramaniam R. Do students know what
they know and what they don’t know? Using a four-tier
diagnostic test to assess the nature of students’
alternative conceptions. Res Sci Educ. 2010;40(3):313–37.
https://doi.org/10.1007/s11165-009-9122-4.
35 Kleinsorgen C, Kankofer M, Gradzki Z, et al. Utilization
and acceptance of virtual patients in veterinary basic
sciences—the vetVIP-project. GMS J Med Educ.
2017;34(2):19. https://doi.org/10.3205/zma001096.
Medline:28584867
36 Horton WK. Leading e-learning. Alexandria, VA:
American Society for Training and Development; 2001.
AUTHOR INFORMATION
Christin Kleinsorgen is a doctoral student at the E-Learning
Department, University of Veterinary Medicine Hannover,
Bu
¨nteweg 2, 30559 Hannover, Germany. Email:
Christin.Kleinsorgen@tiho-hannover.de.
Maren von Ko
¨ckritz-Blickwede, Prof. Dr. rer. nat., is Prof.,
Department of Physiologic Chemistry, University of Veterinary
Medicine Hannover, Bu
¨nteweg 17, 30559 Hannover, Germany.
Email: maren.von.koeckritz-blickwede@tiho-hannover.de.
Hassan Y. Naim, Prof. Dr. phil. nat., is Head of the Department
of Physiologic Chemistry, University of Veterinary Medicine
Hannover, Bu
¨nteweg 2, 30559 Hannover, Germany.
Email: hassan.naim@tiho-hannover.de.
Katja Branitzki-Heinemann, Dr. rer. nat., is a postdoctoral
research assistant at the Department of Physiologic Chemistry,
University of Veterinary Medicine Hannover, Bu
¨nteweg 2, 30559
Hannover, Germany. Email: katja.branitzki-heinemann@tiho-
hannover.de.
Marta Kankofer, Prof. Dr. hab., is Head of the Department of
Biochemistry, Faculty of Veterinary Medicine, University of Life
Sciences in Lublin, 20–033 Lublin, Akademicka 12, Poland.
Email: marta.kankofer@up.lublin.pl.
Mı
´ra Ma
´ndoki, Prof. Dr., PhD, is Associate Professor at the
Department of Pathology and Forensic Veterinary Medicine,
Veterinary Faculty, Szent Istva
´n University, P.O. Box 2, H-1400
Budapest, Hungary. Email: Mandoki.Mira@univet.hu.
Martin Adler, Diploma in Informatics, is Director, Instruct AG,
Kapuzinerstr. 5, 80337 Munich, Germany. Email:
martin.adler@instruct.eu.
Andrea Tipold, Prof. Dr. vet. med., is Vice-President for
Teaching, Small Animal Clinic, Neurology, University of Veterinary
Medicine Hannover, Bu
¨nteweg 9, 30559 Hannover, Germany.
Email: andrea.tipold@tiho-hannover.de.
Jan P. Ehlers, Prof. Dr. med. vet., MA, is Professor for Didactics
and Educational Research in Health Science, University Witten/
Herdecke, Alfred-Herrhausen-Strasse 50, 58448 Witten,
Germany. Email: jan.ehlers@uni-wh.de.
doi: 10.3138/jvme.1016-155r1 JVME 2017; ahead of print article 11
This ahead of print version may differ slightly from the final published version.
http://jvme.utpjournals.press/doi/pdf/10.3138/jvme.1016-155r1 - Wednesday, September 27, 2017 4:06:54 AM - TiHo Hannover IP Address:193.16.4.139
