Conference PaperPDF Available

Do Student-Instructor Co-created eLearning Materials Lead to Better Learning Outcomes? Empirical Results from a German Large Scale Course Pilot Study

Authors:

Abstract and Figures

Universities in Germany face increasing numbers of students, while resources are stagnating or shrinking. Especially large scale lectures suffer from a lack of individualization and interaction, often leading to inefficient learning outcomes and low student satisfaction. We present the concept of instructors co-creating eLearning modules and even parts of the exam with students for improving learning outcomes and student satisfaction. Following an action research approach, we developed, applied, and refined the concept in a mass lecture course setting over four semesters. Evaluation of questionnaires and exam results show that our concept leads to a significant increase in perceived learning satisfaction and learning outcomes. This paper outlines and tests new ways of enhanced learner integration into learning service delivery. It shows the actual impact of student integration into learning content creation, both on service quality and learning outcomes while taking resource efficiency into account.
Content may be subject to copyright.
Please quote as: Do Student-Instructor Co-Created eLearning Materials Lead To
Better Learning Outcomes? Empirical Results from a German Large Scale Course
Pilot Study. In: 45. Hawaii International Conference on System Sciences (HICSS),
Hawaii, USA.
Do Student-Instructor Co-Created eLearning Materials Lead To Better
Learning Outcomes? Empirical Results from a German Large Scale Course
Pilot Study
René Wegener
Kassel University, Information Systems
wegener@uni-kassel.de
Jan Marco Leimeister
Kassel University, Information Systems
leimeister@uni-kassel.de
Abstract
Universities in Germany face increasing numbers
of students, while resources are stagnating or
shrinking. Especially large scale lectures suffer from
a lack of individualization and interaction, often
leading to inefficient learning outcomes and low
student satisfaction. We present the concept of
instructors co-creating eLearning modules and even
parts of the exam with students for improving
learning outcomes and student satisfaction.
Following an action research approach, we
developed, applied, and refined the concept in a mass
lecture course setting over four semesters1.
Evaluation of questionnaires and exam results show
that our concept leads to a significant increase in
perceived learning satisfaction and learning
outcomes. This paper outlines and tests new ways of
enhanced learner integration into learning service
delivery. It shows the actual impact of student
integration into learning content creation, both on
service quality and learning outcomes while taking
resource efficiency into account.
1. Introduction
The OECD (Organisation for Economic Co-
operation and Development) claim that investments
in the educational sector influence the productivity of
an economy, and thus have a high return on
investment for society [28]. However, the German
education sector suffers from a lack of finances and
resources [38], especially since the numbers of
students have been increasing for several years and
are expected to grow even further during the next
years at a possible ratio of 25% [25]. Thus, delivering
a qualitative learning service fitting to individual
needs becomes more and more difficult.
eLearning materials offer the opportunity of
increasing individuality and quality of the learning
process by encouraging students to learn at their own
pace. To implement eLearning, however, instructors
1 Parts of this research were funded by the German Federal
Ministry of Education and Research in the project
BlendedContENT (www.blendedcontent.de), FKZ 01PF08022A
may have to re-design courses, create content,
administrate learning platforms, offer students
advice, and assess learning outcomes [15]. They have
to switch between roles of affective (creating a good
atmosphere in the classroom), cognitive (helping
students achieve learning goals) and managerial
(creating, delivering, and assessing the course from
an organizational perspective) tasks [8].
Considering this situation, we need to look for
new ways of delivering a high quality learning
service that takes into account the possibilities of new
media, didactical needs, and economic constraints.
Web 2.0 platforms, such as Wikipedia, show that
content creation does not exclusively depend on
single domain experts. In addition, many people feel
motivated when contributing to a product like a Wiki.
However, in educational contexts we need the right
processes and tools that support students in content
creation while still providing a guarantee of high
quality. With intelligent integration of students'
minds and information technologies, these goals can
be achieved while leveraging the instructor, thus
helping to deal with stagnating resources.
This paper presents the results of a case study that
was based on a traditional, instructor-centered, large
scale lecture, but was successively enhanced with
more interaction and student created eLearning
modules over a period of four semesters. The main
measures taken were the following:
1. Introducing small tasks during the lecture by
letting students create their own true-false-
items with regard to the course content that
could later be used in the exam, and
2. Supplementing learning materials through a
videoconferencing consultation hour, a video
broadcast and interactive web based trainings
(WBTs) that were implemented not by the
instructors but by the students themselves.
The goal was to enhance the course in two ways:
First, we wanted to offer students a better service
with higher perceived quality. Second, we wanted to
raise the actual learning outcomes. To evaluate the
achievement of objectives, the course was evaluated
after each semester with the help of online
questionnaires and a comparison of exam results.
The rest of the paper is structured as follows. The
next section introduces related didactical and
economic basics. The background of the case study
and the research methodology are described in the
third section. The fourth section outlines the
evaluation tools before the results are presented. The
paper ends with conclusions, limitations, and
recommendations for future research.
2. Related work, didactical and economic
context
The goal of our research was to improve a large
scale course with regards to learning outcomes and
learner satisfaction with economically reasonable
measures. This required converging the two different
and sometimes conflicting perspectives of
economically oriented educational and pedagogical
control [26]. The next paragraphs briefly explain
didactical basics, as well as the economic challenges,
from a service delivery perspective.
2.1 Understanding the learning process
At its core, learning is a change of the state of the
human being expressed in a change in the
individual’s behavior based on experiences [13].
When the process of learning is supported by
information and communication technologies, we
talk about electronic learning or the more common
“eLearning” [20].
Experiences that trigger learning processes are
gained through different ways of interaction, e.g.,
with content or between learners [35]. Learners
actively look for meaning in things [5] and construct
their own understanding of the world. Knowledge is
viable as long as it fits with experienced reality.
When it comes to testing and adapting one’s
knowledge, social interaction plays an important role:
learners need the opportunity to discuss and reflect
on their opinions [1, 6, 14]. In this way, knowledge is
co-constructed through a permanent process of
divergence and convergence, where learners
iteratively adjust their understanding.
While this principle describes the learning process
at its core, learners differ much with regard to their
experiences, personal needs, desires, and preferred
individual learning styles [23] e.g., a more passive
consumption of knowledge, visual representations, or
active engagement in real world scenarios. Offering
eLearning modules helps learners work at their own
pace and choose the way in which they want to
interact with contents [19]. Thus, a learning scenario
should offer different sources of knowledge
acquisition and different ways of interaction with
peers, the instructor, and the learning materials.
2.2 Teaching as service delivery
While a learning arrangement is planned with
regard to didactical aspects at its core, one also has to
keep in mind that delivering a lecture is a resource
demanding service. When taking into account the
different phases of the service delivery [9], resources
are needed for the potential dimension as well as the
factor integration. First, instructors have to keep
available contents, methods, technologies, and
everything else they need for delivering their course.
Second, the delivery process itself demands that
instructors spend their or their team’s time on the
lessons themselves, adjusting materials, and
supporting or assessing students.
This leads to the question of which parts of the
teaching service can now be standardized, automated
or delegated to the customer (the learner) in order to
save resources without reducing the perceived service
quality. While new technology does allow offsetting
face-to-face lessons, service research clearly states
that face-to-face contacts are a crucial part of service
delivery, impacting also perceived quality [11].
These thoughts are supported by learning theory,
which stresses the importance of social processes.
However, when splitting the teaching process into
several parts, some sub-processes that demand most
resources can be automated or delegated to the
learner [37]. For example, learners spend a lot of time
with self-regulated learning. They recapitulate
contents of the lecture with the help of the script or
whatever learning materials are offered. For this
reason, cost efficient ways of creating high quality
learning materials are very beneficial for students.
3. Background of the Action Research
project
The research presented here was conducted at a
German University in the course “Introduction to
Business Informatics” aimed mostly at students of
business administration and economics. The course is
offered each semester with 150 to 300 participants. It
was planned and implemented as a traditional frontal
lecture supplemented by small-class tutorials with
graduate assistants. Students are graded by the scores
of a written exam at the end of a semester. We
wanted to increase learner satisfaction, as well as
learning outcomes, without using too many (or at best
no) additional resources. To achieve our goals, we
went through the typical steps of action research.
3.1 Methodological approach for the Action
Research project
Typically, action research focuses on evaluating
measures in real life situations to solve specific
problems, and is divided into five specific steps
which are taken iteratively [34]: diagnosing, action
planning, action taking, evaluation, and specifying
learning. We followed these steps and tried to
improve our concepts and tools after each semester,
before restarting the cycle from the beginning. With
regard to the case study, the five steps included the
following.
Diagnosing: First, we analyzed our current
situation and the problem/s we were facing. In
general, the large scale lecture was lacking
interactivity and individuality. In addition, we were
expecting increasing numbers of students, and thus
had to look for ways of raising the level of interaction
and individuality without needing to use more
resources. As a first step, we used methods from the
Service Engineering domain, such as Service
Blueprinting [37], to identify critical parts of the
learning and teaching process. We identified two
main challenges. First, interaction is one key element
for learning and learners’ satisfaction. There are
different types of interaction, the most important
being learner-learner, learner-instructor, and learner-
content interaction [27, 35]. In the current situation,
there was nearly no learner-learner interaction during
the lecture and just a slight level of learner-instructor
interaction when the instructor asked some questions.
In addition, learner-content interaction was mostly
focused on a traditional script, offering not much
diversity or visual appeal. Second, a major challenge
in teaching is addressing the students’ different
learning speeds and styles. Students differ in the
sensory channels through which they best perceive
information and also their preferred learning method:
inductive or deductive [10]. Thus, we also wanted to
offer a wide range of high quality learning materials
that students could freely choose from for self-paced
learning.
Action Planning: The new learning materials
should be more appealing, highly interactive, and
diverse. In this context eLearning materials offer the
advantage of self-paced learning as well as
stimulating different ways of perception. As stated by
other authors, we hoped that a higher degree of
learner control might lead to better learning outcomes
[32]. But because development of eLearning
materials takes a lot of time, we decided to involve
our students in the creation process based on the idea
of “learning through teaching” [17]. According to
this idea, learning can be supported by students
switching to the role of a teacher. A student acting as
a tutor has to reflect deeply on the content he wants
to teach. In addition, explaining things to others
requires skills with regard to creativity, self-
confidence, and communication. We adapted this
idea onto the creation of eLearning materials within
the framework of a seminar on “Web Engineering.”
After the “Introduction to Business Informatics”
course, students could choose to take part in this
course. Since they were supposed to learn the basics
of Web Engineering with Flash, this course was
ideally suited for allowing students to create
interactive eLearning modules. This measure will
thus be the focus of our paper, explained in greater
detail in the next section.
These Web Based Trainings (WBTs) were
supplemented by videos of the lecture offered as live
streams, as well as records stored on the learning
management system (LMS). While the videos were
not user generated, they offered the possibility of
creating learning materials quickly and at low cost.
In order to raise the interaction in the classroom,
two small assignments were integrated into the 90
minute lecture. At two times in each lecture, students
were asked to create “true-false-items” with regard to
the contents they had just learned. These items were
exchanged, discussed, and revised between the
students, and then delivered to the instructor. The
best items were used in the exam at the end of the
semester. The idea behind this is that students’
attention declines during a teacher centered lecture,
and active breaks help them to re-engage as well as
reflect on what they have just learned [29].
Action Taking: Delivery of the lecture as a video
stream was easily implemented, thanks to the central
LMS of the university. The assignments where
students created their own “true-false-items” were
also easily integrated into the course, since no
additional material was needed for this. The
challenging task was to get students from the Web
Engineering lecture to create high quality eLearning
modules that would deliver a real benefit to their
peers. The first eight WBTs were used in the first
semester, and then supplemented or exchanged by
further WBTs during the next semesters. The content
production process is outlined in detail in the next
section.
Evaluation: Evaluation focused on the students’
satisfaction as well as learning outcomes. Data were
collected from two sources: 1) online questionnaires
that students filled out at the end of the semester
immediately before the exam, and 2) the results of
the exams with regard to the score students had
achieved were compared in order to measure learning
outcomes.
Specifying Learning: After the evaluation,
measures were taken again in order to improve the
learning scenario further, e.g., the creation process of
the WBTs was adjusted in order to generate more
appealing and interactive eLearning modules for the
next semester.
3.2 Integrating students into content
production
3.2.1 Creating true-false-items during the
lecture
Students were involved in the process of content
production in two different ways. First, as mentioned,
they created true-false-items during the lecture, a
method entitled: “Co-Create Your Exam.” During
traditional classroom teaching, there is the tendency
for students’ attention to drop from time to time;
however, small breaks of about two to three minutes
are helpful to regain their attention. Olmsted [29]
suggests several types of “active breaks,” including
assessments or feedback. We wanted to focus on one
method so that students would not have to adapt to
different ones. However, we chose to combine the
assessment with the feedback. By letting students
create their own true-false-questions, the instructor
would get insights into what content students
struggled with or had already understood
(considering that students would probably focus only
on stuff they felt they had understood well). On the
other hand, by having students pass the questions to
their neighbors and solving them, there was a sort of
peer assessment which can foster learning by
teaching since it supports critical discussions between
students [36].
The implementation was very simple. After about
30 to 60 minutes, the instructor advised the students
to take a sheet and write down three statements
dealing with the content they just heard. At least one
of those statements had to be true and one false. After
a few minutes, students exchanged their sheets and
tried to answer their peer’s questions. The students
also had the opportunity to discuss answers or adjust
what they had written. The sheets were then collected
by the instructor, who would later select some of the
items as part of the final exam, and some were
uploaded on the LMS in electronic format.
In this way, one of the core principles of learning,
the social exchange, is introduced to the large scale
lecture. In addition, the process does not demand any
resources from the instructor, except choosing the
best items for the exam. Creating similar items would
surely take at least the same amount of time. Apart
from that, the created items could even be used as
supplementary learning materials in future semesters.
3.2.2 Creation of web based trainings
The creation of the WBTs was a very complex
process. A WBT is an encapsulated eLearning
module that can be used for self-regulated learning. It
offers all the advantages already discussed in the
background section. Students can learn at their own
pace, anytime, anywhere. The materials can also
stimulate different senses by use of video, audio, and
textual information, which can offer much more
interaction than do paper based materials. The
dynamics of the media also allowed for animations
that could aid understanding of complex processes.
But all these advantages are in effect only if the
materials are created properly. The materials need to
take advantage of the potentials of the medium, be
didactically sound, and have flawless contents.
The underlying concept of what we did is called
learning through teaching [17, 16]. When being
responsible for explaining things to their peers,
students are expected to engage at a deeper level and
improve their communication skills. But the students
need to be guided in this. The seminar “Web
Engineering” was thus extended with some didactical
basics. In addition, the students received guidelines
on how a WBT could be structured, which common
mistakes should be avoided, or how interactive
assignments could be implemented. They also
received a standardized layout for the WBT, along
with several examples of animations or assignments
they could adjust and use in their own modules. In
this manner, we could guarantee that all trainings had
the same look and feel, as well as basic common
features.
The general topics of the WBTs were
predetermined by the instructor, and all dealt with the
lecture “Introduction to Business Informatics.”
However, the students were offered a great deal of
freedom in setting the focus and creating new kinds
of animations or assignments.
The process of implementation of WBTs started
with students creating a storyboard. This was
reviewed by an instructor with regard to the learning
goals and first impressions. A second storyboard was
then developed and again reviewed. Based on this
final review, the students then implemented the
actual WBT.
Further, students of the Web Engineering course
were introduced to the technical basics of Flash and
Web Programming in several lessons. The course
was based on much self-regulated learning. The
instructors were asked to offer their students
considerable freedom when creating the WBTs, since
too much instruction would not lead to greater
motivation and deeper learning processes [2]. To
facilitate the development process, the instructor
regularly posted hints and templates, or answered
questions, thus showing constant presence [31, 22].
Apart from general tips and advice, the instructors
always offered direct 1:1 feedback on questions or
products such as storyboards, as direct feedback is
crucial for motivating learners [21].
This process was iteratively improved by
instructors. For example, many students faced
problems when planning their WBTs addressing
learning goals that were higher than just pure
knowledge acquisition [3]. Even when it came to
teaching modeling techniques - a basic skill in
Business Informatics - students often relied on simple
multiple choice tests as assignments. After realizing
the problem, the instructors offered a new template
for implementing modeling tasks into the WBTs, thus
enabling learners to compose whole diagrams on
their own, instead of just answering simple questions.
This is just one example of how the content
production process was gradually improved. During
the fourth semester, part of the feedback that the
instructors gave on the WBTs was exchanged for
peer-reviews. In this way, even more responsibility
was later delegated to students. During the feedback
phase students could take a look at other’s WBTs and
point at good ideas or flaws. Peer assessments have
proven to be quite reliable in identifying strengths
and weaknesses of students’ learning products [36].
The peer review process was also scaffolded by a
guideline given to the students that was intended to
encourage them not just to identify their peers’
mistakes, but also to offer open-ended and suggestive
feedback, as implied by Topping [36].
The final WBTs were assessed by instructors who
chose which ones were suitable for being used in the
lecture. These WBTs were then finalized by
instructors by identifying and deleting the last flaws
concerning content or technical implementation.
The content was kept up-to-date in two ways.
Smaller changes were implemented by the teaching
staff and the research assistants. If there was need for
major changes in a WBT, it was completely
exchanged with a newer one. This was possible due
to the fact that the general topics of the WBTs were
determined by the teaching staff, and each semester
new WBTs were created by the students.
4. Evaluation
4.1 Methodological Aspects
Deriving statements on the efficiency of the
different actions aimed at increasing learning
outcomes and learner satisfaction requires defined
reference measures not only for the success of each
single action but also for all combined actions (in this
case, measures for the success of the entire blended
learning course). However, in field research it is
difficult to define success measures or cause-effect-
chains for single actions / interventions, as these
affect the users both independently and in
combination with other actions; therefore, their
impact cannot be regarded in isolation. Often it is
only possible to measure and evaluate the sum of
several actions and influences as a whole.
For the proposed formative evaluation, we used
self-reporting data sources (online surveys).
Choosing an online survey as a method to collect data
poses some important consequences for the process
of the investigation and for the design of the
questionnaire (for further details see [12, 4]). Some
basic problems generally occur when conducting
online surveys. The sample is self-selected, since
evaluation participation is voluntary, and therefore
cannot be regarded as being representative; thus,
statements about “non-participants” cannot be made
[18]. The questionnaire used in this study was
structured, tested, and consequently adapted to the
needs of the specific targeted audiences. For this
purpose, a pretest, followed by a discussion with the
test persons, was conducted. In addition, an online-
pretest was carried out, which tested the content and
the functionality of the questionnaire.
4.2 Evaluating students’ satisfaction
We wanted the students to be satisfied with their
learning experience, and for this reason, we used the
questionnaire to keep track of their levels of
satisfaction. Since the course at its core was a typical
large scale lecture, part of the items for the
questionnaire were developed with regard to
traditional dimensions of learner satisfaction [7].
These six dimensions focused mainly on the
instructor and course as a whole. The dimensions
were: skill level of the instructor, instructor’s rapport
(empathy and friendliness), course structure, course
difficulty, interaction, and feedback. The last
dimension referred to whether the instructor told
students about their learning progress or gave them
feedback on their task performance. Since this was a
large scale lecture demanding a lot of self-regulated
learning without direct feedback from the instructor,
this dimension was excluded from the questionnaire.
The second part of the questionnaire was
supplemented by dimensions taken from eLearning
success models. Since the course offered a great deal
of eLearning materials, we wanted to take the
eLearning parts into account separately. However,
there were some technological aspects that could not
be changed by the instructor. For example, the LMS
or technical infrastructure could also be evaluated,
but since both were maintained by the University
itself, there was no possibility of improving these
factors for the instructor. Thus, only factors that the
instructor was in charge of were taken into account.
These factors focused on the different virtual learning
materials and included their technological quality
(like the look and feel), their flexibility, and their
usefulness in the learning process [33, 30].
All items were rated on a Likert scale ranging
from 1 (full agreement) to 5 (no agreement). Table 1
provides an overview of the items.
Table 1. Satisfaction questionnaire items
Course – General
(1) Satisfaction with course in general (2) Satisfaction
compared to other lectures (3) Innovativeness (4) Overall
recommendation
Course – Structure (overall)
(1) Overall structure (2) Contents in general (3) Relevance
for praxis (4) Learning materials in general (5) Clarity of
performance requirements
Instructor – Skill Level
(1) Satisfaction with instructor in general (2) Knowledge of
the instructor (3) Helpful explanations of the instructor (4)
Instructor’s preparation (5) Clarity of question answering
Instructor – Rapport
(1) Instructor sparks student (2) Instructor is pleasant (3)
Instructor raises interest (4) Attainable outside classroom
Course – Difficulty
(1) Amount of content compared to time at disposal (2)
Difficulty of course (3) Amount of content compared to
effort (4) Effort put in the course
Interaction
(1) Interactivity (2) Participation in the course
eLearning materials – Technology
(1) Visual appearance of WBTs (2) Visual appearance of
videos (3) Interactivity of WBTs
eLearning materials – Flexibility
(1) WBTs supporting individual learning speed (2) WBTs
supporting individual learning style (3) Videos supporting
individual learning speed (4) Videos supporting individual
learning style
eLearning materials – Usefulness
(1) WBTs helpful in recapitulating the lecture (2) WBTs as
useful supplement of materials (3) Usage of WBTs (4)
Videos helpful in recapitulating the lecture (5) Usage of
Videos (6) Skype consultation hour as useful supplement
Affective and motivational outcomes
(1) General interest in topics of the course (2) Lecture
enhancing interest in the topic
4.3 Evaluating learning outcomes
The evaluation of learning outcomes is a complex
field. Kraiger et al. (1993) note that there are three
different kinds of learning outcomes: cognitive
outcomes (such as factual knowledge), skill-based
outcomes (such as proceduralization capabilities),
and affective outcomes (such as attitudinal or
motivational ones). Measurement depends on which
kinds of outcomes are addressed [24].
Cognitive outcomes can be measured by power
tests such as multiple choice tests. In this research,
part of the final exam was implemented as a multiple
choice test in order to measure factual knowledge
acquisition.
But higher learning outcomes such as specific
skills demand the learner not only to reproduce
knowledge but also to take the right steps in solving a
task and deciding which learned concepts are
appropriate in a given situation. These skills can be
measured through methods such as observations or
hands-on testing. The latter means that a given task is
evaluated with regard to whether a learner chooses
and performs the right steps to solve the problem. In
our case, this is exactly what an exam does. For
example, in the final exam the students were required
to develop a model such as an entity-relationship-
diagram to a given problem description. They now
had to perform several steps: identifying the relevant
entities, adding the right attributes and relations, and
refining the diagram by using special constructs such
as generalizations in order to avoid redundancy. All
these steps were assessed, and scores were given with
regard to the degree of task completion. Thus, skill-
based outcomes are also part of the measurement of
the exam.
Attitudinal and motivational outcomes refer to
states of individuals that will affect how a person
acts. For example, if a learner gains more self-
confidence throughout a course, he might be able to
act faster and be more self-aware in future situations.
However, these “softer” factors should not be
confused with the motivation that learners feel during
the course. Thinking that a course is interesting and
that the instructor is doing a good job are not learning
outcomes, but could be considered feedback on the
overall course quality. Attitudinal and learning
outcomes are usually measured through self-reports.
In our case, attitudinal and motivational outcomes
were not the focus of the research, but they were
addressed by two items of the satisfaction
questionnaire (see Table 1).
We chose exam scores as a measure for learning
success. Since the exams had a similar proportion of
content in each semester (with around one third
multiple choice, one third modeling, and one third
open questions), the scores (taken in points, not
grades) were deemed to be a reasonably objective
measurement tool for learning outcomes.
5. Presentation of Results
The results of our study are presented in this section.
Since there are four different groups of students (one
for each semester), results are chronologically related
to groups one to four, with group one being the first
group (winter term of 2008/2009) and group four the
last (summer term 2010).
5.1 Results with regard to learning outcomes
The learning outcomes were measured by
comparing mean results of the exam at the end of the
semester. Students could reach a score from 0 to 82
points. Table 2 shows the actual results, together with
the total number of WBTs available in each semester.
A Kruskal-Wallis test of variance was conducted in
SPSS, as well as a Median-test. Both tests show that
differences are highly significant, with p < .001.
Table 2. Comparison of exam results
Group N Mean
score Variance* Median**
Total
WBTs
in use
1 155 34.365 15.621 35.500 0
2 168 33.832 14.853 36.125 9
3 178 38.479 13.585 40.750 18
4 262 42.090 14.409 43.500 18
All 763 37.860 14.965 39.500
* Significant with p < .001 (Kruskal-Wallis Test)
** Significant with p < .001 (Median Test)
The mean score at the beginning was around 34
points. While the scores in the second semester were
a little lower, they increased in semester three and
four by up to 42 points. In the first semester, there
were no WBTs in use, since development started only
at this point. In the second semester, the first nine
student-produced WBTs were offered. However, they
were not directly integrated into the LMS but placed
on a separate website. Since integration did not work
properly, there were issues such as layout problems
(very small frames, e.g.).
These issues were overcome in the next semester
when the total number of WBTs was increased to 18
that were directly placed on the LMS with adjustable
screen sizes. While the results do not allow
concluding any causal relations, it is still salient that
in the first two semesters, when no WBTs were in use
or they covered only some parts of the lecture,
students performed the worst. At the time when 18
WBTs were used (which covered nearly all contents
of the lecture) and properly implemented, student
performance clearly increased.
The hypothesis that the increased performance
can at least be partially ascribed to WBTs is
supported by the results of the questionnaire, where
many students reported that the materials were a
beneficial supplement to the script. Since WBTs were
not part of the learning materials at the beginning, the
question items with regard to the eLearning quality
were only implemented in the questionnaires of the
third and fourth semesters. Table 3 presents the
results of the eLearning quality items taken from the
learning service satisfaction questionnaire (explained
in detail in the next paragraph) of the fourth semester.
Table 3. Satisfaction with eLearning
materials
Dim eLearning element Mean Variance
Techn.
appearance wbts 1.94 .671
appearance videos 1.92 .834
interaction wbts 2.08 .786
Flexibility
wbt - learning speed 2.17 1,050
wbt - learning style 2.26 1.025
video - learning speed 1.83 .813
video - learning style 1.95 .979
Usefulness
wbt - recapitulation 1.91 .983
wbt - useful 1.94 .998
wbt - usage 2.46 1.101
videos - recapitulation 1.68 .845
videos - usage 1.91 1.107
skype - useful 2.04 1.211
The results in general can be considered to be
quite good. The students rated the items on a Likert
scale ranging from 1 (full agreement) to 5 (no
agreement). Thus, a rating of 1.94 for the item “The
WBTs are a useful supplement of the other learning
materials” indeed seems promising. The
recapitulation of contents, as well as the visual
appearance and interactivity, reached similar results.
This seems to support the theory that the eLearning
materials were actually very popular and supported
better learning outcomes. The lowest scores were
reached by the items “WBTs support personal
learning style” and “actual usage of WBTs.” These
items show quite high variances. This might hint at
the fact that there are indeed different types of
learners and learning styles, some of which value the
WBTs, while others make less use of them.
5.2 Results with regard to satisfaction
Student satisfaction was measured with an online
questionnaire. It was conducted at the end of the
semester just before the exam, and thus participation
was voluntary. The results (mean values) are shown
in Table 4. Since our measures were iteratively
improved and thus showed the strongest effects in the
last semester, we also calculated the percentile
increase (or decrease) of an item by comparing the
fourth and the first semester. We also conducted a t-
test of the variance and the medians of the different
question items, comparing these semesters in order to
check whether the increases were significant. Items
that increased significantly are marked in Table 4.
Details of the t-test are shown in Table 5 (please note
that due to constraints of space, only factors that
increased by nearly 10 percent or more are shown in
the table).
Table 4. Results of satisfaction questionnaire
Mean of Group Incr.
(%)
Dimension Item 1 2 3 4
Course
overall
satisfaction 2.7 2.4 2.6 2.4 9.6
comparison * 2.8 2.5 2.7 2.5 11.3
innovativeness 2.3 2.2 2.1 2.0 11.8
recommendation 2.6 2.3 2.4 2.4 7.6
Course -
structure
structure * 2.5 2.2 2.2 2.1 14.9
content * 2.9 2.7 2.8 2.6 10.1
relevance 2.5 2.4 2.5 2.5 -1.4
materials 2.4 2.3 2.1 2.1 11.9
transparency 2.6 2.7 2.9 2.6 1.7
Instructor
Skill level
instructor 2.5 2.2 2.2 2.2 9.6
knowledge 1.3 1.3 1.2 1.2 4.5
explanations * 2.2 1.9 1.9 1.9 14.5
preparations 1.4 1.4 1.4 1.4 5.0
answers 1.9 1.7 1.7 1.7 10.9
Instructor
Rapport
enthusiasm * 2.8 2.6 2.5 2.4 13.8
raising interest * 2.8 2.4 2.5 2.4 16.0
pleasant 2.4 2.1 2.2 2.2 6.4
attainability 2.2 2.0 2.2 2.2 -1.9
Difficulty
difficulty 3.2 2.9 3.4 3.0 5.4
ratio cont./eff. 3.3 3.2 3.6 3.2 4.6
ratio cont./time 3.5 3.3 3.4 3.3 5.7
effort * 2.4 2.3 2.3 2.2 10.6
Interaction interactivity 2.5 2.5 2.3 2.3 9.1
participation 2.3 2.4 2.6 2.3 0.6
Affective &
Motiv.
interest 2.8 2.6 2.7 2.5 8.6
raising interest 3.1 2.8 2.9 2.7 11.5
* Significant increase from semester 1 to 4
Many of the factors that we addressed directly
(such as overall satisfaction, satisfaction with
materials, contents, and interactivity) increased by
about 0.3 points (very roughly estimated, coming to
an increase of about 10%). To check these increases
for significance, the t-test conducted in SPSS first
calculates the significance of the hypothesis that the
variances of the variables are homogeneous (Table 5,
first column). Next, two t-tests for the mean values
were conducted – one for the case where the
probability of variance homogeneity was not
significant (first line of each item) and one for the
other case (second line). The results of the two-tailed
t-test of the means are listed for both cases.
Statistically significant factors include the overall
satisfaction compared to other lectures, satisfaction
with content, explanations of the instructor, the
degree to which the instructor managed to raise
enthusiasm and further interest in the topic, and the
effort students felt they had put into the course. Two
of the most important items for our research, namely,
the overall satisfaction and satisfaction with the
learning materials, have not increased significantly,
but they are at least nearly at a significant level (.077
and .069). Similar results can be identified for the
perceived interactivity (.089). Taking into account
both the percentile increase in satisfaction with the
different aspects of the course, as well as results from
the t-test, we assume that the supplementary
materials indeed helped increase learners’
satisfaction.
6. Discussion of results and limitations
In this paper we presented a concept on how to
integrate students into the process of learning content
production using IT tools and concepts. Students
were encouraged to create true-false-items as part of
the final exam. In a Web Engineering seminar they
also created WBTs that were used as supplementary
learning materials in the large scale lecture
“Introduction to Business Informatics,” thus making
students co-designers of WBTs for their peers.
Content creation was implemented as a co-
construction process between the students and
instructors. The WBTs were then used in the lecture,
and the questionnaire as well as exam results were
gathered over four semesters. Results show a
significant increase in learners’ satisfaction as well as
learning outcomes. We assume that these results are
significantly caused by the new forms of interaction
and implementation of the WBTs. Our assumptions
are supported by the online questionnaires, where
most students stated that the WBTs indeed were a
useful supplementation to other materials.
Table 5. Overview of significant results
Item Test of
Variance T-Test of Means
Sig. Sig.
(biv.)
Mean
Diff.
Stand.
Error of
Diff.
satisfaction .042 .086 .254 .147
.077 .254 .143
comparison .864 .032 * .312 .144
.031 .312 .143
innovativeness .127 .075 .273 .152
.072 .273 .151
structure .008 .014 .369 .149
.011 * .369 .144
content .511 .043 * .296 .146
.039 .296 .143
materials .068 .073 .282 .156
.069 .282 .154
instructor .078 .164 .238 .170
.154 .238 .166
explanations .006 .058 .317 .166
.049 * .317 .160
answers .288 .127 .213 .139
.114 .213 .134
enthusiasm .408 .033 * .391 .182
.031 .391 .180
raising interest .244 .011 * .455 .177
.010 .455 .174
effort .031 .054 .258 .133
.047 * .258 .129
interactivity .015 .100 .227 .137
.089 .227 .133
raising interest .467 .059 .351 .185
.057 .351 .183
* Significant with p < .05
There are, however, several limitations due to the
nature of an action research approach. Working with
real students in complex learning arrangements
means that it is very difficult to identify causal
relations. As we compare learning scenarios that are
influenced by countless different factors, the usage of
single tools or methods can hardly explain the
outcomes on their own. For this reason, we cannot
prove causal relations from the data thus far, as we
cannot strictly exclude the influence of factors that
were not addressed in this paper directly. Surely, the
general approaches of the instructor and his staff to
improve the lecture and the script will also have
influenced the outcomes.
In addition, measuring learning outcomes by
exam results is problematic, since the exams changed
from semester to semester. Although the structure
and rating scheme stayed similar, it cannot be
guaranteed that the difficulty and rating were the
same each semester.
Still, the significant increases strongly hint at the
fact that the concept of co-construction does work.
This result is important for researchers in the domain
of pedagogy who are looking for new ways of learner
integration and self-regulated learning. In addition,
the methods of co-construction described in this
paper can also be adapted by practitioners in their
own lectures in order to create valuable content.
There is still much need for further improvements
of the processes described. With regard to Web 2.0
technologies and trends such as Open Innovation, it
seems reasonable to enhance the content creation and
review process even more by use of these methods
and tools. Also, raising interaction in the classroom
with the use of mobile devices which allow digital
content production seems very promising. Thus, we
will develop our course further with the help of Web
2.0 and mobile devices in order to create a stronger
sense of community and collaboration.
7. References
[1] M. Alavi, G. Marakas and Y. Yoo, "A Comparative
Study of Distributed Learning Environments on Learning
Outcomes", Information Systems Research, 13 (2002), pp.
404-415.
[2] H. An, S. Shin and K. Lim, "The effects of different
instructor facilitation approaches on students' interactions
during asynchronous online discussions", Computers &
Education, 53 (2009), pp. 749-760.
[3] L. W. Anderson, D. R. Krathwohl, P. W. Airasian, K.
A. Cruikshank, R. E. Mayer, P. Pintrich, J. Raths and M.
Wittrock, "A taxonomy for learning, teaching, and
assessing: A revision of Bloom's taxonomy of educational
objectives", (2000).
[4] B. Bantinic, K. Moser and B. Puhle, Der WWW-
Fragebogengenerator, in W. Bandilla, B. Bantinic and L.
Graef, eds., Online Research - Methoden, Anwendungen
und Ergebnisse, Verlag fuer Psychologie Dr. C.J. Hogrefe,
Goettingen, 1999, pp. 93 - 102.
[5] G. M. Bodner, "Constructivism - A Theory Of
Knowledge", Journal of Chemical Education, 63 (1986),
pp. 873-878.
[6] E. Chernobilsky, A. Nagarajan and C. E. Hmelo-Silver,
Problem-based learning online: multiple perspectives on
collaborative knowledge construction, Proceedings of th
2005 conference on Computer support for collaborative
learning: learning 2005: the next 10 years!, International
Society of the Learning Sciences, Taipei, Taiwan, 2005.
[7] P. A. Cohen, "Student ratings of instruction and student
achievement: A meta-analysis of multisection validity
studies", Review of Educational Research, 51 (1981), pp.
281-309.
[8] N. W. Coppola, S. R. Hiltz and N. Rotter, Becoming a
virtual professor: pedagogical roles and ALN, System
Sciences, 2001. Proceedings of the 34th Annual Hawaii
International Conference on, 2001, pp. 1-10.
[9] A. Donabedian, "Evaluating the Quality of Medical
Care", The Milbank Memorial Fund Quarterly, 44 (1966),
pp. 166-206.
[10] R. M. Felder and L. K. Silverman, "Learning and
teaching styles in engineering education", Engineering
education, 78 (1988), pp. 674-681.
[11] J. A. Fitzsimmons and M. J. Fitzsimmons, Service
management : operations, strategy, and information
technology, McGraw-Hill/Irwin, Boston, 2005.
[12] A. Gadeib, Ansprueche und Entwicklung eines
Systems zur Befragung ueber das World Wide Web, in W.
Bandilla, B. Bantinic and L. Graef, eds., Online Research -
Methoden, Anwendungen und Ergebnisse, Verlag fuer
Psychologie Dr. C.J. Hogrefe, Goettingen, 1999, pp. 103 -
111.
[13] R. M. Gagne, "Learning outcomes and their effects:
Useful categories of human performance", American
Psychologist, 39 (1984).
[14] I. Giannoukos, I. Lykourentzou, G. Mpardis, V.
Nikolopoulos, V. Loumos and E. Kayafas, Collaborative e-
learning environments enhanced by wiki technologies,
Proceedings of the 1st international conference on
PErvasive Technologies Related to Assistive
Environments, ACM, Athens, Greece, 2008.
[15] P. Goodyear, G. Salmon, J. Spector, C. Steeples and S.
Tickner, "Competences for online teaching: A special
report", Educational Technology Research and
Development, 49 (2001), pp. 65-72.
[16] K. Goto and J. Schneider, "Learning through
Teaching: Challenges and Opportunities in Facilitating
Student Learning in Food Science and Nutrition by Using
the Interteaching Approach", Journal of Food Science
Education, 9 (2010), pp. 31-35.
[17] J. Grzega and M. Schöner, "The didactic model LdL
(Lernen durch Lehren) as a way of preparing students for
communication in a knowledge society", Journal of
Education for Teaching, 34 (2008), pp. 167 - 175.
[18] P. Hauptmanns, Grenzen und Chancen von
quantitativen Befragungen mit Hilfe des Internet, in W.
Bandilla, B. Bantinic and L. Graef, eds., Online Research -
Methoden, Anwendungen und Ergebnisse, Verlag fuer
Psychologie Dr. C.J. Hogrefe, Goettingen, 1999, pp. 21 -
38.
[19] P. J. H. Hu, W. Hui, T. H. K. Clark and K. Y. Tam,
"Technology-assisted learning and learning style: A
longitudinal field experiment", IEEE Transactions on
Systems Man and Cybernetics Part a-Systems and Humans,
37 (2007), pp. 1099-1112.
[20] E. Kahiigi, L. Ekenberg and M. Hansson, Exploring
the e-Learning State of art, Conference on E-Learning,
Academic Conferences Limited, 2007, pp. 349-368.
[21] F. Ke, "Examining online teaching, cognitive, and
social presence for adult students", Comput. Educ., 55
(2010), pp. 808-820.
[22] K. P. King, "Identifying success in online teacher
education and professional development", The Internet and
Higher Education, 5 (2002), pp. 231-246.
[23] A. Y. Kolb and D. A. Kolb, "Learning styles and
learning spaces: Enhancing experiential learning in higher
education", Academy of Management Learning &
Education, 4 (2005), pp. 193-212.
[24] K. Kraiger , J. K. Ford and E. Salas, "Application of
cognitive, skill-based, and affective theories of learning
outcomes to new methods of training evaluation", Journal
of Applied Psychology (1993), pp. 311-328.
[25] Kultusministerkonferenz, Prognose der
Studienanfänger, Studierenden und Hochschulabsolventen
bis 2020, Bonn, 2005.
[26] M. Mohr and H. Krcmar, Bildungscontrolling: State of
the Art und Bedeutung für die IT-Qualifizierung,
Technische Universität München, Lehrstuhl für
Wirtschaftsinformatik, Garching b. München, 2005.
[27] M. G. Moore, "Editorial: Three types of interaction",
American Journal of Distance Education, 3 (1989), pp. 1-7.
[28] OECD, The Sources of Economic Growth in OECD
Countries, Organisation for Economic Co-Operation and
Development, 2003.
[29] J. Olmsted III, "The mid-lecture break: When less is
more", Journal of Chemical Education, 76 (1999), pp. 525.
[30] S. Ozkan and R. Koseler, "Multi-dimensional students'
evaluation of e-learning systems in the higher education
context: An empirical investigation", Computers &
Education, 53 (2009), pp. 1285-1296.
[31] S. Peacock and J. Hooper, "E-learning in
physiotherapy education", Physiotherapy, 93 (2007), pp.
218-228.
[32] G. Piccoli, R. Ahmad and B. Ives, Web-BasedVirtual
Learning Environments: A Research Framework And A
Preliminary Assessment Of Effectiveness In Basic IT Skills
Training, MIS Quarterly, MIS Quarterly & The Society for
Information Management, 2001, pp. 401-426.
[33] P.-C. Sun, R. J. Tsai, G. Finger, Y.-Y. Chen and D.
Yeh, "What drives a successful e-Learning? An empirical
investigation of the critical factors influencing learner
satisfaction", Computers & Education, 50 (2008), pp. 1183-
1202.
[34] G. I. Susman and R. D. Evered, "An assessment of the
scientific merits of action research", Administrative science
quarterly, 23 (1978), pp. 582-603.
[35] V. Thurmond and K. Wambach, "Understanding
interactions in distance education: A review of the
literature", International Journal of Instructional
Technology and Distance Learning (2004).
[36] K. Topping, "Peer assessment between students in
colleges and universities", Review of Educational
Research, 68 (1998), pp. 249-276.
[37] R. Wegener, P. Menschner and J. M. Leimeister,
Analyse und Optimierung von Lehrdienstleistungen mittels
Service Blueprinting - Konzeption und erste empirische
Befunde, Multikonferenz Wirtschaftsinformatik (MKWI)
2010, Göttingen, 2010.
[38] Wissenschaftsrat, Empfehlungen zum arbeitsmarkt-
und demographiegerechten Ausbau des Hochschulsystems,
Wissenschaftsrat, Berlin, 2006.
... The chosen ID method selection and application depends on the educational need analysis that is the initial step of ID (Beetham & Sharpe, 2013;Raspopovic, Cvetanovic, & Jankulovic, 2016;Wegener & Leimeister, 2012). The scopes, target abilities, presentation types and evaluation to measure learning have to be determined in this stage (Raspopovic et al., 2016). ...
... ID has been executed by the educational technologists or instructional designers that should plan the designing of new environment and principles to teach (Cengizhan, 2007). Additionally, the subject specialists have to organize the information (Keengwe et al., 2014) and evaluate the suitability of presenting components in accordance with the learning outcomes (Desai & Mevada, 2015;Macleod & Sinclair, 2015;Paquette, 2014;Raspopovic et al., 2016;Wegener & Leimeister, 2012). The main responsibilities of educational technologists are to analyze education need and to determine the storyboard necessities. ...
... Moreover, they have an active role in the development phase to prepare the components -visual, audio, movie, fiction, integration (Raspopovic et al., 2016) in accordance with their sub-expertness. Because of this cooperation-based process, all of the communication channels have to be functional and, the evaluation phase has to be continued during each of the phases (Desai & Mevada, 2015;Paquette, 2014;Raspopovic et al., 2016;Wegener & Leimeister, 2012). This cycling and repetitive structure indicate that every material development process is the design -based research (Barab, 2006;Barab & Squire, 2004;Raspopovic et al., 2016;Wang & Hannafin, 2005). ...
Article
Full-text available
The purpose of this study is to demonstrate the most delicate parts of eLearning material development process and tools from the Instructional Design (ID) perspective. In this context, the birth lesson eLearning material modules have been progressed in accordance with the design - based research approach. The selected two modules have been completed by the project team - 3 instructional designer, 1 subject expert, 1 education technologist and 2 subject matter consultants- in 8 months. The ID process has been started with the education need analysis in order to determine the learners’ needs, learning outcomes and other expectations. The explanatory storyboard has to be offered to the educational technologists and the learning theories has to be determined by the designer. After the several revised, the themes and modules have been developed via several eLearning material development tools. The ID process consists several work packages for the team. Correspondingly, eLearning development is of toilsome process that has been included in planning, storyboard, communication between instructional designers and subject experts, and time. Also, it has to be designed with an expert team, learning theories and other scientific methods.
... It was distinct in allowing students to independently create learning resources in Moodle, which is more frequently used as a teaching delivery tool. This approach was embraced because researchers such as Wegener and Leimeister (2012) have shown that the creation of developing e-learning content does not depend on specialists and that we need to provide students with the 'processes and tools….in content creation while still providing a guarantee of high quality'. ...
... As Biggs and Tang (2011: 70) state 'technology can be used to enhance the dialogue between teacher and learner as new ways of engaging students in learning become available'. The evidence from the literature on co-creating technology enhanced learning with students suggests that there should be a 'new way' of developing learning which allows students to feel motivated and included (Wegener and Leimeister, 2012). The co-creators on this project fed back that they felt many benefits from taking part and developed new skills. ...
Article
Full-text available
Successful induction has been evidenced to strengthen students’ learning, engagement and feelings of belonging. Technology offers opportunities for enhancing the student induction experience, especially pre-arrival, but has been under-utilised. This article provides an evaluation of an online induction learning resource for pre-arrival students in the Faculty of Arts at the University of Warwick in 2019. There will be particular focus on the method of co-designing the resource with a group of current students. The article will demonstrate how online learning resources for pre-arrival students can support successful induction. It argues that co-designing digital student experience resources in collaboration with students aids the development of materials that students find engaging and that co-design has a range of benefits for staff and students who are involved in the process.
... Users who have so far been passive consumers are increasingly becoming 'prosumers' (Ritzer and Jurgenson 2010). They provide significant contributions to research and development of solutions (Zhao et al. 2009), add to collective intelligence (Wegener and Leimeister 2012) and provide valuable insights from often under-explored 'sticky' need information (Baldwin and von Hippel 2011). Information technology supports the user integration in innovation activities by various kinds of instruments, including online forums, marketplaces, design toolkits along with digital fabrication technologies like 3d-printing (Moeslein and Fritzsche 2017). ...
Conference Paper
Full-text available
Toolkits for user innovation and design democratize innovation by offering users a solution space to develop solutions that meet their diverse needs. Advancements in manufacturing and IS have given users space for creativity and innovation and the danger of overloading users with too many design decisions, especially when they also have unknown implicit needs. This design science study presents a toolkit that generates and recommends complete solutions to users. It identifies users' implicit needs with a critiquing technique, where users iteratively evaluate recommendations by visually inspecting and selecting matching solutions. It presents artifacts that make up the toolkit and its evaluation by comparing it with a traditional toolkit. The smart toolkit further expands innovation capabilities to society by learning the needs of non-expert users and enabling them with completed solutions instead of the traditional toolkit approach, where users follow a slower, manual, learning-by-doing search process through the needs and solution space.
... Students also feel more empowered and responsible for the learning process as they work toward the mastery of the material (O'Flaherty & Phillips, 2015). Other researchers have found several factors that are key in aiding student's retention such as the ability for students to transfer knowledge, consistency throughout the activity/course, social presence of faculty and peers (Boston & Ice, 2011), small-group assignments, having a strong instructor who is able carry out different facilitating roles, and interactive face-to-face meetings (Wegener & Leimeister, 2012). Each of these factors is evident in a well-designed flipped classroom. ...
Chapter
In 2007, Jonathan Bergmann and Aaron Sams, two secondary school chemistry teachers from Colorado, USA, began video recording their lectures and sharing those videos with students before the students came to class. “Flipping the classroom” has now grown in popularity in both K–12 and university settings across the world. This book is intended to provide examples from around the world as authors share original research and synthesize others’ research to explain the impact flipping the classroom has had across a wide variety of educational settings and audiences.
... Die gemeinsame Erstellung von Inhalten durch Lernende aus unterschiedlichen Sektoren kann nur erfolgreich sein, wenn die zu bearbeitende Fragestellung einen Rahmen aufspannt, die allen beteiligten Personen die Gelegenheit bietet, eine individuelle Lernerfahrung zu machen (STIGMAR, 2016). Die Auseinandersetzung mit den Fragestellungen erfolgt durch die Produktion von Inhalten, die nachgelagert von Dritten nutzbringend zum Lernen eingesetzt werden können sollen (WEGENER & LEIMEISTER, 2012). Somit ergeben sich drei Zielgruppen, die von der Erstellung der Inhalte profitieren sollen: ...
Preprint
Full-text available
SchülerInnen und Studierende können durch kollaborative Entwicklung von Lernmaterialien nicht nur ihr Verständnis des jeweiligen Themengebiets vertiefen, sondern erwerben auch Kompetenzen im Bereich der Wissensaufbereitung und - vermittlung. Durch das gemeinsame Arbeiten lernen SchülerInnen den Betrieb an Hochschulen niederschwellig kennen; Studierende werden für didaktische Fragestellungen sensibilisiert und sammeln Erfahrung in der Wissensvermittlung. Der Beitrag stellt ein Konzept zur Anleitung und Unterstützung von kollaborativer sektorenübergreifender Entwicklung von Lerninhalten vor und zeigt dieses an einem Pilotprojekt zur Erstellung von Unterlagen zum vorwissenschaftlichen Arbeiten.
... The second step was to analyse the co-creation model references mentioned in the "Co-creation process" section, as we wanted to focus on those that encourage HA characteristics. From this analysis we highlighted four models: [28], [32] , [35] and [40], because each of them allows the student to participate, to create real things, and to validate their creations. We realized that these models have similar phases and activities that we could re-organize to suit our own purposes. ...
... An example of co-creation, described in [9], seeks to enhance student satisfaction. This process has five steps, the first of which is "diagnosis", where the critical parts of the learning process are identified. ...
Article
Full-text available
The increasing digitalization and automatization in the manufacturing industry as well as the need to learn on the job has reinforced the need for much more granular learning, which has not yet impacted the design of learning materials. In this regard, granular learning concepts require situated learning materials to support self-directed learning in the workplace in a targeted manner. Co-creation approaches offer promising opportunities to support employees in the independent design of such situated learning materials. Using an action-design research (ADR) approach, we derived requirements from co-creation concepts and practice by conducting focus group workshops in manufacturing and vocational training schools to develop design principles for a co-creation system that supports employees through the creation process of work-process-related learning material. Consequently, we formulate four design principles for the design of a collaborative learning and qualification system for manufacturing. Using an innovative mixed methods approach, we validate these design principles and design features to demonstrate the success of the developed artifact. The results provide insights regarding the design of a co-creation system to support learners in the co-creation of learning material with the consideration of cognitive load (CL). Our study contributes to research and practice by proposing novel design principles for supporting employees in peer creation processes. Furthermore, our study reveals how co-creation systems can support the collaborative development of learning materials in the work process.
Article
The research evaluates a collaborative case study to co-create the curriculum for a tourism undergraduate module. In three course design team (CDT) meetings, students selected and discussed topics and learning styles. Qualitative analysis revealed that the students involved strove to be independent learners. They favoured active learning styles and ‘non-vocational’ aspects of tourism. However, university bureaucracy represents a block in the design process. The paper concludes by advocating a more equal relationship between students and tutors. This involves a shift in power relations, inverting traditional hierarchies in which teachers act as gatekeepers of knowledge and students are passive recipients.
Article
Full-text available
This paper reports the findings of a 1-year research project into the role of e-learning as a mechanism to support and enhance the learning environment for pre- and post-registration physiotherapists. The findings reveal tutor and student perceptions about what study entails, the anticipated respective roles of individuals in the learning process and how those individuals believe learning should occur when supported by e-learning in a tertiary education institution. Critical differences between the two groups of students, at different stages of their professional education, and their different uses of virtual learning environments are highlighted. This study raises some key issues that need to be addressed by educational institutions deploying e-learning in order to prepare students to engage with such a learning medium, which is likely to be unfamiliar to them at the outset of their undergraduate studies. In addition, physiotherapists need the skills, time and resources to regularly access and actively participate in the online environment. These points are essential if online communities such as interactiveCSP (www.interactivecsp.org.uk) are to be sustainable. Employers have a crucial role in promoting the professional development of staff by supporting such initiatives and ensuring that they are inculcated into an organisational culture which promotes the sharing of expertise and practice that is evidence based.
Article
The outcomes of learning are persistent states that make possible a variety of human performances. While learning results are specific to the task undertaken, learning investigators have sought to identify broader categories of learning outcomes in order to foresee to what extent their findings can be generalized. Five varieties of learning outcomes have been distinguished and appear to be widely accepted. These categories are intellectual skills (procedural knowledge), verbal information (declarative knowledge), cognitive strategies (executive control processes), motor skills, and attitudes. Each of these categories may be seen to encompass a broad variety of human activities. It is held that results indicating the effects on learning of most principal independent variables can be generalized within these categories but not between them. Five categories exist because (1) they differ as human performances, (2) the requirements for learning them are different despite the pervasiveness of such general conditions as continguity and reinforcement, and (3) the effects of learning differ. It is argued that these categories represent a functional middle ground and are well-suited as a basis for future research. (50 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
A definition and typology of peer assessment between students in higher education is proposed, and the theoretical underpinnings of the method are discussed. A review of the developing literature follows, including both process and outcome studies. This indicates that peer assessment is of adequate reliability and validity in a wide variety of applications. Peer assessment of writing and peer assessment using marks, grades, and tests have shown positive formative effects on student achievement and attitudes. These effects are as good as or better than the effects of teacher assessment. Evidence for such effects from other types of peer assessment (of presentation skills, group work or projects, and professional skills) is, as yet, more limited. Computer-assisted peer assessment is an emerging growth area. Important factors in successful implementation are summarized, and recommendations for future research and practice are made.
Article
The present study used meta-analytic methodology to synthesize research on the relationship between student ratings of instruction and student achievement. The data for the meta-analysis came from 41 independent validity studies reporting on 68 separate multisection courses relating student ratings to student achievement. The average correlation between an overall instructor rating and student achievement was .43; the average correlation between an overall course rating and student achievement was .47. While large effect sizes were also found for more specific rating dimensions such as Skill and Structure, other dimensions showed more modest relationships with student achievement. A hierarchical multiple regression analysis showed that rating/achievement correlations were larger for full-time faculty when students knew their final grades before rating instructors and when an external evaluator graded students’ achievement tests. The results of the meta-analysis provide strong support for the validity of student ratings as measures of teaching effectiveness.