ArticlePDF Available

Assessing Backstage--A Backchannel for Collaborative Learning in Large Classes

  • QualityMinds

Abstract and Figures

To reduce the passivity of students in large lectures, we propose to enhance the classroom with the digital backchannel Backstage, which aims at promoting awareness and participation. We tested Backstage in a preliminary user study at a relatively early stage of development. Besides gaining first impressions of usability, we were particularly interested in the kinds of questions raised on Backstage and how they correlate to the participants' attitudes towards questioning. The study confirms known issues regarding questioning that are not fully overcome with the tested prototype. Thus it is concluded that the concept of Backstage needs to be extended in order to embrace remedial assistance in how to ask adequate questions during lectures.
Content may be subject to copyright.
Assessing BackstageA Backchannel for
Collaborative Learning in Large Classes
A Formative Study on its Usability and Influence on Students‘ Questioning
Vera Gehlen-Baum
Educational Technology
Saarland University
Saarbrücken, Germany
Alexander Pohl, François Bry
Institute for Informatics
University of Munich
Munich, Germany
AbstractTo reduce the passivity of students in large lectures, we
propose to enhance the classroom with the digital backchannel
Backstage, which aims at promoting awareness and
participation. We tested Backstage in a preliminary user study at
a relatively early stage of development. Besides gaining first
impressions of usability, we were particularly interested in the
kinds of questions raised on Backstage and how they correlate to
the participants' attitudes towards questioning. The study
confirms known issues regarding questioning that are not fully
overcome with the tested prototype. Thus it is concluded that the
concept of Backstage needs to be extended in order to embrace
remedial assistance in how to ask adequate questions during
Collaborative Learning, Enhanced Classroom, Digital
Backchannel, Questioning
Maybe everyone in her academic life has attended some
instructor-centered lectures with large and anonymous
audiences and only few students actively participating in the
lecture. Although findings suggest that active participation
contributes to better learning (e.g. [1,2]), large class lectures do
not provide much space for an actively participating and
collaborating audience. Certainly, social mediation, i.e., the
feasibility of social interaction between a teacher and her
audience as well as among the students, is a limiting factor: the
larger the class, the fewer social interaction takes place. In
groups with more than 30 members, a manager is needed to
facilitate discourse. In many cases, the teacher plays this role
Discourse between persons often follows certain triadic
communication structures or patterns. For example, a discourse
pattern frequently observed in classrooms is the IRE (Initiation,
Response, and Evaluation) pattern: the teacher initiates
discourse by posing a question; a student responds to that
question that is finally evaluated by the teacher. Besides
evaluation, the discourse can also follow the IRF (Initiation,
Response, and Follow up) pattern: the teacher may follow up
on student‘s response with a further comment [3].
Yet, in such a discourse usually only few persons, i.e. the
teacher and a few students, are involved. The large part of the
audience basically remains passive. According to the
constructivist‘s perspective, however, learners construct their
own knowledge and understanding, which requires active
participation in the learning process [4]. Furthermore,
educational scientists have developed the theory of
communities of learners, according to which learning is not
understood as a transfer of knowledge between experts, nor an
acquisition of experts‘ knowledge by novices, but rather as a
transformation of the learner‘s participation in the learning
process [5].
One way to increase active participation of and social
interaction among students may be to employ textual
computer-mediated communication in the classroom, since
―Text-only CMD [Computer Mediated Discourse] is a
surprisingly effective way to ‗dointeractional work […]‖ [6]
(p. 11). Besides, CMD is effective in encouraging students to
participate in the discourse. For example, in the field of second
language learning, it has been observed how CMD is adopted
by students for their learning and how it affects their
participation [7]. It is reported on the observation that every
student participated in CMD, including those students who are
shy and more self-conscious about the mistakes they make than
other students. Furthermore, the balanced nature of CMD has
been highlighted: students that dominate classroom discourse,
e.g. by answering every question posed by the teacher, do not
dominate CMD. Every student gets a chance to speak without
being interrupted.
We propose to enhance the classroom with what are called
digital backchannels, i.e., computer-mediated synchronous
(textual) communication tools, making non-disruptive
information exchange among the audience during the lecture
possible. In particular, we hypothesize that providing a social
environment in which questions can be raised during the
lecture is a major benefit of using backchannels in large class
lectures. However, we contend that current digital
backchannels do not sufficiently meet the conditions and
requirements imposed by large class lectures.
A. Using Backstage in Large Classes
In a joint effort of computer and educational scientists, we
developed the digital backchannel Backstage, which provides a
rich set of interactions specifically tailored for the use in large
lecture classes. Its functionalities comprise fast information
exchange, various kinds of feedback dedicated to both the
student and the lecturer, and incentives to partake. In the
remainder of this section, we provide a short overview of
Backstage. For a more detailed description the interested reader
is referred to [8].
By means of a microblog, students may exchange ideas and
comments through short messages, comprising only a few
words. The most prominent example of a microblog is
, in which messages comprise a maximum of 140
characters. Microblogs allow messages to be written and read
quickly, and hence are well suited for the use as digital
backchannels in large class lectures. Backstage provides
different modes of communication: messages can be visible
either to the public or to certain users only (private
communication). Besides, the author‘s username may be
shown along with each of her message or be hidden
(anonymous communication). Indeed, one can argue for or
against the usefulness of all these modes of inter-personal
communication. For example, anonymous communication
certainly lowers the barrier to partake, especially during the
first turns at the beginning of a semester, but may likewise
simplify the dissemination of offending messages. We think
that whether certain modes of communication should be
available largely depends on the compilation of the audience
and the topic of the lecture. Thus, Backstage is highly
customizable by the lecturer, meaning that the lecturer, even
during lecturing, can easily change the availability of the
different modes of communication provided by Backstage.
Students may assign messages to certain predefined
categories, e.g. ―Question‖ or ―Lecturing Pace‖. According to
these categories, messages can be filtered out, aggregated and
be displayed at the lecturer‘s dashboard. This provides her with
a concise and simple topic-related overview of the backchannel
discourse. Thus, categories can be used by students to give
immediate feedback to the lecturer related to contents (e.g.
―Question‖, ―Remark‖) or to lecturing style (e.g. ―Lecturing
Messages can furthermore be rated by students to assess the
relevance of a message for the lecture. This is provided on
Backstage by means of a simple rating scheme allowing
students to quickly express approval or rejection of a message.
Rating in the context of Backstage serves two important
purposes. First, it makes possible to determine a ranking of the
top k messages that can be displayed at the lecturer‘s
dashboard. Combined with the topic-related aggregation of
messages described in the previous paragraph the lecturer
obtains a very concise, yet meaningful overview of the
backchannel discourse. Second, rating provides feedback to
students given by their peers. This feedback can furthermore be
accumulated to form a kind of status for a student. The better
her messages are rated the higher the status she gains.
To promote active participation the lecturer can conduct
short quizzes, reminiscent to Audience Response Systems
(ARS; e.g. [9-11]) via Backstage. A students concentration
drastically declines after 20 minutes of continuous lecture [2].
Quizzes allow the instructor to break up the lecture into smaller
Audience Response Systems are also referred to as Student Response
Systems, Personal Response Systems, and Clicker Systems, among others.
sections and to assess factual knowledge and students‘
retention. In addition to fostering active participation, quizzes
provide feedback about the student‘s performance relative to
their peers. When the lecturer starts a quiz, the students
dashboards display a multiple-choice item. The students‘
responses are displayed and updated in real-time at the
lecturer‘s dashboard. As soon as the lecturer closes the voting
phase, the responses are aggregated and displayed at both the
lecturer‘s and the students‘ dashboards for further review.
B. UsabilityA Tradeoff between Economy of Effort and
In [12], usability is defined as a quality attribute that
assesses how easy user interfaces are to use―, and is
operationalized by five variables. While some of these, e.g. the
number and severity of errors the user makes (errors) or the
time necessary to rebuild the knowledge acquired after some
time of not using the system (memorability), are important to
consider for all kind of software, the remaining three
variableslearnability, efficiency, and satisfactionare of
particular importance for computer-based learning
In most cases it is rational to simplify user interactions by
means of a clear and concise user interface (learnability) that
enables users to quickly accomplish their tasks (efficiency).
However, the resulting economy of effort, especially when
considering user interfaces of applications for learning, might
not always be the best way to improve learning processes and
results. Studies on expertise have shown that many activities
are quickly learned to an extent sufficient for daily life. In
order to become an expert, however, it is necessary to
perpetually reflect, and possibly deviate from, the activities as
being taught [13]. Therefore it could be fruitful to provide user
interactions that initiate some cognitive activity in the learning
Learning environments often adopt user interactions from
well-known applications such as Facebook
or Twitter which
are familiar to most learners. This allows users to quickly grasp
the way to interact with the application (learnability) and to
quickly accomplish the tasks without much need for settling in.
However, besides mere usability, furthereducationaldesign
parameters need to be taken into account in order to ease
interactions as well as provide for occasions in which the user
can reflect on her interactions. Similar to Piaget‘s Conceptual
Change [14] that can be observed in children being stretched
to their limits due to insufficient mental models of the world,
adult learners should be stretched to their limits when carrying
out learning tasks in a sub-optimal manner. Likewise, they
should be provided with instructions supporting them in
optimizing their learning. In Computer Supported
Collaborative Learning (CSCL) this is frequently achieved by
the provision of collaboration scripts: learners obtain roles
according to which the tasks and the orders in which to carry
out these tasks are specified. Several studies could substantiate
the superiority of this approach compared to respective controls
(e.g. [15-17]).
Undoubtedly, usability is an essential aspect of the design
of learning platforms. However, it might prove reasonable
with sound judgementto attach more importance on
educational claims (e.g. the conveyance of competencies in
questioning) rather than on mere usability principles.
C. Asking the Right Kinds of Questions to Improve
In several articles (e.g. [18,19]) it has been argued that
questioning is an important strategy to elaborate new
knowledge. Unfortunately, students often find it difficult to ask
for help and further explanation [20]. This is not surprising,
considering that formulating questions already requires some
understanding of, and former knowledge about, the topic (e.g.
[18]). Questioning is thus subject of various fields of research.
For example, reciprocal teaching [21] and script research (e.g.
[22,23]) deal with questioning or at least highlights its
importance [24].
Not only the quantity, but also the quality, of questions is
important for learning [23]. In ibid. a distinction is made
between three kinds of questions: factual, comprehension, and
integration questions. The Table I summarizes the descriptions
of questions and their distinctions regarding knowledge.
Tab. I Kinds of Questions (from [23], p. 350, with minor adaptations).
Asks for recall of facts or other information
explicitly covered in the lesson.
Encompasses knowledge restating: Simple
statements of fact or information gleaned directly
from the lesson or prior knowledge.
Asks for a process or term to be described or
Encompasses knowledge assimilation:
Definitions, descriptions, and other material
paraphrased in student‘s own words.
Goes beyond what was explicitly stated in the
lesson, connects two ideas together, or asks for an
explanation, inference, justification, etc.
Encompasses knowledge integration: Makes new
connections or goes beyond what was provided in
the lesson explanations, inferences,
relationships between ideas, justifications,
statements, linking session content to material
from outside the lesson (prior knowledge and
personal experience).
In order to achieve better learning results, it is especially
important to formulate integration questions as these are
connected to a deeper elaboration, though it is quite uncommon
that integration questions are raised without being fostered by
appropriate guidance. These were the findings of the study
presented in [23], were it has been shown that integration
questions could be trained and lead to a ―more complex
knowledge construction‖ (p. 338).
D. Research Question
We tested the Backstage prototype in a preliminary user
study at an early stage of development. We focused on the
following questions. (1) What are the participants‘ impressions
of the usability of Backstage? (2) What kinds of questions were
asked on Backstage in comparison to traditionally held
presentations without any backchannel support? (3) How do
these questions correlate with the participants attitudes
towards asking questions on Backstage?
A. Participants and Design
Nineteen participants from different subjects (mainly
Educational Science and Informatics but also Psychology and
German as foreign language) participated in this study. The
mean age of the participants was M = 26.32 (SD = 3.33). Five
(26.32 %) participants were female and fourteen (73.68 %)
were male. Since few can be said about students‘ prior
knowledge in large lectures, we decided to omit a pretest for
this preliminary study and thus chose an experimental design
with one control group. The participants were picked randomly
from the different fields of study. Every participant in the
experimental group was assigned a working place equipped
with a PC, resembling traditional computer-based learning
environments. Although unnecessary, each participant of the
control group was also assigned a PC working place.
To assure that the prototypical implementation of
Backstage was able to cope with the amount of data that was to
be processed we subdivided the experimental group (fourteen
participants) into three subgroups with five and four
participants, respectively. For the control group, we used the
equal group size of five participants in order to ensure the
participants weren‘t inhibited by a larger group size.
B. The Learning Environment
A 30-minutes presentation was held in front of each group
in a room with PC workplaces. The presentation was about two
educational subjects which were presented for ten to fifteen
minutes each. The first talk was about the notion of transfer
the definition, the reasons why it occasionally fails to take
place and the ways to foster it. The second part was about the
research on expertisethe definition and a short overview of
this field. We tried to eliminate additional talk in the
classroom. The participants of the control group listened to the
presentation in a traditional way and thus were invited to raise
Prior to each trial, every participant in the experimental
group was already logged into Backstage using pseudonyms.
The experimental group could only use certain features of
Backstage. As the study was conducted at an early stage of
development, several features were not completely
implemented or unstable. The participants were informed about
the state of implementation and the features that could be used.
The Figure 1 shows the layout of the students‘ dashboards.
The microblog communication capabilities as described in
Section I.A were completely functioning and available. The
participants could also rate messages, although the rating
scores of the messages were not aggregated and rendered along
with each message. The participants could also assign
messages to categories that were already aggregated properly
and were rudimentarily displayed at the presenter‘s dashboard.
The top-k ranking of messages was also not available but the
presenter strived to respond to the questions that were raised on
the backchannel. The quiz module was basically available,
although the rendering of quiz results was still in need for
C. Instruments and Dependent Variables
The dependent variables were (1) usability, (2) the
participants‘ attitudes towards asking questions, and (3) the
number and kinds of questions asked during the presentation.
The first two variables were mainly measured by questionnaire
items and by analysis of the backchannel communication.
We tested the usability with fourteen multiple choice
questions (Likert scale
from 1 to 6) and three open items.
Since the participants were using Backstage for the first time,
we considered more important to measure the time needed to
get acquainted with the application, than other usability
aspects, e.g. the number of errors the participants made. Thus,
the learnability was measured by the time needed to get
acquainted with Backstage and usability was measured by
multiple-choice questions like ―The application‘s user interface
is clear and concise‖. The reliability of the usability questions
were measured using Cronbachs α [25] that yields α = 0.91. In
The linguistic grades corresponding to the numeric grades 1 to 6 are:
―definitively disagree (1), disagree, tend to disagree, tend to agree,
agree, and ―definitively agree (6).
addition, the three open items were meant to give a deeper
insight in what the participants liked or missed. The answers
were sighted by two coders and certain categories like
―questions to the presenter‖ were counted. Cohen‘s Kappa, by
which the consensus between the two coders is measured [26],
yields κ = 0.92.
The attitude towards questions was measured with seven
multiple choice questions like I felt unsure when raising a
question‖. The reliability yields α = 0.77.
The backchannel communication was exported to log files
and the statements of the control group were transcribed
accordingly. The statements were classified twice by distinct
coders into question, response to question, and ―feedback
to the presenter‖. Since no other kinds of statements ended
with questions marks, questions could be easily identified. A
response to a question could be determined by its adjacency to
the respective question or by context. Feedback to the presenter
could be extracted in a similar way. In a second step,
statements belonging to ―question‖ were further subdivided
according to their content into the classes given in the Table I
(κ = 0.74).
Since the control group was not to answer questions
regarding the usability of the application, we provided a similar
questionnaire that focused on questioning. The outcomes of
this questionnaire, however, are not presented in this article
which focuses on the analysis of the process data.
D. Procedure
We conducted each trial during a 30-minutes presentation.
The participants were assigned to the prepared places. For the
control group, the places were additionally equipped with pen
Figure 1 A participant's dashboard on Backstage. The picture shows the microblog editor and timeline (left) and the results of a quiz that has just been
conducted (right). The dashboard shown is in German, since the user study was conducted in German.
and paper and for the experimental group the login to
Backstage had already been accomplished. The experimental
group was provided with a brief introduction to Backstage and
how to use the different functions. Furthermore, both
experimental and control group were provided with a short
summary about the outline of the study. The control group was
particularly instructed to raise hands in order to pose a question
during the presentation. The presentation was subdivided into
two partsthe first part was about transfer, the second about
expertise. After each section both groups were invited to raise
questions. Immediately after the presentations, which were
held comparably in contents and style, the participants were
asked to fill out a questionnaire.
E. Statistical Analysis
Due to the small group sizes we decided to forgo inductive
statistical analysis and to restrict ourselves to descriptive
analysis. However, in most cases the heterogeneous t-tests (not
reported on in this paper) reveal significant differences
between traditional presentations and presentations with
Backstage running.
A. Usability
The mean M = 4.42 (SD = 0.78) on usability we measured
is between tend to agreeand agree‖, thereby indicating that
the participants found it easy and intuitive to work with
Backstage: every participant posted at least one message on the
backchannel during the presentation. Also, the participants‘
comments point in this direction. Notably, the time the
participants stated to be necessary in order to get acquainted
with Backstage highly varied. While the average was M = 8.79
(SD = 8.34) minutes, the minimum time stated was one minute
and the maximum was 30 minutes.
By means of the open items we could figure out that the
participants appreciated the possibility to directly send
feedback to the presenter (mentioned eight times). The process
data showed that the experimental groups gave feedback to the
presenter more often than the control group (M = 5.43) which
gave only short feedback, e.g. ―ok‖ for acknowledging
(M = 0.40). Also, Backstage‘s quiz functionality to assess
current knowledge was appreciated by the participants of the
experimental groups (mentioned five times).
However, they missed the easy-to-use communication
known from other communication tools. Similar statements
were made regarding the lack of presentations slides integrated
in Backstage (mentioned twice). Three participants also noted
the risk of getting distracted by the use of Backstage.
B. Questions
With Backstage the participants gave approximately nine
times more remarks during the presentation than without the
backchannel (Figure 2A).
Among the messages on the backchannel, only a small
fraction was off-topic (M = 2.5, SD = 2.82). A considerable
amount of the messages were feedback (M = 5.42, SD = 5.57).
The largest part was questions (Figure 2A).
Participants of the experimental groups raised questions
more frequently than of the control group. This is also reflected
by the positive attitude towards asking questions on Backstage
(M = 4.56; SD = .87). While few questions are about using
Backstage, the large part of questions refer to the presentation
contents. The questions of participants of the control group did
not refer to the use of Backstage, but to the presentation‘s
content only. The distribution of questions regarding the
classification of questions as given in the Table I is illustrated
in the Figure 2B.
Figure 2 A) A participant‘s average number of contributions and questions, in
particular. The numbers in brackets indicate the respective standard variances.
B) A participant‘s average number of questions split in the kinds described in
the Table I. The numbers in brackets indicate the respective standard
Since this article reports on a preliminary study mainly
focusing on first impressions of the usability of Backstage,
only few statements can be made regarding the usefulness of
Backstage from an educational point of view. However, further
studies to investigate the educational usefulness of Backstage
are currently in the planning process.
A. Usability
The average time of nine minutes needed to get acquainted
with Backstage is rather short, which can be attributed to the
similarity to well-known platforms such as Twitter. However,
the variability around the average could be a ramification of the
participants‘ different notions of ―becoming acquainted with
Backstage‖, their backgrounds, or their levels of experience
with new media. For clarification, however, further
investigation is necessary.
Although the analysis of the questionnaires suggests that
the participants liked working with Backstage, in the open
items, the usability has been remarked to be in need of
improvements. Keeping in mind that on Backstage the
interaction design aims at triggering reflective thinking of
users, this remark did not come up unexpectedly. Furthermore,
the responds to the multiple choice questions suggest that most
participants did not find the deficiencies too disturbing. The
observation that every participant posted at least one message
during the presentation indicates that Backstage is convenient
to use and positively influences participation.
Whether learners are content with such a ―rectified‖ user
interaction design (satisfaction) or reject them, but at the same
time achieve better learning outcomes, is an essential aspect
that needs to be carefully considered. The learner may,
similarly to motivation [27], develop satisfaction due to
improved learning outcomes and positive feedback, despite
initial rejection stemming from potential cuts in usability.
However, one needs to clarify whether satisfaction and
impressions of usability may change in the long run.
The remarks regarding the quiz and the feedback
functionalities of Backstage suggest that the effects of ARSs on
active participation also apply to Backstage. To sum up, we
may conclude that the users get along well with Backstage,
even with the first (yet unfinished) prototypical
implementation. No fundamental criticism was given, but
further functionalities encouraged, e.g. the integration of
presentation slides in Backstage. For the next stage of
development we seized on this suggestion. The integration of
presentation slides, however, needed careful consideration to fit
the design rationale behind Backstage. This will be reported on
in a companion paper in the near future.
B. Questions
Comparing the experimental to the control group, we could
find indications for positive effects on the activity of the
participants when using Backstage. Thus, more questions were
raised on Backstage than being asked by the control group. The
same applies to feedback addressed to the presenter. It should
be noted that the participants of the control group only asked
questions after invitation, maybe to avoid interrupting the
presenter. In how far the increase of questions on Backstage
has impact on the learning results needs to be investigated in a
further study. However, we may assume that an increase of
activity positively influences learning success and, therefore,
motivation (e.g. [28]). However, as being stated by the
participants, the raising of more questions may also contribute
to distraction. Further investigation of this issue is required.
Regarding questioning, the findings of [23] could be
confirmed. In the experimental group the majority of questions
were factual questions, lesser were comprehension questions,
and the least integration questions. We consider this outcome
as an indication that questioning on Backstage needs guidance.
In a further step in Backstage‘s development we thus
reconsidered the interactions and the way they need to be made
feasible in order to account for the need of guidance. This issue
will also be presented in a forthcoming companion paper.
The high number of integration questions of the control
group neither confirms the findings of [23], nor does it meet
our expectations. However, this can be explained by the
following two circumstances: (1) each remark in the smaller
control group (compared to the experimental group) is in
average of a greater influence (compared to the experimental
group). (2) In the control group, most participants, i.e. three out
of five, were students of Psychology and Educational Sciences
and about to finish their studies. Since integration questions
highly depend on prior knowledge, the described constellation
of the group could explain the unexpected outcome. We cannot
rule out that the topics chosen for the presentation have
impinged on the participantsquality of questioning.
In summary, the findings of this preliminary study suggest
that Backstage has positive influence on the activation of
learners during presentations. Another aspect of active
participation that deserves investigation, besides the support in
asking questions, is the use of Backstage in large lecture
classes. Participants noted the risk of distraction when using
Backstage in a real lecture setting. Findings on the basis of the
Cognitive Load Theory support the participants‘ appraisal [29].
If predominant distraction can be verified in forthcoming
studies, then it is necessary to adapt lectures appropriately in
order to leverage Backstage. For example, this can be achieved
by conceding breaks for questioning on the backchannel. A
more thorough structuring of the lecture by means of macro-
scripts [30] may additionally support the versatile use of
Backstage. In using these macro-scripts we can also include
additional activities like collaboration in the lecture and
support them in various ways.
The authors would like to thank Jochen Rick at Educational
Technology, Saarland University, for the fruitful discussions.
[1] A.W. Chickering and Z.F. Gamson, ―Seven Principles for Good
Practice in Undergraduate Education,‖ AAHE Bulletin, 1987, pp. 3-7.
[2] J. Cuseo, ―The Empirical Case against Large Class Size: Adverse
Effects on the Teaching, Learning, and Retention of First-Year
Students,Journal of Faculty Development, vol. 21, 2007, pp. 5-21.
[3] H. Nassaji and G. Wells, ―What‘s the Use of ‗Triadic Dialogue‘?: An
Investigation of Teacher-Student Interaction,‖ Applied Linguistics, vol.
21, 2000, pp. 376-406.
[4] G. Mietzel, Pädagogische Psychologie des Lehrens and Lernens
[Educational Psychology of Teaching and Learning], Göttingen:
Hogrefe, 2003.
[5] B. Rogoff, ―Developing understanding of the idea of communities of
learners,Mind, Culture, and Activity, vol. 1, 1994, pp. 209-229.
[6] S.C. Herring, ―Computer-Mediated Discourse,‖ Handbook of
Discourse Analysis, D. Tannen, D. Schiffrin, and H. Hamilton, eds.,
Oxford: Blackwell, in press.
[7] O.R. Kelm, ―The Use of Synchronous Computer Networks in Second
Language Instruction: A Preliminary Report, Foreign Language
Annals, vol. 25, 1992, pp. 441-454.
[8] F. Bry, V. Gehlen-Baum, and A. Pohl, ―Promoting Awareness and
Participation in Large Class Lectures: The Digital Backchannel
Backstage,‖ Proceedings of the IADIS International Conference e-
society, Spain, Avila: 2011, pp. 2734.
[9] R.H. Kay and A. LeSage, ―Examining the Benefits and Challenges of
using Audience Response Systems: A Review of the Literature,‖
Computers & Education, vol. 53, 2009, pp. 819 - 827.
[10] C. Fies and J. Marshall, ―Classroom Response Systems: A Review of
the Literature,Journal of Science Education and Technology, vol. 15,
2006, pp. 101-109.
[11] R.W. Preszler, A. Dawe, C.B. Shuster, and M. Shuster, Assessment
of the Effects of Student Response Systems on Student Learning and
Attitudes over a Broad Range of Biology Courses,‖ CBE Life Science
Education, vol. 6, 2007, pp. 29-41.
[12] J. Nielsen, ―Usability 101: Introduction to Usability,‖ 2003.
[13] K.A. Ericsson, ―The Influence of Experience and Deliberate Practice
on the Development of Superior Expert Performance,The Cambridge
Handbook of Expertise and Expert Performance, K.A. Ericsson, N.
Charness, P.J. Feltovich, and R.R. Hoffman, eds., Cambridge:
Cambridge University Press, 2006, pp. 683-703.
[14] J. Piaget and R. Fatke, Meine Theorie der geistigen Entwicklung [My
Theory about Intellectual Development], Beltz, 2003.
[15] L. Kobbe, A. Weinberger, P. Dillenbourg, A. Harrer, R. Hämäläinen,
P. Häkkinen, and F. Fischer, ―Specifying Computer-Supported
Collaboration Scripts,‖ International Journal of Computer-Supported
Collaborative Learning, vol. 2, 2007, pp. 211-224.
[16] I. Kollar, F. Fischer, and J.D. Slotta, ―Internal and External
Collaboration Scripts in Web-based Science Learning at Schools,‖
Proceedings of the Conference on Computer Support for Collaborative
Learning: Learning 2005: The Next 10 Years!, International Society of
the Learning Sciences, 2005, pp. 331340.
[17] A. Weinberger, F. Fischer, and K. Stegmann, ―Computer-Supported
Collaborative Learning in Higher Education: Scripts for Argumentative
Knowledge Construction in Distributed Groups,‖ Proceedings of the
Conference on Computer Support for Collaborative Learning:
Learning 2005: The Next 10 Years!, Taipei, Taiwan: International
Society of the Learning Sciences, 2005, pp. 717726.
[18] B. Ertl and H. Mandl, Kooperationsscripts als Lernstrategie
[Cooperation Scripts as a Learning Strategy], München: 2004.
[19] B. Rosenshine and C. Meister, ―Reciprocal Teaching: A Review of the
Research.,‖ Review of Educational Research, vol. 64, 1994, pp. 479-
[20] V. Aleven, E. Stahl, S. Schworm, F. Fischer, and R. Wallace, ―Help
Seeking and Help Design in Interactive Learning Environments,‖
Review of Educational Research, vol. 73, 2003, pp. 277 -320.
[21] A.S. Palinscar and A. Brown, ―Reciprocal Teaching of
Comprehension-Fostering and Comprehension-Monitoring Activities,‖
Cognition and Instruction, vol. 1, 1984, pp. 117-175.
[22] A. King, ―Scripting Collaborative Learning Processes: A Cognitive
Perspective, Scripting Computer-Supported Collaborative Learning,
F. Fischer, I. Kollar, H. Mandl, and J.M. Haake, eds., Boston, MA:
Springer US, 2007, pp. 13-37.
[23] A. King, ―Guiding Knowledge Construction in the Classroom: Effects
of Teaching Children How To Question and How To Explain,‖
American Educational Research Journal, vol. 31, 1994, pp. 358-68.
[24] D. Dansereau and A. O‘Donnell, ―Scripted Cooperation in Student
Dyads: A Method for Analyzing and Enhancing Academic Learning
and Performance., Interactions in Cooperative Groups. The
Theoretical Anatomy of Group Learning, New York: Cambridge
University Press, 1992, pp. 120-141.
[25] L.J. Cronbach, ―Coefficient Alpha and the Internal Structure of Tests,‖
Psychometrika, vol. 16, 1951, pp. 297-334.
[26] J. Cohen, Statistical Power Analysis for the Behavioral Sciences,
Routledge, 1988.
[27] E.L. Deci and R.M. Ryan, ―The Support of Autonomy and the Control
of Behavior,‖ Journal of Personality and Social Psychology, vol. 53,
1987, pp. 1024-37.
[28] E. Deci and R. Ryan, Die Selbstbestimmungstheorie der Motivation
und ihre Bedeutung für die Pädagogik [Self Theories on Motivation
and their Implications for Pedagogy],‖ Zeitschrift fürdagogik, vol. 2,
[29] J. Sweller, ―Cognitive Load Theory, Learning Difficulty, and
Instructional Design,‖ Learning and Instruction, vol. 4, 1994, pp. 295-
[30] P. Dillenbourg and P. Jermann, ―Designing Integrative Scripts,‖
Scripting Computer-Supported Collaborative Learning, F. Fischer, I.
Kollar, H. Mandl, and J.M. Haake, eds., Boston, MA: Springer US,
2007, pp. 275-301.
... First, Edmodo allows conversations to be threaded, which allows greater discussion about a specific subject within the backchannel (Cerratto, 2001). Secondly, Edmodo demonstrates learnability, since it closely resembles commonly used social media (i.e., FaceBook), and efficiency, since comments are posted quickly and easily (Gehlen-Baum, Pohl, & Bry, 2011). Thirdly, Edmodo allows the audience to be 'wired' to each other continuously throughout the professional development. ...
... A recognized problem with traditional teaching and professional developments is the large amount of passivity by the learners (Ebner, 2009;Du, Rosson, & Carroll, 2010;Gehlen-Baum, Pohl, Weinberger, & Bry, 2012;Pohl, Gehlen-Baum, & Bry, 2011). Additionally, learners may feel uncomfortable asking questions because they do not want to interrupt the lecturer, they do not want to be perceived as incompetent by their peers, they may have a shy nature, or they may not know enough about the subject to construct meaningful questions (Bry, Gehlen-Baum, & Pohl, 2011;Du et al., 2012;Ebner, 2009;Fredrick, 2013;Gehlen-Baum et al., 2012;Nobarany & Haraty, 2009;Pohl et al., 2011;Yardi, 2006). ...
... A recognized problem with traditional teaching and professional developments is the large amount of passivity by the learners (Ebner, 2009;Du, Rosson, & Carroll, 2010;Gehlen-Baum, Pohl, Weinberger, & Bry, 2012;Pohl, Gehlen-Baum, & Bry, 2011). Additionally, learners may feel uncomfortable asking questions because they do not want to interrupt the lecturer, they do not want to be perceived as incompetent by their peers, they may have a shy nature, or they may not know enough about the subject to construct meaningful questions (Bry, Gehlen-Baum, & Pohl, 2011;Du et al., 2012;Ebner, 2009;Fredrick, 2013;Gehlen-Baum et al., 2012;Nobarany & Haraty, 2009;Pohl et al., 2011;Yardi, 2006). By offering communication through a backchannel, there is an opportunity to participate in the learning process through interacting with peers without the disruption of the presentation, as well as reducing apprehension about asking questions and giving feedback (Boyle & Nicol, 2003;Cogdill et al., 2011;Gehlen-Baum et al., 2011;Pohl et al., 2011;Yardi, 2006). ...
... It was not clear how this phase is separated from alternatives discussion in most studies, especially in the studies that used discussion forums and chatting as means for interactions. Among the few studies that clearly expressed the output of this stage, the work presented in [20][21][22][23], and [47]. In [20], a concept map was created to express the shared understanding the group has reached. ...
... Also in [21], a database of all herbs with illustrations that help students clarify meanings about herbal uses and names was created. Moreover, in [47], students were able to assign messages to categories that were aggregated properly and displayed at the presenter's dashboard. In some studies, the tools used can support this phase, although not clearly mentioned. ...
... The formalization phase was intentionally covered in some studies in the form of written artifacts [5], [18][19][20], [22], [46], [32][33], or a generated model or solved puzzle [43] and [39]. However, in most studies, no clear artifacts were generated, e.g., [3], [13], [23][24], [28][29], [30][31], [34], [37][38][39], [40][41][42][43][44][45], [47][48][49][50][51][52]. This was true, especially for the tools that use textbased articulation platforms. ...
... These are the most frequently mentioned factors in the used studies. As shown in the table, the student factors were usually missing (except for few studies which included the gender e.g., [19], [20]). Also, it should be noted that the most of the studies show positive effects on learning by collaborative knowledge transfer. ...
... 3. Multi-touch Tabletops ( [29], [22], [30], and [31]). 4. Tangible interfaces ( [22] , and [24]). 5. LMS that combines chat, discussion, video conferencing, etc.( [19], [20], [32], and [33] ) 6. Gamification platforms ( [34], and [35]). 7. Other semantic web tools (discussion boards; social media: Facebook, Google+; wikis,..) ( [36], [37], [38], [39], [40], [41], [42], and [43] ...
... How can teachers make effective use of the EQF for designing, monitoring, and moderating their lessons? [19] 19 9 up to 10 graduates The digital backchannel Backstage. Students may assign messages to certain predefined categories, e.g. ...
Conference Paper
Full-text available
The integration of technology in education is of concern to both researchers and to practitioners especially computer-supported collaborative learning (CSCL) which has gained a lot of interest lately. This field of research focuses on how computer-based technology can support and promote collaborative learning. Thus, research on technology and its potential use in collaborative learning play a key role in understanding and addressing issues regarding technology-blended learning. In this paper, we present a systematic review of the literature on different technology-driven platforms that support either the design, monitoring, or even motivates learners to embrace collaborative learning activities. In particular, we present a detailed analysis of 27 relevant papers that we identified in accordance with our review protocol. The review results are organized to address two research questions. Through our answers to these questions, we present the platforms or systems that have been used in collaborative learning and the affordances provided by each platform in accordance to social collaborative learning. Overall, review results indicate that while researchers have conducted several studies on this topic, there are still research gaps that need to be addressed. In addition, it was noted that although most of the used platforms or systems do not explicitly enforce or support some of the key theoretical elements of collaborative learning, analysis of the used studies reveals that collaborators are motivated by the used tools to be compliant with some of these elements.
... Dazu zählen beispielsweise die Applikationen LectureTools 1 , myTU 2 und UnderstoodIt 3 . Die Literatur weist hier bereits einiges an Erfahrungen mit unterschiedlichen Systemen auf, beispielsweise mit den Entwicklungen ClassTalk (Anderson et al., 2003), ActiveClass (Ratto et al., 2003), AuthoringOnTheFly (Datta & Ottmann, 2001), Realfeedback (Haintz et al., 2014) oder weiteren speziell entwickelten Systemen (Purgarthofer & Reinhaler, 2008;Ebner, 2010;Gehlen-Baum et al., 2011;Attkinson, 2009;Yardi, 2006). ...
Full-text available
Interaktion zwischen Lehrenden und Studierenden in gefüllten, großen Hörsälen ist eine große Herausforderung. Audience-Response-Systeme werden als eine Möglichkeit betrachtet, schnelle Rückmeldungen von den Studierenden zu erhalten und so deren Einbindung und die Interaktion im Hörsaal zu erhö- hen. Im vorliegenden Beitrag wird die Entwicklung, das Konzept sowie das Design eines Prototypen einer Anwendung vorgestellt („Backchannel“), die es den Studierenden ermöglicht, den Vortragenden kontinuierlich Rückmeldung zum Vortrag zu geben. Die Lehrenden erhalten ein aggregiertes visuelles Feedback in Echtzeit und können darauf reagieren. Ausführlich werden dabei die Ergebnisse des Testeinsatzes dargestellt sowie zukünftige Adaptionen und Entwicklungsmöglichkeiten diskutiert.
... First, the lecturer asks a question, typically supported by a slide showing the question and the different answers in a written form. Second, all students can answer the question by pressing a hard-or software button, i.e. on a so called "Clicker" device or respectively, running a specific software on their mobile device [35], [36]. The results of multiple-choice questions are often visualized as a bar-chart. ...
Conference Paper
Mobile devices, such as laptops, smartphones and tablets, are ubiquitous in lectures. Students report to use their mobile devices for lecture-related activities (e.g. taking notes). Observational data shows, that students use mobile device mainly for lecture-unrelated activities, like Facebook or playing games. So currently, mobile devices seem to distract learners from the lecture and ultimately hinder student-teacher interaction. In this study, we investigated how students (n = 75) use their mobile devices (N = 80) in a traditional lecture setting when supported with the technological support system “Backstage” or not. Backstage entails functions for quizzing students (Audience-Response-System) and a backchannel allowing students to interact with each other, commenting on slides, asking questions, and providing feedback to lecturers. The results show that this technology increases students’ focus on lecture-related activities.
... One way to do so is to offer students slides with which they can work with, like taking notes on them (Hartley, 2006). Another way to focus more on slides is by including them into a so called backchannel and help students communicate with the lecturer or each other by asking written questions via slides Gehlen-Baum et al., 2011). Further research regarding educational technology can help to understand which technological support involves students in lectures. ...
Lectures are designed to deliver new information to a large group of students. Apart from actual lecturing, lecturers may also encourage elaboration of learning material with advanced instructions, like advance organizer, summaries and repetitions as well as questions. Prerequisite to learning from lectures is that students focus on the lecture and cognitively process what is being presented. In today’s lectures, mobile devices, (i.e. laptops and smartphones), may aid students to research additional information online or to take notes, but may also distract students. In this descriptive study, 86 students with 91 mobile devices out of five lectures were observed with respect to how they utilized mobile devices for lecture-related and -unrelated activities. Additionally, we observed 21 lecturers and coded their behavior. The results indicate that lecturing correlates only slightly with student activities. Students use media mostly in a lecture-unrelated way. Giving negative feedback seems to foster students’ lecture-unrelated media use.
20th century iconic examples of human collaboration are the assembly line, centralised planning, bureaucracies, vote‐based decision making, and school education. These examples, and more generally all forms of human collaboration of the 20th century, are characterized by predefined human roles and little adaptable processes, that is, 20th century collaboration comes at the price of a restricted individual freedom. With the turn of the century, new forms of human collaboration have become widespread that exploit information and communication technologies, data generated by humans, Data Science in general and Machine Learning in particular, and let humans contribute as they like, when they like, and as much as they can, the lack of predefined roles and processes being accounted for by software. The phrase “Human Computation” coined for denoting the new forms of human collaboration stresses a core aspect of the paradigm which can be a downside: With Human Computation, humans become contributors to collaboration‐enabling algorithms that can also control and restrict how collaboration takes place. This article introduces to Human Computation and to its role in applications of Machine Learning, presents Human Computation prototype systems developed during the last decade at Ludwig‐Maximilian University of Munich, discusses ethical issues of Human Computation and Machine Learning, points to on‐going research in the field at Ludwig‐Maximilian University of Munich, and concludes with a reflection on the future of Human Computation. The original contribution of this article is a comprehensive presentation of recent research the main part of which has already been published in more detail elsewhere.
Purpose Several challenges of today’s higher education were motivations to reconsider the contents and formats of lectures and tutorials and to conceive the classroom communication system Backstage, a social media platform supporting a novel form of large-class teaching. The purpose of this paper is to report on the challenges met, on the novel teaching form and on an evaluation of this teaching form. Design/methodology/approach The use of Backstage in two courses is evaluated. One of the courses has been specially adapted to promote student participation, the other course has been held in a traditional way. To investigate the usefulness and acceptance of Backstage in the given settings the data collected on Backstage and student responses in surveys are analyzed. Findings The results indicate that Backstage can foster interactivity and awareness in large-class lectures when used in combination with a teaching format that provides opportunities for and encourges lecture-relevant communication. Furthermore, students appreciated the use of Backstage. Research limitations/implications This paper reports on a case study which lacks generalizability. Further studies under controlled conditions and of the learning effectiveness of the approach are still outstanding. Practical implications This paper describes an approach fostering a form of Active Learning in large classes. Since large classes are widespread in higher education, the approach has a considerable practical potential. Social implications The paper describes an approach to large class higher education teaching in using social media. Originality/value Similar results have not been published so far.
Exploratory studies have started to demonstrate the potential value of digital backchannels for enhancing interaction in university lectures. The present study was conducted in a third year engineering course and involved the use of specialised backchannel software with the following features: students could anonymously post questions, vote on questions, give the lecturer feedback regarding the pace of the lecture or simply alert the lecturer that they were 'lost'. The study used a mixed-method data analysis design that, in addition to data automatically logged through the backchannel, included the use of observations, surveys, lecturer interviews and student focus groups. It was found that students used all features of the backchannel, with especial use of the 'like' feature, not currently available in most commercial backchannel packages. The backchannel increased the number of questions asked in class, and also resulted in a broader range of students participating in such interactions. There was limited evidence of the backchannel proving a distraction; on the contrary some students said that it helped them to focus more in class. From the lecturers' perspective the backchannel did require some modification of their lecturing style but they felt that the additional feedback that they achieved was valuable.
communication;interdisciplinary study;discourse analysis;social practice;computer networks
The fact that the spoken texts of classroom interaction - particularly those involving the teacher with the whole class - are co-constructed relatively smoothly, despite the number of participants involved, suggests that they are organized in terms of standard strategies, embodied in typical forms of discourse that have evolved for responding to recurring types of rhetorical situation (Miller 1984; Kamberelis 1995). That is to say that, like written texts, they can be thought of as being constructed according to one of a set of educational genre specifications. One such rhetorical structure, the ubiquitous 'triadic dialogue' (Lemke 1990), also known as the IRE or IRF sequence (Mehan 1979; Sinclair and Coulthard 1975). It has attracted considerable attention in recent years, and has variously been seen as, on the one hand, essential for the co-construction of cultural knowledge (Heap 1985; Newman et al. 1989) and, on the other, as antithetical to the educational goal of encouraging students' intellectual-discursive initiative and creativity (Lemke 1990; Wood 1992). Drawing on episodes of teacher - whole-class interaction collected during a collaborative action research project, this paper will show, however, that the same basic IRF structure can take a variety of forms and be recruited by teachers for a wide variety of functions, depending on the goal of the activity that the discourse serves to mediate and, in particular, on the use that is made of the follow-up move.