Using a Spaced-Repetition-Based Mobile Learning Game in Database Lectures

Abstract

Forgetting is one of the common problems in learning. While students might grasp an idea when it is being presented in a lecture they might only vaguely remember it two weeks later and they might have completely forgotten it when the final exams are due. One way out of this is concentrated learning right before the test, also called cramming, massing or bulimic learning in order to highlight the fact that knowledge is only retained for a very brief time. Another approach is spaced repetition learning in which learning items are presented in growing intervals in such a way that the students never really forget and that knowledge is reinforced with each repetition. This paper presents a study in which learning items were incorporated into a mobile learning game in order to make the spaced repetition approach as transparent as possible to the students. The game was used in a database lecture and students reported good usability and learning efficiency.

Full text

USING A
SPACED-REPETITION-BASED MOBILE LEARNING GAME
IN DATABASE LECTURES
Florian Schimanke, Robert Mertens,
Florian Hallay, Arkadij Enders
Dept. of Computer Science
HSW University of Applied Sciences
Hameln, Germany
{schimanke, mertens, hallay, enders}@hsw-hameln.de
Oliver Vornberger
Dept. of Computer Science
University of Osnabrueck
Osnabrueck, Germany
oliver@uos.de
Abstract: Forgetting is one of the common problems in learning. While students
might grasp an idea when it is being presented in a lecture they might only vaguely
remember it two weeks later and they might have completely forgotten it when the
final exams are due. One way out of this is concentrated learning right before the
test, also called cramming, massing or bulimic learning in order to highlight the fact
that knowledge is only retained for a very brief time. Another approach is spaced
repetition learning in which learning items are presented in growing intervals in such
a way that the students never really forget and that knowledge is reinforced with
each repetition. This paper presents a study in which learning items were
incorporated into a mobile learning game in order to make the spaced repetition
approach as transparent as possible to the students. The game was used in a database
lecture and students reported good usability and learning efficiency.
Introduction
When it comes to organizing one’s own learning, there are different scientific approaches, which can
be followed. For students, “massing” or “cramming” often appears as the most promising approach because
they seem to make studying easy and fast due to their short-term performance. On the other hand, “spacing”
may appear as a much slower approach but is proven to having a better effect on long-term learning. We have
therefore chosen this approach for our research on improving mobile learning games regarding their
effectiveness and their long-term performance. In this context, “spaced repetition” refers to repeating the same
learning content after a certain amount of time over and over again, which helps flattening the so called
forgetting curve (Ebbinghaus, 1885) and therefore improving the retention of this content. In order to achieve
the best learning results, the intervals between repetitions of the same content should also increase the better the
learner remembers the correct answer (Pimsleur, 1967). There are already some sophisticated algorithms
available, which try to determine those intervals by judging the learner’s performance in past presentations and
combining it with other factors like the number of already done repetitions. One of the most widely spread
algorithms is called “SM2”, which was originally developed for learning with learning cards. However, in
earlier research (Schimanke, 2013) we have already shown that it can also be adopted to be used in game-based
learning and have therefore decided to use it in the learning game for this evaluation.
Those learning games are widely seen as a way to distribute learning content in a motivating and
engaging manner to the learners (Gee, 2003). However, spaced repetition may not fit well with all kinds of
learning games. While this approach appears promising for games which aim at providing fact-based
knowledge, it may not fit for learning games, which follow a more explorative approach. In this paper, we are
focusing on a learning game, which deals with the concepts of database design and entity relationship models.
This game uses a story-telling approach, creating a scenario, which needs to be implemented as a database.
Based on the given story, the students need to analyze a corresponding entity relationship model and fix a
problem or answer a question. In the background, the SM2 algorithm analyzes the provided solutions and
schedules the next repetition for the respective task. As we have shown in earlier work (Schimanke, 2014b)
using the strictly time-based SM2 algorithm alone may lead to corrupted calculation values if the learners
decide to play the game several rounds in a row. We have therefore integrated our round-based FS algorithm to
solve this problem.

Combining the motivating effect of learning games with the effectiveness and the long-term
performance of the spaced repetition approach promises a huge potential for optimizing the learners’
performance. In this paper, we want to analyze and evaluate if this is really the case. To do so, the mentioned
database learning game was provided as an iPad app to students of a database class at our university. We did not
make any rules how the game should be used but provided them with some background information about the
idea behind the spaced repetition approach. Every time the game was used, some values (like the time of play,
the next scheduled repetition and the learner’s performance) were collected in the background and stored on the
device. After the test period, which lasted from the second half of the semester to the closing test at the end of
the semester, the students sent us the logged data via e-mail for our evaluation.
Besides analyzing and evaluating the learning process of each student and the group in general, we
were also able to draw some conclusions about how and when the learning game was used and if the students
stuck to the calculated intervals for each task. After the test period, we also provided each student with a
questionnaire about how they evaluate their experiences with using our app compared to traditional learning
approaches and about the approach of spaced repetition compared to massing or cramming in general.
The remainder of this paper is organized as follows. In the related work section, we present the status
quo of related fields of research and some basic information about learning techniques like spacing, cramming
and massing before we lead over to our evaluation setup and a brief description of our prototype learning game,
which was used for it. After that, we take a look at the evaluation results before we discuss them with regards to
a questionnaire among the participants taken after the evaluation period about their experiences with our
approach. Finally, we draw a conclusion from our findings and give a forecast for future work.
Related Work
Organizing one’s personal learning often depends on subjective estimations about the desired goal and
the achieved effect. In this context a lot of learners get mislead by the short-term performance of their learning.
Studies have shown that learners tend to prefer massing or cramming over spacing because of the illusion that it
is faster and more effective (Baddeley 1978 and Kornell 2009). Those two techniques are also very common
when students prepare for a test. In this context, cramming is a special case of massing which describes learning
something intensely and often for the first time (Kornell 2009). In most cases, cramming leads to storing the
learning items in the learner’s short-term memory, which makes them reproducible in the test. A positive test
result then makes the learners believe that this is an effective learning strategy. However, according to Kornell
(2009), spaced learning is not only more effective than cramming or massing, it leads also to a better long-term
memorization. On the other hand, massing and cramming are often seen as a perfect example of the so called
“bulimic learning” (Bensley & Ellsworth, 1992), where students try to memorize as much content as possible,
only to regurgitate it again after a short amount of time.
Learning facts is often done by using flashcards (Kornell & Bjork 2008) and most experiments about
the effectiveness of massing, cramming and spacing are based on this technique. When learning with flashcards,
learners need to decide how many cards they want to study at a given time. The number of cards in a stack then
decides about the spacing between two representations of the same learning item. That means that the larger the
number of cards in a stack is, the larger is the spacing between repetitions of a given card when the last learned
card is always put on the back of the stack. This amount of time is also referred to as the within-session spacing.
Another decision affects the time between two learning sessions, which is the time between two (or more)
repetitions of the same stack of learning cards. This amount of time is also referred to as the between-session
spacing. When using paper-based learning cards, the learners must decide about the amount of cards within a
stack and by that about the spacing by themselves. However, when using computer-based learning cards, there
are already sophisticated algorithms like the “SM2” algorithm, which make those decisions for the learners and
schedule the next repetitions based on scientific evidence (Wozniak, 1994).
Making decisions about how to learn is often influenced by metacognitive judgments about one’s own
learning performance (Nelson, Dunlosky, Graf, & Narens, 1994). For students massing or cramming often
appears to be more effective than spacing, especially when preparing for a test on the next day. Therefore, the
time-component often has an effect on the within-session spacing as well as on the between-session spacing. As
mentioned above, another factor, which affects the learners’ decision, is the subjective estimation about the
general effectiveness of the respective learning technique. Using computer-aided algorithms can avoid these
misjudgments by the learners, as long as they stick to the calculated learning intervals.

There is, however, still a discussion about whether cramming or spacing leads to a better learning
performance. Studies have shown that students learning by cramming often achieve good results in exams
(Vacha & McBride, 1993). This may be one of the reasons why many students prefer this way of learning,
especially when preparing for a test (Sommer, 1968). Several experiments have also shown that the less time
and distraction there is between cramming and a test, the better the students perform during the test (Barrouilet,
Bernadin & Camos, 2004; Norris, Baddeley & Page, 2004). Since many learners tend to focus on the outcome
of a test or on school grades, achieving good results here might lead to the assumption that cramming is not
only a timesaving and effective, but also a promising learning technique. Indeed, Pease (1930) draws from his
experiments the conclusion that cramming not only leads to a better test performance but also to a better long-
term retention.
On the other hand, there are several studies that have proven spacing as the better way to contribute to
long-term retention, beginning with Ebbinghaus (1885). Kornell (2009) adds to that that spacing is also more
effective than cramming and Van Note (2009) found out that there is no difference in terms of test results when
interrupting the time between cramming and a test or not. While there is only little consensus about which
learning technique has a better effect on long-term retention, most studies found out that cramming has indeed a
positive effect on short-term memorization and is therefore popular among students when preparing for an
exam. However, looking at the forgetting curve by Ebbinghaus (1885) clearly shows that learning and
remembering is basically a matter of time and retention. He created a formula showing the degradation of
memories: R = e(−t/S) where R is memory retention, S is the relative strength of memory, and t is time. The solid
line in the following graphic (Figure 1) shows an example of this formula. As can be seen, with each repetition
the forgetting curve starts anew and thus gets flatter over time, which is represented by the dotted lines after
each repetition.
Figure 1: Alteration of the forgetting curve through repetition Ebbinghaus (1885) and Paul (2007)
This effect shows how important it is to repeat learning the same subject multiple times over a long
period with different intervals. Therefore, the more an item is reviewed and remembered correctly, the longer
intervals between the repetitions may be scheduled. Since the forgetting curve gets flatter over time it can
clearly be stated that spacing has a positive effect on long-term memorization.
Evaluation Setup
Learning games are widely seen as a way to distribute learning content in a motivating and engaging
manner to the learners (Gee, 2003). We therefore want to combine the motivating effects of a learning game
with the positive impact on long-term retention and the efficacy of spaced repetitions. While we have already

shown that spaced repetition algorithms can very well be used in learning games, there are also some hurdles to
be tackled. One question is how to encourage learners to use the game only at the calculated times according to
the spaced repetition approach. While playing games generally has a motivating effect, too much motivation
may lead to playing the game outside those times, which may undermine the idea behind this learning technique
regarding the spacing within and between the learning sessions.
In our experiment with using spaced repetitions in a mobile learning game, we want to completely
avoid any within-session repetitions while algorithmically determining the between-session spacing for each
learning item. In order to do so, we use the combination of a spaced repetition algorithm, a round-based
algorithm and different approaches to motivate the learners to only learn at the calculated times. As spaced
repetition algorithm, we use the well-known SM2 algorithm, which was actually developed for (digital)
learning cards and has already proven its effectiveness. It therefore seems to be a natural selection for
algorithmic content selection in a learning game.
However, in earlier research we already found out that there are some considerations to be made when
using SM2 in a learning game. Since SM2 is an approach strictly based on time between sessions, it faces some
problems when learners play the game (and by that a given learning item) several times in a row. In this case
the values used by SM2 to calculate and schedule the next repetition might get corrupted which leads to a
distortion of the desired learning intervals. We have therefore developed the FS algorithm (FS = Follow-Up
Sequence) as a helper algorithm for SM2, which takes over the content selection from SM2 if the learner
decides to play the game more than once at a scheduled repetition. Furthermore, the FS algorithm introduces a
flag, which locks the last played item in order to avoid back-to-back presentations of the same learning content,
which could make the game boring to the learners over time. However, since playing the learning game over
and over again would lead to a similar effect as massing or cramming, we want to avoid this behavior.
Therefore, we are informing the learners with pop-ups when they start the game outside the calculated intervals
and when they have reached the end of scheduled items for a learning session.
Setting
Our prototype learning game was distributed to all interested students which were in possession of an
iPad in a class about database concepts. Ultimately, ten students from the fourth semester participated in our
study. Five of them used their own iPads, five other iPads were given to students who also wanted to participate
but did not have an own iPad. Since our prototype learning game is currently iPad-only, we had a limitation of
the participants in our study based on the number of available iPads. The participating students were provided
with our prototype app through Apple’s Testflight portal. The game consisted of 25 stories and related
interactive tasks from the database domain. An example for that can be seen in Figure 2.
Figure 2: Example task from database learning game

An example for a story for the task in Figure 2 would be: “Paul has signed up for a new music
platform and wants to rate all his favorite songs now. Unfortunately, he is not allowed to add more reviews
after issuing his first one. Analyze the entity relationship model and solve the problem. To do so, please tap on
the wrong cardinality and select the correct one. To finish the task, tap on the “Done” button.” Figure 2 shows
how the corresponding task is designed.
After the given task was answered, the students get an immediate feedback whether their solution was
correct or incorrect. If the answer was right, the learners will automatically be taken to the next task as
calculated by the respective algorithm. In case of an incorrect answer, the learners are given the opportunity to
review the task, the correct solution, and their own provided answer and can then manually switch to the next
task. The participating students were also provided with a short explanation about how to use the game and
a little background information about the idea behind spaced repetitions. However, we did not make any rules
about using the game according to this approach. Therefore, the students were free to use the game whenever
they liked. Every time the game was played, we logged some values about the utilization of it. That data
involved for example the time of play, which algorithm was in charge, the next scheduled repetition and the
learner’s performance. After the test period, which lasted from half of the semester to the closing test at the end
of the semester, the students sent us the logged data via e-mail for our evaluation. In addition to the data
collected within the app, we also provided the participating students with a questionnaire about their
experiences with the database learning game and with spaced repetitions in general after the test period.
Evaluation
Our test period lasted six weeks and ten students took part in testing our prototype database learning
game. All participating students studied in their second semester and took a class in database concepts. After the
test period, all participating students returned the data logged within the app to us and took part in our
mentioned questionnaire. The data we logged within the app enfolded the following values for every time a task
was played:
Time of play The date and time when the current task was played
Next scheduled repetition The date and time for which the next repetition was scheduled
for the current task
Times played The number of times the current task was already played
E-Factor The value of the Easiness Factor for the SM2 algorithm
Score The value of the quality of response for the SM 2 algorithm
Relevance The value of the relevance, used by the FS algorithm for
content selection
Algorithm in charge Indicator which algorithm was in charge for content selection
in this round of play
Table 1: Values logged while playing the learning game
From this data, we were able to evaluate how and when the students used the app, whether they stuck
to the calculated learning intervals according to the spaced repetitions approach and how their performance
developed throughout the test period for each task.
As expected, all of the students did not always stick to the scheduled repetitions and played the game
outside of the calculated intervals. This could be observed in both directions. The game was played before the
calculated time had elapsed, as well as several days after a repetition was scheduled. Some reasons for that were
later revealed in our questionnaire after the evaluation period. However, playing the game outside the intervals
according to the spaced repetitions approach did not seem to have a negative effect on the students’ learning
progress since it developed positively throughout the test period either way. Almost all of the students were able
to solve the tasks correctly after the first repetition. This circumstance quickly raised the value of the score and
therefore made the interval quite big in a short amount of time.
However, it could be observed that there was a huge difference in how many times the students used
the learning game. While one student played the game 238 times, which means, he played each task 9.52 times,
another student played only six tasks, which means each contained task 0.24 times. On average the ten

participants played 96.5 tasks during the six weeks evaluation period or each task 3.86 times. It could also be
observed that the students seem to have lost interest in playing the game over time since the game was played
quite a lot during the first days after it was provided to the students and then more and more less towards the
end of the evaluation period. This happened despite notifications were displayed on the device reminding the
students to learn each time a calculated learning day for a task was reached.
From a technical point of view, we were able to see that both used algorithms (SM2 and FS) worked as
intended as the SM2 algorithm was only in charge when there was a scheduled repetition. Otherwise, the FS
algorithm took over the content selection and did not alter the values used by SM2. Therefore, the repetition
intervals stayed correct according to the spaced repetition approach used by SM2 and were not rescheduled
when the FS algorithm was in charge.
In addition to the algorithmic- and performance-data we logged within the app we also evaluated the
students’ overall opinion about the spaced repetitions approach, our prototype database learning game and their
general experiences throughout the test period. We therefore developed a questionnaire, which every test person
completed after that period. The test group consisted of six male and four female participants between 18 and
27 years of age. All of them study business informatics in their second semester and did not take a class in
database concepts before. 30% of the participating students had heard about the spaced repetitions approach
before and 80% of them believe that this concepts leads to a more efficient learning while 60% believe that it
leads to a generally better learning. All of the participants think that doing spaced repetitions over a longer
period of time leads to a better long-term memory. However, there are quite mixed opinions about whether
spaced repetitions alone are helpful for preparing for a test and whether the intervals between the learning
sessions are too long or too short. In the latter case, there is a tendency that the intervals were slightly too long.
Figure 3: Results from our questionnaire about spaced repetitions
Generally, all students found our prototype game to be fun and intuitively usable. There were no
technical issues in using the game and all of the participants believe that using a learning game similar to the
one provided is generally motivating. According to our questionnaire, the key aspects for a good learning game
are good user interface design, a good visualization and a motivating effect. Other important aspects are easy
navigation and general user friendliness. With that said, there are some improvements to be made in our
learning game. 60% of the participants felt that the graphical design of our game was satisfying but not good
and half of the participants think that there should be some improvements to be made about the on-screen
notifications when a learning day has been reached.
One of the most important parts of a learning game like ours is the content. While 70% of the
participants believe that the game generally helped them to better understand the underlying database concepts,
all of them also believe that a bigger amount of tasks and therefore more variation would be helpful both for the

learning success as well as for the motivation to use the game. The provided amount of content in our prototype
game (25 tasks) was only seen as satisfying. Furthermore, the level of difficulty of the tasks should be reviewed.
While 60% of the participants rated the level to be just right, 40% found the tasks too easy while none of the
students found them too hard.
Figure 4: Results from our questionnaire about the content in our prototype learning game
In our questionnaire, we also asked the participants about their experiences with a spaced repetition
based learning game especially on a mobile device. 90% of them believe that mobile devices are predestined to
be used in this scenario. This is mainly because of the on-screen notifications which inform the learner when it
is time to play the game again according to the calculated intervals by the spaced repetition algorithm. 80% of
the participants found this to be helpful. However, only 30% of them claim that those notifications helped them
to stick to the calculated intervals.
In a free-text field at the end of the questionnaire, the participants had the opportunity to provide us
with some additional information about their experiences with the learning game and the underlying concept as
well as with some criticism and recommendations. Some of the students pointed out that they do not use their
iPad on a daily basis, which is why they sometimes missed the notifications that it was time to learn. Providing
a learning game on a mobile phone would ease this problem since this is basically used more frequently. Some
participants also suggested that there should be explanations for a correct answer when a task was not solved
correctly. Currently the game only shows the correct solution but not an explanation. Generally, most comments
draw a positive picture about the provided concept. However, most of the students follow their own learning
schedule, which is not always in line with the calculated learning intervals according to the spaced repetitions
approach, which is another reason, why the game was not always played on the calculated days.
Discussion of the evaluation results
The evaluation results revealed several interesting facts about how our students used the learning game and
which impact this usage had on their learning progress. Despite showing alerts for different occasions related to
the spaced repetitions idea, those alerts seem to not always have had the desired effect. For example, we were
showing alerts each time a calculated time for learning was reached. However, this did not always make the
students play the game. According to some data from our questionnaire, this could partly be blamed on the
game being for the iPad only. Some students pointed out that they do not use their tablet every day and
therefore simply did not get the notification in time. Providing a learning game on a smartphone could ease this
problem since those devices are used far more frequently.
We also showed an alert every time the game was played outside of the calculated times and when all
items scheduled for that session were played. Those alerts also did not always have the desired effect. Students
were allowed to play the game anyway and did so almost every time. Some of them proposed that they started
playing the game at a particular time because they wanted to learn right then and not when the algorithm told

them. It can be said from that that students often follow their very own learning rhythm, which is only now and
then in line with the calculated learning times according to the spaced repetitions approach.
It can also be said that every student follows a different learning strategy. The appeal of the short-term
success, which can be achieved using massing or cramming often leads to preferring them over more long-term
oriented strategies like spaced repetitions. While 40% of the participants believe that spaced repetitions alone
are not a good way to prepare for a test, 90% them believe that it has a better impact on long-term memory than
the other two strategies. However, in a result-oriented society, most students prefer better grades over long-term
learning. Another reason might be that 70% of our students had never heard about spaced repetitions before and
therefore did not really know how to use this idea for their benefit.
The reasons for not sticking to the calculated intervals might also be related to that. Furthermore, some
students blamed the iPad-only restriction for that. Providing a learning game which relies heavily on alerts on a
smartphone therefore has a better chance of success than providing it on a tablet. However, this is always
depending on the respective content. For example, our database learning game needs to show a lot of content on
the screen. On a smartphone, users would have to zoom, pan, and scroll the content, which is not seen as being
user-friendly. User-friendliness on the other hand was seen as one of the key aspects, which can make a
learning game a success.
While 70% of the participants believe that the game was helpful in understanding the underlying
database concepts, 40% also mentioned that the provided tasks were too easy and insufficient in order to be a
real factor in preparing for a test. Furthermore, all of our students claim that a bigger variety of tasks would be
more motivating and helpful to understand the underlying concepts. Therefore, more tasks from a wider field of
content in different difficulties might be helpful here.
Interestingly all students showed a rising performance throughout the test period, despite not sticking
to the calculated learning intervals. There might be several reasons for that. On the one hand, this might occur
due to the tasks being too trivial and therefore easy to solve. On the other hand, the learning game was only one
part of the lecture and the students’ learning progress was also catalyzed by the lecture itself. Another reason
might be that every student has its own learning strategy, learning schedule and learning speed. This might be
individually better than sticking to the calculated times. Therefore, it can be said that while spaced repetitions
with calculated, fixed intervals offer a good starting point to improve one’s long-term memory, it might not
always be the best way of learning for every student, especially when they focus on an upcoming test.
Looking at the scheduling of the tasks based on the learners’ performance there should also be made
some improvements on the used algorithms. Especially the score, which is currently only incremented or
decremented, based on whether a task was solved correctly or incorrectly should be calculated in a more
sophisticated way to ensure a more accurate determination of the learners’ actual performance and therefore a
more precise scheduling of the next repetition of a given task.
It could also be observed that if a learner had solved a task correctly in a previous presentation, he kept
solving this task correctly in the following presentations. In fact, there was only one occurrence where a student
solved a task incorrect after solving it correctly before. It is unclear if this can be related to the general
improvement of the learners’ performance, to the well-designed tasks or to the level of difficulty of the tasks.
However, regarding the obvious effects on long-term memory, spaced repetitions in mobile learning
games are still a promising approach to facilitate those effects. While half of the participating students do not
believe that the alerts on the screen helped them to stick to the calculated intervals 90% of them also think that
using spaced repetitions on a mobile device is a promising idea. In that case, the smartphone has a clear
advantage over tablets since they are used more frequently, are always at hand, and can therefore provide the
alerts just in time. However, using smartphones raises several other questions regarding the design and the
visualization of the tasks and might therefore not always be suitable.
While the advantages of mobile devices are quite obvious for the spaced repetitions approach, the
participants also ruled out some key factors, which should be followed in order to make the mobile learning
game appealing to the learner. User-friendliness, good user interface design, good visualization and navigation,
and the potential for generating motivation were identified as important factors. However, making a spaced
repetition based mobile learning game too appealing to the user introduces new challenges, which may impact
the idea behind spaced repetitions as we have shown in earlier work (Schimanke, 2014a).
Conclusion and Future Work
As our evaluation and questionnaire have shown, spaced-repetition-based mobile learning games offer
a huge opportunity to create a motivation to learn as well as to strengthen the long-term memory of the learners.

However, they also pose special requirements to the design and the content. Especially the latter one needs a lot
of attention when creating a spaced-repetition-based mobile learning game since too little amount of content
might be de-motivating to the learners and might lead to a non-satisfying learning result. This also applies to the
difficulty level of the content. There should be a huge variety of tasks from different levels to keep the learners
motivated and to impart the knowledge as intended. The amount and the difficulty level of the content also have
an impact on the scheduling of repetitions according to the spaced repetition approach. If the tasks are too easy,
this may lead to a quickly rising score value which (among others) is used to calculate the learning intervals. On
the other hand, too little content might seduce the learners to play the game several rounds in a row which has
no effect on the scheduling of the next repetition thanks to the FS algorithm, but which is not in line with the
spaced repetition approach and might lead more to massing or cramming.
While spaced repetitions are generally seen as a good way to strengthen the long-term memory, we
were not able to prove that it leads to a better learning progress during our evaluation period. Students often
prefer massing or cramming (i.e. learning the same content repeatedly in a short amount of time) and often do
so shortly before a test, due to the illusion that it is more efficient regarding the time spent learning and the
outcome, which for example is the grade in a test. Even though the participants in our evaluation hardly stuck to
the calculated repetition intervals according to the spaced repetitions approach, all of them showed a positive
learning progress during the evaluation period. However, we were not able to tell whether this was due to the
tasks being too easy, the accompanying database class or our app. It is therefore hard to judge whether massing
and cramming are bad and spacing is good. There are several studies, which have already proven that spacing
has a positive impact on long-term memory. On the other hand, most students learn goal-oriented towards a test
or an exam to achieve a good grade, which makes massing and cramming a valid alternative. In order to do a
more long-term learning, there needs to be a change in the learning paradigm of those learners since every
student has his own learning schedule and learning speed. One way to sneak the spaced repetition approach into
there could be to blur this concept from the learner by not telling him about it and instead simply remind him to
learn with notifications when a calculated day to learn has arrived. However, this approach would need a deeper
consideration of the short-term memory effect massing or cramming have, since the students might not always
stick to the calculated times.
The findings of our evaluation provide a good groundwork for some future work on this topic. One
example would be to improve the auxiliary algorithm based on examinations about the impact of the user
playing several rounds of the game in a row on the retention. This should then be taken into consideration in
order to achieve an even better calculation of the next scheduled repetition. At the current stage, the prototype
apps alter the score for each category, which is used by the algorithm as “quality of response” only by right, or
wrong answers and increments or decrements the value accordingly. At a future stage, this altering should be
more sophisticated to better reflect the actual learning performance of the user. One approach to this could be to
analyze the time between the presentation of the content and the learner’s answer and then draw conclusions
from that. This will also go hand in hand with the aforementioned examination of the impact of playing several
rounds in a row.
In other future work our take on spaced repetition in learning games will be tested over a longer period
of time, with more and in difficulty varying content and with a bigger amount of participants, divided into a test
group and a reference group. This would enable us to evaluate the true impact of spaced repetitions on the
learning progress of the different groups. However, as we have seen in the evaluation at hand, there are still
several considerations and adjustments to be made in order to properly react on learners not sticking to the
calculated learning intervals.
In order to realize a wider test of our a approach we are also planning to release a framework which
contains our algorithms for repetition scheduling and content selection and the needed interfaces, which can
then be integrated in other learning games as described in Schimanke (2014b) and send anonymous data about
the learning progress and the app usage back to us for analysis. By this, we will be able to make a more
statistically significant evaluation to get a deeper insight in how learners would use our mobile learning game
and to further improve our concept.
References
Baddeley, A.D., & Longman, D.J.A. (1978). The influence of length and frequency of training session on the
rate of learning to type. In Ergonomics (Vol. 21, pp. 627-635)

Barrouilet, P., Bernardin, S., & Camos, V. (2004). Time constraints and resource sharing in adults’ working
memory spans. Journal of Experimental Psychology: General. 133, 83–100.
Bensley, R.J., Ellsworth, T (1992). Bulimic learning: a philosophical view of teaching and learning. J School
Health (Vol. 62(8), pp. 386–387).
Ebbinghaus, H. (1885). Memory: A Contribution to Experimental Psychology. New York: Dover.
Gee, J.P. (2003). What Video Games Have to Teach us About Learning and Literacy. New York:
Palgrave/Macmillan.
Kornell, N. (2009). Optimizing learning using flashcards: Spacing is more effective than cramming. Applied
Cognitive Psychology (Vol. 23, pp. 1297-1317). Wiley InterScience.
Kornell, N., Bjork, R.A. (2008). Optimizing self-regulated study: The benefits and costs of dropping flashcards.
Memory (Vol. 16, pp. 125-136).
Nelson, T. O., Dunlosky, J., Graf, A., & Narens, L. (1994). Utilization of metacognitive judgments in the
allocation of study during multitrial learning. Psychological Science, 5, 207–213.
Norris, D., Baddeley, A. D., & Page, M. P. A. (2004). Retroactive effects of irrelevant speech on serial recall
from short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 1093–
1105.
Paul, K. (2007). Study Smarter, Not Harder. Self-Counsel Press
Pease, G.R. (1930). Should teachers give warnings of tests and examinations? Journal of Educational
Psychology, 21, 273-277.
Pimsleur, P. (1967). A Memory Schedule. The Modern Language Journal, 51(2), 73-75, Blackwell Publishing.
Schimanke, F., Mertens, R. & Vornberger, O. (2013). What to learn next? Content Selection Support in Mobile
Game-based learning. Presented at the AACE E-Learn World Conference on E-Learning 2013, Las Vegas, NV.
Schimanke, F., Mertens, R. & Vornberger, O. (2014a). Designing for Motivation: Design-Considerations for
Spaced-Repetition-Based Learning Games on Mobile Devices. Presented at the AACE E-Learn World
Conference on E-Learning 2014, New Orleans, LA.
Schimanke, F., Mertens, R. & Vornberger, O. (2014b). Architecture Considerations for Spaced Repetition
Based Mobile Learning Games on iOS. Presented at the IEEE International Symposium on Multimedia 2014,
Taichung, Taiwan.
Sommer, R. (1968). The social psychology of cramming. Personnel and Guidance Journal, 47, 104-108.
Van Note, K. (2009). Cramming and Test Performance: An Investigation into the Efficacy of a Common
College Practice. Sentience, Vol. 2, 11-13
Vacha, E.F., & McBride, M.J. (1993). Cramming: A barrier to student success, a way to beat the system or an
effective learning strategy? College Student Journal, 27, 2-11.
Wozniak, P.A. (1994). http://www.supermemo.com/english/princip.htm. Accessed: March 08, 2015.