Wikis: Do They Need Usability Engineering?

Abstract

Today, Web 2.0 is one of the most used buzzwords for a conglomerate of technologies within the elearning community. However, these are a new arrangement of standard technologies, the applications of which are primarily motivated by the slogan “the user is the content”. But are Wikis, Blogs, Podcasting etc. actually the great promise for the learning future? Studies and research work in this aera are still too rare for empirical evidence to substantiate these theories. In this paper we present the use of a Wiki system within a particular setting in Higher Education. Based on the model of Atkinson, we determined the influence of possible variables on this type of system. On the basis of two empirical studies, we are able to conclude that both lack of motivation and lack of usability are the main reasons why learners show so little enthusiasm for using wiki systems, thereby confirming the continued validity of the Atkinson’s model. Summarizing, we consider that the ease-of-use aspect is going to be the crucial factor of the Web 2.0

Full text

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
Wikis: Do they need usability engineering?
Michael Kickmeier-Rust 1)
Martin Ebner 2)
Andreas Holzinger 3)
Abstract
Today, Web 2.0 is one of the most used buzzwords for a conglomerate of technologies within the e-
learning community. However, these are a new arrangement of standard technologies, the
applications of which are primarily motivated by the slogan “the user is the content”. But are
Wikis, Blogs, Podcasting etc. actually the great promise for the learning future? Studies and
research work in this aera are still too rare for empirical evidence to substantiate these theories.
In this paper we present the use of a Wiki system within a particular setting in Higher Education.
Based on the model of Atkinson, we determined the influence of possible variables on this type of
system.
On the basis of two empirical studies, we are able to conclude that both lack of motivation and lack
of usability are the main reasons why learners show so little enthusiasm for using wiki systems,
thereby confirming the continued validity of the Atkinson’s model. Summarizing, we consider that
the ease-of-use aspect is going to be the crucial factor of the Web 2.0.
1 Introduction
Over the past years, we have been facing a tremendous increase in the spread of wiki-systems;
there is literally “a wiki hype”. They are used for knowledge management, online collaboration,
and establishing comprehensive encyclopedias both in and out of educational settings.
The concept of wikis was introduced by Bo Leuf and Ward Cunningham in 1995 [11]. The name
wiki was derived from the Hawaiian word wikiwiki which means quick. A wiki system is an online
platform which allows each and every user to create articles and also to edit, revise, extend, or link
existing articles. The initial aim was to develop an easy-to-use knowledge management system
enabling effective and efficient online collaboration. Wiki systems therefore provide markup
languages which are mostly based on HTML elements, basically reducing it to the very basics, or
they provide editors enabling the incidental creation of contents. Wikis are generally supposed to
be easily usable by all users, experts and novices.
1 Department of Psychology, University of Graz, Universitätsplatz 3, 8010 Graz, Austria
2 Computer and Information Services, WG Social Learning, Graz University of Technology, Steyrergasse 30/I, 8010
Graz, Austria
3 Institute of Medical Informatics, Statistics and Documentation, Medical University Graz, Auenbruggerplatz 2/5,
8036 Graz, Austria

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
The first implementation of the wiki principle was wikiwikiweb (www.c2.com/cgi/wiki) by Ward
Cunningham in 1995. Further implementations were CoWeb by Mark Guzidal in 1997 [9] and
Nupedia in 2000. Nupedia, originally, was a rather traditional approach to establish an online
encyclopedia where authors were experts and articles were reviewed in a seven-step editorial
process. Due to a lack of participants and the extensive review process, Nupedia failed in the end.
Nupedia’s successor was Wikipedia (www.wikipedia.org) which was established in 2001 by Jimmy
Wales. It fully incorporated the described principles of openness and freedom. Today, it is the most
successful wiki system. Due to the great success, the possibilities of the wiki principle were also
discovered for educational purposes and technology enhanced learning; to communicate, to
collaborate, or to contribute to a common pool of knowledge. Research has demonstrated that wikis
can successfully be applied in education, for example refer to [10], [3].
Besides the great freedom granted by wikis making a whole community quickly and easily
collaborating and sharing knowledge online, wikis do have a number of disadvantages and bear a
number of potential risks. First, even if wikis use simplified mark-up languages, some computer
skills are required and WYSIWYG editors are implemented sparsely. Second, the freedom and
openness makes wikis vulnerable for destructive activities and vandalism. Third, in many wikis an
organizing structure is lacking and thus orientating in the system and retrieving certain information
might be difficult. Finally forth, points of critique have been expressed concerning accuracy and
completeness of contents (because there is generally no review process), unclear expertise of
contributors, or a lack of citing resources of information.
2 Wikis, motivation, and usability
Despite tremendous spread and popularity, wikis were subject to usability studies very sparsely in
the past. This is remarkable since wikis claim to enable quick (“wikiwiki”) and easy use.
Consequently, the usability of wiki editors as well as the wiki itself is crucially important aspects
for voluntary and quick contributions. On the one hand, there is a body of research demonstrating
that wikis can successfully be applied to a variety of purposes, especially education [10], [5], [4],
[14]. On the other hand, however, there is some evidence that active contributions are rather sparse
when users are not “forced” to contribute to a wiki. This, for example, is the case in many studies
regarding wikis in education where students and pupils are “encouraged” by teachers, e.g. by
grades for contributions or because using and contributing to wikis is an obligatory work for the
class. In contrast, Ebner [7] found in two studies, carried out along a whole semester, that none of
in total 287 students actively wrote an article or edited an existing one. These findings are
supported by usage statistics of Wikipedia, which yielded that only 2.5 percent of Wikipedia users
actively contribute to the development and refinement of contents [17].
Reasons for the lack of active, voluntary contributions to wikis are likely motivational
reasons, mutually interdependent with additional factors like social components or usability. As
theoretical background, the motivation to voluntarily contribute to a wiki, either in form of creating
new articles or in form of editing existing ones, can be described in terms of achievement motive
[1]. Motivation for achievement-oriented behavior is seen as the result of conflicting tendencies of
approach and avoidance. Each achievement-oriented behavior is associated with the possibility of
success (e.g., the feeling of pride) and the possibility of failure (e.g., the feeling of shame) and thus
connected to hope for success and fear of failure. The tendency to strive for success is characterized
by the subjective judgment of the probability of success (Ws), the expected incentives (As), and an
individual disposition (Ms). In turn, the tendency to avoid failure is characterized by the subjective
judgment on the probability of failure (Wf), the expected consequences of failure (Cf), and again an
individual disposition (Mf). The subjective expected utility (SEU) is considered to be an additive

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
function of both tendencies. Atkinson & Litwin [2] found evidence for the validity of the model in
a ring-toss experiment. In an interesting longitudinal study, McCelland & Franz [12] could
demonstrate that a person’s approximately financial income could be predicted by this model quite
well ten years in advance. Weiner [15], [16] extended the model by the component of attribution of
cause (internally vs. externally).
This model can be considered as an underlying principle for the impact of expected ease or
difficulty of using/contributing to a wiki. In summary, Atkinson’s model claims the tendency (T)
for an achievement-oriented behavior is given by
T = Ws * (Ms * As) + Wf * (Mf * Cf).
Consequently, the expectation that contributing to a wiki is difficult decreases Ws (the expected
probability of success) and increases Wf (the expected probability of failure) at the same time.
Because on the one hand, individual dispositions are independent from the usability of a wiki or a
wiki editor respectively, Ms and Mf are constant terms. If we hypothesize that the incentives of
success (As), for example social acknowledgment, and the subjective judgment on the
consequences of failure (Cf) are balanced, we recognize that the tendency or motivation to actively
contribute to a wiki is tremendously depending on the usability of a wiki and the belonging editor.
Very slight changes in the judgment of probability for success or failure can change the behavioral
tendency into a positive or negative value. Unfortunately, wikis are not particularly known for their
ease of use. Generally, WYSIWYG editors are implemented sparsely, thus, a certain amount of
expertise is required to use the mostly simplified markup languages.
Surprisingly, only a very little number of studies focused on the usability of wikis (e.g [6]), in terms
of navigation through the body of existing articles as well as regarding the possibilities to create or
edit articles. Sometimes is assumed that the broad range and vast number of contributions to
existing wikis might serve as a sufficient indicator for usability (cf. [6]). And sometimes it is
argued that usability would be a “nice to have” for wikis. The literature concerning wikis lack an
endeavor investigating an improving the usability of wikis. Moreover, specific usability
characteristic for evaluating and describing wikis is lacking.
The focus of the present work is based on the theoretical background regarding motivation,
as outlined above. As a hypothesis, the achievement motive is significantly determined by a user’s
estimation of the probability of success in attempting to contribute to a wiki, and by the estimated
probability of failure in this attempt. Consequently, a user’s expectation of usability might crucially
impact the contribution frequency. More evidence for this idea comes, for example, from Fries [8]
who found that the label High-End made students learn better with an eLearning system and
evaluate it better in contrast to the label Beta-version. This clearly shows the importance of
expectancy.
3 Experimental Setting
As a first step in order to clarify the importance and impact of expectancy on user behavior and
judgments regarding usability of a wiki we analyzed the results of two studies. The data were
collected in the framework of studies regarding the pros and cons of using wikis in higher
education settings.
3.1 Experimental Design

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
The experimental basis for the present data was two studies conducted during a whole semester
each at the University of Technology Graz and the University of Applied Science FH Joanneum
Graz. In the first study students of both universities were supposed to collaboratively collect
knowledge from the subject structural concrete, in the second students from at the University of
Technology Graz should collect knowledge from the subject of visual basic for applications (VBA).
Students were supposed to use the wiki during a whole semester. As basic motivation students were
allowed to use the wiki’s content in the written exam. The aim of both studies was to record usage
data across the semester and uncovering factors influencing the amount of usage.
At the beginning of the semesters we distributed questionnaires to gather information like
the previous knowledge about structural concrete, use of Wiki in general and about teamwork for
learning, during the semesters we recorded usage statistics from the wiki systems, and finally at the
end of the semester we distributed questionnaires asking for the general use of the wiki. To provide
an initial basis of articles, in study 1 we created 30 articles and in study 2, as a tutorial, students
were required to create one article at the beginning of the semester.
In total, 287 students from both universities participated in the studies. Due to a limited
response rate for questionnaires we analysed the data of 165 students.
3.2 Bauwiki
Figure 1 Screenshot of Bauwiki used for both studies.
We used a TWiki system for both studies, which is a freely available system (www.twiki.org) and
which is one of the largest and most powerful systems among the approximately 200 existing wiki
species. TWiki is based on cgi-bin scripts developed in PERL. PERL (Practical Extraction and
Report Language) is a general-purpose programming language, having as its biggest strength,

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
excellence in the area of string and data processing, since most programs process strings or data or
both. It provides access to C library functions for fine-grained control of files, processes, and
network sockets, while, at the same time, handling the details of low-level memory management
and offering powerful built-in data structures [13]. Moreover, TWiki is driven by the numerous
available plug-ins, for example to insert formulas (LaTeX), tables, animations, or java applets. The
implementation of Twiki for both studies was called Bauwiki (Figure 1).
3.3 Bauwiki’s Markup Language
Editing or writing an article is easily. The TWiki markup language is very similar to usual HTML
tags or in other words there are only few describing pattern an editor has to know. For example if a
heading is needed only three dashes and a plus (---+) followed by the text have to be entered.
Words get bold by enclosing them in asterisks (*), italic by enclosing them in underscores (_). If a
list item is necessary the markup language code is therefore: 3 spaces and an asterisk.
Unbelieveable but with only few tags the whole text will be automatically formatted and each
contribution looks the same as each other.
One of the disadvantages as we mentioned above is the lacking of organized structures. But the
basic idea was to build a highly hyperlinked system, without a prior structure. The user themselves
should organize the content by creating a totally free linked system. Of course this means that
implementing a link or even a hyperlink must be a simple as possible. Therefore TWiki uses the
CamelCase syntay in the same way as all other famous Wiki clones do. If a word is written in
CamelCase style (at least two words written without spacing, but with beginnin Upper Case letter;
for example: TwikiWord) an internal link will automatically created. This link points to a page with
the same name. If this topic already exists only the link will be installed, if the topic doesn´t exist
the user is forced to write something.
Further funktioncs of the markup language are the possibilty to implement hyperlinks, tables, lines
and so on.
4 Results
Both studies revealed quite surprising results. While 79.22 percent of students in study 1 and 95
percent in study 2 passively accessed articles from the Bauwiki, none of the 152 students of study 1
and the 135 students of study 2 actively added a new article or edited an existing one (refer to
figure 2). Although, as in most previous studies, students were not forced to contribute to the wiki,
we provided the incentive of allowing the use of the wiki during written exams.
As shown in Figure 2, reasons for not actively contributing to the wiki were in study 1, in
summary, related to usability problems or expected usability problems (56 percent) and related to
motivational factors (36 percent). Because of additional measures (each student has to write at least
one article) in study 2 the arguments “didn`t try” and “problems editing articles” get redundant, as
Figure 2 show. But the proportion between motivation and usablity problems was still the same
(42% to 47%).

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
Figure 2 Reasons for not adding or editing articles in Bauwiki
Further analyses concerned the ease-of-use of the wiki and the quality of articles. The data yielded
that the average rating was 1.70 (SD = 0.83) on a scale from 1 to 5, where 1 was the best rating and
5 the worst. Additionally, the average quality of articles was rated with 2.63 (SD = 0.78). More
interestingly for the aims of the present work is the student’s expectation regarding the ease-of-use
and the efforts of time required to create and edit articles prior to using the wiki. In total 54.70
percent of students expected to need less than half an hour practicing to get familiar with the
system, 11.11 percent expected to need half an hour to an hour, and finally 34.19 percent expected
to need more than one hour to get familiar with the wiki. In this respect, we also analysed the
relationship between prior experiences with wikis and their use and the estimated time to get
familiar with such system. A Spearman-Rho correlation yielded r = 0.44, which is significant on the
1 percent level.
5 Discussion
The data of both studies underlying the present work revealed surprising results. Although to ease-
of-use of Bauwiki was judged rather positive, and the quality of existing articles on a medium level,
none of the in total 287 students actively added a new article or edited an existing one. The reasons
reported were closely related to a lack of motivation and a lack of usability.
The aim of this work was to make a first step to focus on the students’ expectations
regarding usability and possible incentives of contributing. The present results can be incorporated
in the achievement motive model of Atkinson [1], introduced initially. Almost the half of the
students expected to need more than 30 minutes to get familiar with the wiki. If we interpret these
results as a generally high expectation of difficulty, in terms of Atkinson this means a rather high
probability of failure and a rather low probability of success. Such interpretation is emphasized by
the fact that 56 percent (or 47 percent in study 2) of students reported usability-related reasons for
not contributing (Figure 2). In the same way, reported motivational reasons for not contributing can
be mapped to the incentives of Atkinson’s model. The fact that 36 percent (or 42% in study 2) of
students reported such motivational aspects, e.g. 27 percent (in study 2 even 42%) reported that

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
there no incentives of contributing to the wiki, is a clear indication, that wikis in a limited
educational scenario, as in the current studies, are not able to spur students on contributing in an
active way. – Even if students could potentially benefit from such contribution in the writing
exams. Summarizing, with a high expectation of failure, a low expectation of success, and a low
amount of incentives, the model would predict a behavioral tendency to desist from active
contribution. – And exactly that’s what we found in the present data and what is also indicated by
the comparably small percentage of Wikipedia user who actively contribute to the system [17].
These results and interpretations give some evidence for the crucial importance of usability
and expected usability of wikis. However, this importance is still underestimated. Some authors [5]
argue that increasingly implementing WYSIWYG editors is not necessary. Désilets [6], one of the
few who conducted usability studies with wikis, for example, argues that it would be a good result
that a class of 15 students could use a wiki after 30 minutes of training to collaboratively web-
based stories. At the same time these authors encountered a significant number of usability
problems where one third to one half was catastrophic. We argue that these results are alarming,
especially because the task in this study was rather simple and students performed this task in the
classroom meaning that they were required to do so.
Thus, from our point of view and based on the current results, future work must increasingly
address usability of wikis. This is true for wiki editors and markup languages but also for the
structure of contents, navigation, or visual presentation formats. To give an example, an interesting
approach might be an evaluation or comparison of different markup languages used in wikis.
Moreover, the SEU-model of achievement motivation introduced in this work offers an interesting
basis to predict user behavior and to empirically investigate the usability of wikis.
6 References
[1] Atkinson, J.W. (1957): Motivational determinants of risk-taking behavior. Psychological Review, 64 (6), 359-372.
[2] Atkinson, J.W., & Litwin, G.H. (1960). Achievement motive and test anxiety conceived as mtive to approach
success and motive to avoid failure. Journal of Abnormal and Social Psychology, 60, 52-63.
[3] Augar, N., Raitman, R., & Zhou, W. (2004). Teaching and learning online with wikis. In: Atkinson, R., McBeath,
C., Jonas-Dwyer, D., & Phillips, R. (Eds.). Beyond the comfort zone:
[4] Boyer, E.L. (1990). Campus life: In search of community. San Francisco: The Carnegie Foundation for the
Advancement of Teaching.
[5] Chau, T. & Maurer, F. A Case Study of Wiki-based Experience Repository at a Medium-sized Software Company.
In Proceedings of Knowledge Capture 2005, October 2-5, 2005, Banff, Canada.
[6] Désilets, A., Paquet, S., & Vinson, N.G. (2005). Are wikis usable? In Proceedings of the 2005 International
Conference on Wikis, San Diego, CA, New York: ACM Press.
[7] Ebner, M., Kickmeier-Rust, M.D., Holzinger, A. (2006). Utilizing Wiki-Systems in Higher Education Classes:
Success or Failure? A Case Study. Submitted to Universal Access in Technology Enhanced Learning (UATEL),
Special Issue of Springer International Journal Universal Access in the Information Society (UAIS).
[8] Fries, S., Horz, H. & Schremmer, S. (2002). Pygmalion in front of the Screen: Effects on Quality Expectations on
Learners’ Performance. Paper prestented at the 5th Workshop of the SIG 6 Instructional Design in the European
Association for Research on Learing and Instruction (EARLI), Erfurt.

The final version is available in:
M3 – Interdisciplinary Aspects of Digital Media & Education, (2006) Pohl, M., Holzinger, A., Motschnigg, R., Swertz, C.; Conference Proceeding of
the 2nd Symposium WG HCI&UE, Österreichische Computergesellschaft, Wien, S. 137-144
[9] Guzdial, M., Hmelo, C., Hübscher, R., Nagel, K., Newstetter, W., Puntembakar, S., Shabo, A., Turns, J., &
Kolodner, J. L. (1997). Integrating and Guiding Collaboration: Lessons learned in Computer-Supported
Collaborative Learning Research at Georgia Tech. In R. Hall, N. Miyake, & N. Enyedy (Eds.), Proceedings of
Computer-Supported Collaborative Learning '97 (pp. 91-100), Toronto, Ontario, Canada.
[10] Guzdial, M., Rick, J., & Kehoe, C. (2001). Beyond adoption to invention: Teacher-created collaborative activities
in higher education. The Journal of the Learning Sciences, 10(3), 265-279.
[11] Leuf, B., & Cunningham, W. (2001). The Wiki Way. Quick Collaboration on the Web. Addison-Wesley.
[12] McClelland, D.C., & Franz, C.E. (1992). Motivational and other sources of work accomplishments in mid-life: A
longitudinal study. Journal of Research in Personality, 60, 679-707.
[13] Prechelt, L. (2003) Are scripting languages any good? A validation of Perl, Python, Rexx, and Tcl against C, C++,
and Java. Advances in Computers, Vol 57. San Diego, Academic Press Inc.
[14] Raitman, R., Augar, N., & Zhou, W. (2005). Employing Wikis for Online Collaboration in the E-Learning
Environment: Case Study. In Proceedings of the Third International Conference on Information Technology and
Applications (pp. 142-146), IEEE Computer Society, July 04-07, 2005, Sydney, Australia.
[15] Weiner, B. (1974). Achievement motivation and attribution theory. Morristown, N.J.: General Learning Press.
[16] Weiner, B. (1980). Human Motivation. NY: Holt, Rinehart & Winston.
[17] Wikipedia (2006). Wissenschaftliche Analyse. Retrieved August 10, 2006 from
http://de.wikipedia.org/wiki/Wikipedia#Wissenschaftliche_Analys