Content uploaded by Niall Sclater
Author content
All content in this area was uploaded by Niall Sclater on Aug 08, 2015
Content may be subject to copyright.
Center for Applied Research
Research Bulletin
Volume 2008, Issue 12
June 10, 2008
Large-Scale Open Source
E-Learning Systems at the
Open University UK
EDUCAUSE
Niall Sclater, Open University
4772 Walnut Street, Suite 206 Boulder, Colorado 80301 www.educause.edu/ecar/
Overview
The Open University (OU) is Europe’s largest higher education provider and one of a
handful of mega-universities1 across the world. As a distance education university that is
currently undergoing a process of increasing migration from paper to online, the OU
requires exceptionally feature-rich, robust, and scalable e-learning systems.
After extensive scoping and requirements-gathering activities, it was decided that
adopting and enhancing an open source system to form the heart of the university’s
learning management system (LMS) would be the most successful strategy. While this
made it possible to host the first course websites within months, building the full system
required a large amount of development. During a three-year, $10 million program of
work, almost all code developed has been given back to the open source community in
order to further enhance and boost the sustainability of the product.2
Adoption of the new instructional platforms and media is being accompanied by changes
to work practices for staff and faculty who are accustomed to developing learning
materials and tutoring using traditional methods. This research bulletin examines the
factors leading to the selection of the open source LMS at the OU, details the many
aspects of development work that had to be undertaken, and describes the issues
involved for institutions participating in an open source community. It also looks at some
of the many business and cultural challenges the institution has faced and at how faculty
are being encouraged to move toward a model of education incorporating increasing
amounts of e-learning content and activity.
Highlights of Open Source
E-Learning Systems
The Open University was founded in 1970 as the United Kingdom’s first major distance
learning provider. Since then it has become Europe’s largest university, with 180,000
students, a wide range of courses, and a reputation for high-quality content and student
support delivered through a network of 7,000 part-time tutors. While much of the course
content is still developed in-house for print, the Internet has become increasingly
important as a means of providing interactive content and for communicating with
students. Over the past decade, the OU developed considerable expertise in areas
such as online conferencing and e-assessment. These activities, however, were taking
place through a range of disparate systems, and there was increasing pressure to
consolidate them, to provide a unified design, and to increase the functionality available
to students.
Existing E-Learning Systems and the
Need for a Unified LMS
In 2004, planning began for the development and roll-out of an integrated LMS across
the university. An LMS (known as a “virtual learning environment” in the UK) is software
designed to facilitate learning over the Internet. It runs on a web server usually hosted
by an educational institution and is accessed by students using web browsers. Anything
2
that can be presented on the web, including text, graphics, sound, animations, and
videos, can be made available in the LMS. LMSs are different from other websites in
that institutions can restrict access to students enrolled in a particular course, present
materials automatically at staged intervals, and monitor who is accessing what.
In addition to providing access to content, LMSs enable learning to take place
collaboratively. They include forums and are increasingly incorporating other social
software such as blogs and wikis. The asynchronous functionality allows participants to
contribute and participate at times that suit them. Synchronous tools such as video
conferencing, audio conferencing, and instant messaging are now also being integrated
with LMSs, although these usually remain as separate systems outside the LMS. LMSs
also make it possible to allow students to interact with the computer using simulations or
online assessments with automated feedback.
Build, Buy, or Choose Open Source?
Extensive interviews with faculty about their requirements for LMS functionality produced
a long list of requirements, and it became clear that these went beyond the capabilities
of any commercial LMS. E-learning systems are arguably even more central to the
business of a distance learning provider than to an institution that also holds classes
face-to-face, although the increasing number of “hybrid” courses that require a
combination of face-to-face and online participation make e-learning systems important
across all sectors. At the OU there were concerns about the ability of vendors’ products
to cope with so many distance learners and tutors using an LMS at a scale unknown in
campus-based universities. Integration with a range of other systems at the OU, such as
an assignment-handling system, was a further complication. Opting for a less
customizable commercial package was considered to be an unacceptable risk to the
institution, and subsequently it has become clear both at the OU and elsewhere that
commercial systems that faculty perceive as inflexible can lead to resistance to the
adoption of e-learning. When faculty feel that they have greater control over the learning
environment, there appears to be increased acceptance of the LMS.
By mid-2005, having rejected the commercial options, the OU considered building its
own LMS by linking together existing systems for assessment, conferencing, and
content provision and then building or commissioning further functionality. However,
there were serious concerns that this would have resulted in an overly complex software
engineering project, requiring a need to understand, integrate, and maintain a range of
commercial, open source, and in-house systems. At that stage there were doubts that
web services technologies were at an advanced enough stage to facilitate such
integration at scale. Examples elsewhere of ambitious, new single-institution LMSs were
not encouraging; the high-profile UK eUniversity, for example, had recently failed,
bringing its expensive homegrown e-learning platform down with it.3
An open source development might therefore provide the answer. With the basic
framework in place already, there would be less reinvention of the wheel and reduced
danger of an unsustainable behemoth being created in-house. Further benefits would
include the engagement of the OU in a vibrant, worldwide community of developers and
users, and a product that was continually undergoing innovation and able to be
3
customized to the institution’s requirements. While the payment of commercial license
fees was not a primary consideration, the funds this freed up could be redeployed
toward enhancing the product, and further enhancements could either be outsourced or
be carried out by in-house developers rather than developed by expensive vendor
consultants.
A number of promising new collaborative and “free” LMSs and frameworks were gaining
ground, notably Sakai, initiated by a few leading U.S. institutions, and Moodle, led by
Martin Dougiamas from Perth, Australia. Dougiamas had managed to gather a large
developer and user community around him and, like Linus Torvalds of Linux fame, had
the skills and personality to harness the efforts of others to enhance the product. Such
leaders tend to understand the entire application, insist on optimizing the performance of
the product at every opportunity, can spot new requirements and ensure they are
fulfilled, and are natural leaders.4
The Open University used the Business Readiness Framework5 to assess the suitability
of four open source LMSs. This framework rates open source software according to
seven weighted criteria:
Functionality (how well the software provides basic LMS functions such as
quizzes and user management): 25%
Usability (how easily users become proficient in using the software): 20%
Documentation (the availability and quality of user-maintained documentation for
administrators and users): 15%
Community (the size and activity level of the community, as measured by the
e-mail forums and the numbers of people contributing code): 12%
Security (the number and severity of security alerts—and how quickly they are
addressed): 10%
Support (the amount and quality of volunteer and commercial support
available): 10%
Adoption (the number of installations and their size): 8%
Despite much debate as to whether the university should invest in Sakai, Moodle proved
the front-runner in every category of the evaluation. At the time of the evaluation, Sakai
was not functionally mature, and it had a complex architecture with a steep learning
curve. As well, it required the Oracle database system, which was not then supported at
the OU. Sakai required the payment of an annual membership fee to become part of
what was then a relatively small group of institutions. At the time, Moodle use was
growing rapidly by contrast, demonstrating a hopeful long-term future.
It should be noted that it is not always easy to obtain accurate figures to carry out
comparisons based on the criteria above, and it is not therefore an exact science. It is
also worth noting that Sakai is now a much more usable system and that its community
has become larger and more open, although it is still much smaller than that of Moodle.
4
While Moodle incorporates the standard features of an LMS, it is claimed to maximize
socio-constructivist learning; in other words, it focuses on users building their knowledge
and understanding in conjunction with other learners. It can also be argued that LMSs
are relatively pedagogy-neutral and are merely shells in which to place content and
activities. While Moodle may have reasonably sophisticated tools for communication, it is
also perfectly possible to use the system as a relatively uninspiring repository for PDF
files and PowerPoint presentations.
Enhancing the Product
In October 2005, the OU selected Moodle as the basis for its LMS and launched itself
into a tight schedule of work to enhance the product considerably, integrate it with
existing systems, and encourage its adoption among faculty during a three-year,
institution-wide program. Development work was structured into five categories:
integration with existing systems, bug fixing, infrastructural changes, enhancements to
existing modules, and new modules.
Integration with Existing Systems
The OU already had single sign-on for most of its student-facing systems, and this was
extended to include Moodle. Details of students and tutors, and which courses they are
allocated to are now transferred daily to Moodle. Integration of the LMS with other
systems including the student portal and the assignment-handling system is ongoing.
This integration involves extensive changes to graphic design and the combining of
systems owned by several groups; it is perhaps more challenging politically than
technically.
Bug Fixing
Moodle contained considerably more bugs than had been anticipated, and many
enhancements to the code base were made in the initial months by OU developers.
There is now an excellent issue tracking system provided at http://moodle.org/ that
ensures that bugs are recorded and fixed systematically by the Moodle community. The
system has proved extremely robust, with minimal periods of downtime caused by
problems with the local authentication system and the database system rather than by
Moodle itself.
Infrastructure Changes
The most significant infrastructure change required for Moodle to be usable at the OU
was the creation of a new roles and permissions architecture. Before Moodle there was
only a small number of roles available: student, teacher, course creator, administrator,
and non-editing teacher. Now it is possible to specify any number of roles and a fine-
grained level of permissions, making the system far more usable, particularly for larger
organizations with complex procedures. For example, on an experimental basis, some
UK prisons are intending to deliver OU courses to their prisoners through a restricted,
read-only access. Before these roles and permissions were established, prisoners
would have been unable to sign up for the growing number of OU courses requiring
Internet access. A “prisoner” role can now be added to Moodle to allow access to online
5
course materials without the possibility for engagement in forums, wikis, and so forth.
Thus, offenders’ access to educational opportunities is maintained while assuring
prison governors that offender learners’ opportunities for outside communication are
removed.
Many enhancements have been specified by OU accessibility experts to enable the
system to conform to international accessibility guidelines. Now, for example, the system
is far more accessible to visually impaired users with screen readers.
Another group of users—mathematicians and scientists—will benefit from a project to
enhance the input and display of mathematical and scientific content in Moodle. This will
result in significantly greater possibilities for the online teaching of mathematics and
science.
LMSs tend to be organized around courses with a defined group of students, usually
available for a specific duration. Universities also group courses into programs, and
sometimes information and tools must be provided at the program level. The OU has
extended its LMS to provide information, learning resources, and tools for a particular
subject community such as earth sciences, where students may come from a range of
backgrounds and be taking a variety of courses. Subject communities are a particularly
important means of increasing retention and reenrollment at the OU, where students
may take courses over a number of years and gradually build up enough credits for a
degree, with gaps of a year or more between periods of study.
Such “meta-courses” are relatively easy to implement in LMSs, but a more fundamental
change to the infrastructure of Moodle is currently under development at the OU. Tutors
and students will be able to set up ad hoc groups with wikis, forums, and other standard
Moodle functionality. This has the possibility to transform the ways in which LMSs are
deployed and to bring some of the student engagement and empowerment evident in
emerging social software into institutional e-learning systems.
There have also been other infrastructure changes, such as optimization of the
database system, Postgres. Much of this was carried out independently by a Moodle
partner organization in New Zealand, a direct benefit to the OU of using an open source
product. Meanwhile, as Moodle has grown in popularity, others have carried out the
development work to integrate it with the Microsoft SQL Server and Oracle database
systems.
Enhancements to Existing Modules
Moodle is organized into discrete modules such as the forum, wiki, blog, and quiz.
Requirements-gathering activities with faculty and tutors, and user testing with students,
identified many desired enhancements to these modules. Following accessibility and
usability testing, changes have been made to specific modules as well as to Moodle as a
whole. One of the most significant modules to be further developed by the OU has been
the quiz module. It now has improved navigation and design, and it has been integrated
with another open source system, OpenMark, to provide better support for mathematics
questions, and with a commercial system, Intelligent Assessment, for the automated
6
marking of short text responses. The OU has also specified and outsourced the
development of a much-enhanced gradebook and is currently developing a range of new
question types.
User consultations resulted in a number of improvements to the forum being built. Some
of these, such as the ability to summarize a range of messages and see which students
have read the messages, are used heavily by tutors who have been accustomed to
these features in the existing conferencing system, FirstClass, for many years. Such
features were so important to their practice that tutors would have been highly resistant
to switching from FirstClass to Moodle forums had they not been in place in the new
system.
Finally, one of the most useful parts of Moodle that has been further developed is the
calendar system. It is increasingly used as the springboard to access content and tools,
organized for students on a weekly basis. A checkbox facility was added so that learners
can keep track of the activities that have been carried out. These data are fed to tutors,
who may then be able to take supportive action for students who are behind in their
studies. New work processes are being devised to make effective use of this and the
other data being accumulated on student activity.
New Modules
It was considered easier to build from scratch than to adapt some of the Moodle
modules that did not provide adequate functionality, such as the voting module and the
wiki, or that were not being developed quickly enough in the open source community.
Other entirely new modules that did not already exist in Moodle have also been
developed. These include the e-portfolio system and an audio recording facility, mainly
for use by language students submitting recordings of themselves for assignments. Also,
an offline version of Moodle is being developed to allow students to access the LMS at
times when they do not have an Internet connection.
What It Means to Higher Education
Much discussion has taken place in universities throughout the world as to which LMS
should be chosen and what the feature set should be. The merger of Blackboard and
WebCT has sparked another round of debates about whether institutions should rethink
their LMS strategies. The selection of an LMS is only the start of the process, however.
As illustrated in this bulletin, there are many issues involved in integrating the LMS with
other enterprise systems in the institution and in encouraging faculty adoption. If an
open source solution has been selected, then the institution can benefit from full
engagement with the community behind that product, but there are drawbacks as well as
advantages in doing so.
Engaging with an Open Source Community
Engaging fully with the open source community around Moodle became a priority for
the OU, and a decision was made early to make freely available all the enhancements
the university was making to Moodle. This has increased the visibility of the university
7
internationally and has enhanced the OU’s opportunities for partnerships. It also means
that the product has become more sustainable and more useful with a greater number
of institutions using it, identifying and fixing bugs and adding their own innovative
features.
Ensuring that as many OU developments as possible are integrated into the core of
Moodle simplifies the task of merging code every time a new version of Moodle is
released. From a more altruistic perspective, giving back to the educational community
all the enhancements the OU has made to Moodle fits with the institution’s mission to
develop educational opportunities around the world.
The OU’s decision to opt for Moodle is regarded by many as the tipping point for the
system, and it has given Moodle added legitimacy for many institutions making LMS
procurement decisions.
Being part of the Moodle community has not, however, been without its difficulties.
Initially a mistake was made in tying internal release dates for new versions of the LMS
to the stated release dates of new versions of Moodle. Early on, functionality that the
university was depending on was often a key part of the next version of Moodle, and the
Moodle releases were inevitably delayed. Open source communities have timelines that
are somewhat more fluid than is typical in the commercial world, although even very
large companies have difficulties meeting stated deadlines for the release of new
systems.
A second issue has been the need to achieve consensus among the Moodle
community before adding new features to the core system. In the case of the quiz
module this was not a major problem for the OU because the university became the
official maintainers of that part of the system. Other aspects of Moodle are somewhat
more controversial, however. For instance, there has been considerable debate as to
whether the Moodle blog should incorporate comments or not. Some argued that blogs
were for personal reflections and that forums should be where comments are made.
Meanwhile, requirements gathering at the OU showed that commenting was considered
to be a key feature of a blog and that faculty and students were unlikely to use the
Moodle blog if it did not have this facility. Such debates can hold up developments for a
considerable time until consensus has been achieved. In the case of Moodle, the best
way to make progress is to convince its founder, Martin Dougiamas, who retains a high
level of respect among developers and users, though Dougiamas is continually
sensitive to the need to achieve a democratic consensus among the Moodle
community.
Another complication has been the existence of more than one module doing the same
thing within Moodle. One example was the wiki, a fairly basic and buggy system.
Although a “new wiki” was being developed in Spain and promised to be much better,
the OU had no control over this development, which was also taking too long. In the end
the OU built a new wiki module from scratch and now has the system it needs. The
community still has to decide whether Moodle adopts the feature-rich but apparently less
well-written Spanish wiki or the OU’s smaller but more robust system.
8
Although this option will be avoided if possible, the university could decide to
discontinue use of the core Moodle product and the university’s participation in the
Moodle development community while maintaining the version of Moodle that the OU
currently uses and developing the system on its own from that point forward. The
university would then have its own version of Moodle (it does to some extent already)
and would have to decide whether to continue to develop it internally or to start a new
open source community around this version of the system. It is conceivable, for
instance, that large-scale university requirements might necessitate a more complex,
robust, and scalable system than is required at the K–12 level. However, there is no
indication currently of a requirement for this, and relations remain excellent with the
Moodle community.
Encouraging Faculty Adoption
While the OU is a distance learning university and spends a large percentage of its
resources developing materials for print, there are increasing similarities with other
institutions that are attempting to change the culture to one where e-learning forms a
significant part of the student experience. Issues arising at the OU and their proposed
solutions are therefore detailed below.
Communication
A fundamental issue is that there is considerable lack of awareness among faculty,
media staff, and tutors about how to move from a primarily print-based educational
paradigm to one that also effectively exploits the dynamic, interactive, and
communicative aspects of the Internet. Many faculty have been engaged in developing
texts for a large part of their working lives and do not have the skills or inclination to think
about how to deliver parts of their courses as podcasts or wikis. This is being addressed
by an ongoing campaign of online and paper communications, meetings, and
awareness-raising events. Continual communication is required of 1) institutional
direction and policy in the area of e-learning; 2) successes (and failures) in e-learning,
both at the university and elsewhere; 3) new learning technologies deployed at the OU
and how they can be accessed; and 4) external funding opportunities for e-learning
projects.
Evidence Gathering
Linked to the communication strategy is an evidence-gathering activity involving case
studies of e-learning practice at the university and elsewhere. Evidence of success is
vital for faculty to see the relevance of deploying online pedagogies in their own
teaching. Improved statistics and reporting of who is doing what with the LMS will
enable easier identification of internal successes. A concerted intelligence-gathering
operation is planned, involving ongoing review of key journals and external project
reports and coordinated monitoring of conference papers by OU attendees. There is, of
course, a danger of information overload with this approach, so the lessons from
elsewhere will need to be carefully filtered to inform and reinforce the overall e-learning
strategy.
9
Skills Development
Training courses for faculty incorporating both face-to-face workshops and online
communication and materials have been developed and delivered. Face-to-face
workshops tend to have a higher participation rate than online training, perhaps
because faculty commit their time when they are physically extracted from their normal
working environment. The costs of providing such events for 7,000 tutors are
prohibitive, however, so online staff development courses will form a key part of the
future strategy.
Incentives
Although there is an e-learning strategy and broad encouragement from senior levels of
the university for faculty to develop more e-learning activities, course teams still have
limited incentive to produce courses that involve online learning. Some view the
publication of printed media as the key aspect of their role as academics, and they may
see less value in the development of online materials and activities. Faculty are more
likely to engage in developing their skills when they see the point of doing so. Enhancing
the learning experience for students is a big driver for those involved in teaching. There
may need to be more in it for individuals to move away from cherished working
practices, however. Incentives involving additional payments, changes to promotion
procedures so that career advancement is linked to e-learning innovation as well as
research output, and recognition for those involved in successful online practices are all
being considered.
Academics are, of course, increasingly working in the online world, and consequently,
there appears to be lower resistance to engaging with online learning. A key issue that
remains unresolved at the OU is how many features of the LMS should be restricted.
Ensuring central quality control, copyright clearance, branding, good design, and high-
quality audio recordings often mean that faculty and tutors feel that they have less
autonomy and can be less creative than they wish. Complaints have been made that
anyone can download a more feature-rich version of Moodle than is available for staff
and students at the OU. This is because many of the Moodle modules are not
considered to be sufficiently robust or accessible and therefore have not been released
for teaching and learning at the university.
Funding
The allocation of funding is another key mechanism to drive the implementation of
e-learning in courses. This is perhaps easier in distance universities where more funding
is allocated to the production of course materials than in campus-based universities
where lecturing remains a primary activity for faculty. New policies that stop funding for
courses that do not have considerable elements of e-learning would be an instantly
effective measure. The danger is that course teams see this as a box-ticking exercise:
place large amounts of textual materials online and fail to engage with more effective
forms of e-learning.
10
The OU’s financial model is based on large up-front investments in the production of
course materials, with much smaller budgets for updating materials once the course has
been produced. Budgetary allocation also needs to change to reflect the need for
ongoing updates to online courses.
Conclusion
The selection of the open source learning management system Moodle has enabled the
Open University to develop an effective online learning platform for its 180,000 distance
learners. Considerable financial investment was necessary to enhance the LMS to an
adequate level and to integrate it with existing university systems. These investments
have resulted in dramatic improvements to the underlying functionality of Moodle, both
at the level of individual modules such as the wiki and across the entire system in areas
such as accessibility and roles and permissions. Engagement with other users of the
open source system has in general been a very positive experience, with the OU
benefiting from enhancements developed elsewhere. Meanwhile, the university has
ensured that its improvements are integrated into the core product for the benefit of
others and the ongoing growth of the Moodle community.
As has been discussed, the development and roll-out of an LMS is only the start of the
process; changes to pedagogies, processes, and work patterns need to be addressed
with a strong commitment from senior management and at all other levels of the
institution to make effective use of e-learning. This is by no means a trivial task, and it
involves a reexamination of the communications processes, funding arrangements,
faculty incentives, and skills development, combined with evaluation and evidence-
gathering activities designed to ascertain which elements of the online learning
experience are of most benefit to learners.
All of this is being carried out in an atmosphere where the availability of high-quality, free
social networking tools and other Web 2.0 facilities elsewhere on the Internet is posing a
direct challenge to institutional strategies to deploy core e-learning technologies inside
the university. Arguments are being made that LMSs are tools of institutional control and
that the way forward for higher education is the development of personal learning
environments instead. Another ECAR research bulletin details the arguments emerging
in the blogosphere and elsewhere both for and against the LMS.6 It will examine whether
LMSs are destined to continue as the primary means of organizing the online learning
experience for university students.
Key Questions to Ask
What criteria should we use to assess whether our LMS is meeting our
requirements? Might we be better served by a different (possibly open source)
product?
What are the benefits and the challenges of our institution’s engaging with an
open source community, given its inevitable compromises and delays?
11
In what ways are we using our LMS to control the experience of learners, and
how are we using it to empower them?
How can we avoid getting tied up in discussions surrounding technologies and
keep our focus on finding solutions that enhance the learning experience for our
students?
Where to Learn More
Brooks, Lois. “Considering Open Source: A Framework for Evaluating Software
in the New Economy” (Research Bulletin, Issue 1). Boulder, CO: EDUCAUSE
Center for Applied Research, 2007, available from
http://www.educause.edu/ecar.
Dougiamas, Martin. “Moodle: a case study in sustainability,” OSS Watch.
http://www.oss-watch.ac.uk/resources/cs-moodle.xml.
Open University. OU Virtual Learning Environment FAQs.
http://conclave.open.ac.uk/ouvlefaq/.
Sclater, Niall. Virtual Learning Blog. http://www.sclater.com/blog/.
Endnotes
1. John S. Daniel, Mega-Universities and Knowledge Media: Technology Strategies for Higher Education
(Sterling, VA: Stylus Publishing, Inc., 1996).
2. Niall Sclater, “Putting the Open University on the Internet” (presentation from the 22nd ICDE World
Conference on Distance Education, Rio de Janeiro, Brazil, September 3–6, 2006),
http://ccvap.incubadora.fapesp.br/portal/materiais/462.pdf.
3. Barry Sheerman, comment on the release of the House of Commons Education and Skills Committee, UK
e-University: Third Report of Session 2004–05 (London, The Stationery Office, March 3, 2005),
http://www.publications.parliament.uk/pa/cm200405/cmselect/cmeduski/205/205.pdf.
4. Dan Woods and Gautam Guliani, Open Source for the Enterprise: Managing Risks, Reaping Rewards
(Sebastopol, CA: O’Reilly, 2005).
5. Business Readiness Rating, A Framework for Evaluating Open Source Software, 2007, available at
http://www.openbrr.org/wiki/index.php/Home
6. Niall Sclater, “Web 2.0, Personal Learning Environments, and the Future of Learning Management
Systems” (Research Bulletin, Issue 13) (Boulder, CO: EDUCAUSE Center for Applied Research, 2008),
available as of June 24, 2008, from http://www.educause.edu/ecar.
12
13
About the Author
Niall Sclater (N.L.Sclater@open.ac.uk) is Director of the Virtual Learning Environment
Programme at the Open University.
Copyright
Copyright 2008 EDUCAUSE and Niall Sclater. All rights reserved. This ECAR research bulletin is proprietary
and intended for use only by subscribers. Reproduction, or distribution of ECAR research bulletins to those not
formally affiliated with the subscribing organization, is strictly prohibited unless prior permission is granted by
EDUCAUSE and the author.
Citation for This Work
Sclater, Niall. “Large-Scale Open Source E-Learning Systems at the Open University UK” (Research Bulletin,
Issue 12). Boulder, CO: EDUCAUSE Center for Applied Research, 2008, available from
http://www.educause.edu/ecar.