Learning about web accessibility: A project based tool-mediated approach

Abstract

Many websites remain inaccessible for people with disabilities, despite the availability of relevant guidelines and tools. This is mainly due to lack of appropriate training of Web designers on accessibility technology. In this paper, a project based learning activity designed to instruct Web accessibility guidelines and good design practices is presented. The activity is mediated by a web-based learning environment, which presents real-world examples of accessibility impasses that arise when certain, established guidelines are violated, and then provides advice on how to avoid or resolve them. The learning material contained in the tool is offered through a faceted browsing approach, thus enabling active exploration by the learner. A within-subjects case study compared the learning effectiveness of traditional academic instruction (pre-condition) with the proposed project based activity (post-condition) in the context of a University course. A significant improvement in students’ academic performance and perceived learning was found.

KeywordsHuman–computer interaction–Web accessibility–Project based learning–Educational tool–Accessibility study

Full text

Learning about web accessibility: A project based
tool-mediated approach
Christos Katsanos1*, Nikolaos Tselios1,2, Athanasios Tsakoumis1, and Nikolaos Avouris1
1HCI Group, Dept. of Electrical and Computer Engineering, University of Patras,
Rio, Patras, 26500, Greece
2ICT in Education Group, Dept. of Educational Sciences and Early Childhood Education, University of
Patras, Rio, Patras, 26500, Greece
Abstract Many websites remain inaccessible for people with disabilities, despite the availability of
relevant guidelines and tools. This is mainly due to lack of appropriate training of Web designers on
accessibility technology. In this paper, a project based learning activity designed to instruct Web
accessibility guidelines and good design practices is presented. The activity is mediated by a web-based
learning environment, which presents real-world examples of accessibility impasses that arise when
certain, established guidelines are violated, and then provides advice on how to avoid or resolve them.
The learning material contained in the tool is offered through a faceted browsing approach, thus
enabling active exploration by the learner. A within-subjects case study compared the learning
effectiveness of traditional academic instruction (pre-condition) with the proposed project based
activity (post-condition) in the context of a University course. A significant improvement in students’
academic performance and perceived learning was found.
Keywords Human−computer interaction; Web accessibility; project based learning; educational tool;
accessibility study.
1 Introduction
The Web is used by a constantly growing number of people of different ages, cultures, education, and
with different physical and cognitive abilities. As a result, it is important to strive for equal access to
the Web and provide the same opportunities for all people (Henry 2005). Web accessibility refers to the
practice of making websites usable by people of all abilities and disabilities (Clark 2003). According to
the definition used in Section 508 of the Rehabilitatition Act of 1973, as amended by the US Congress
in 1998: “Web sites are accessible when individuals with disabilities can access and use them as
effectively as people who don't have disabilities” (Slatin and Rush 2002). Moreover, increased
accessibility for people with disabilities usually leads to improved usability for all users (Petrie and
Kheir 2007).
People with disabilities typically use assistive hardware and software technologies to access the
Web. For instance, people who are blind may be accessing a webpage using a screen reader, such as
JAWS, that transforms the content of the computer screen into synthesized speech. As a second
example, people who have motor disabilities may be using a non-traditional input or output device to
access a website, such as a head-mouse or a Braille display. However, assistive technologies are only
helpful if the website is designed in a way that allows access through such technologies. Nielsen (1996)
argues that “making the Web more accessible for users with various disabilities is to a great extent a
matter of using HTML the way it was intended: to encode meaning rather than appearance”. For
instance, HTML heading tags are often used to serve presentation purposes instead of conveying
document structure. This approach invalidates the headings-based navigation mechanism provided by
typical screen readers. As a second example, using device-dependent event handlers, such as
*Corresponding author
e-mail: ckatsanos@ece.upatras.gr
Please cite this do cument as: Katsanos, C., Tselios, N., Tsakoumis, A., & Avouris, N. (2012). Learning about web accessibility: A
project based tool-mediated approach. Education and Information Technologies, 17(1), 79-94. doi: 10.1007/s10639-010-9145-5.
Note: This is the author’s-created version of the paper. The publishers’ version (i.e. Springer) is available at
http://www.springerlink.com/content/3402t5k05338w66u/

“onmouseover”, can render parts of a website’s content inaccessible to people who cannot use typical
input devices (Clark 2003).
Key knowledge about developing accessible websites comes into the form of guidelines. The two
most widely-known sets are Section 508, a USA law (www.section508.gov), and Web Content
Accessibility Guidelines (WCAG) (Caldwell et al. 2008; Chisolm et al. 1999) published by W3C, the
international organization for Web standards. Various countries, such as Canada, Philippines, Spain,
Sweden and United Kingdom, have also produced national accessibility guidelines (Harper and
Yesilada 2008). In addition, many software tools that evaluate conformance of websites to such
guidelines have been also developed. Representative examples of such tools are IBM-RPT (former
WebExact), and AChecker (www.atutor.ca/achecker).
However, despite the abundance of available guidelines and tools, studies show that at least 70% of
websites have major accessibility problems (Ceaparu and Shneidermann 2002; Hacket et al. 2003;
Sullivan and Matson 2000; Zaphiris and Ellis 2001; Zaphiris and Zacharia 2001). A study (Lazar et al.
2004) involving 175 webmasters identifies lack of training, lack of managerial support, lack of client
support and confusing guidelines as some of the major obstacles to developing more accessible
websites.
One of the main problems of guidelines is that they are often stated at such an abstract level that
tends to make unclear how to operationalize them (Ivory et al. 2001). In addition, they come in a
context-independent form that tends to make their rationale hard to understand. Furthermore,
conceptualization of the importance of each guideline varies according to the type of disability the
users experience and the technology used to access the provided information. Moreover, guidelines
usually come into long, formal documents that can discourage practitioners from investing time and
effort to read them. Therefore, new ways to communicate the value of Web accessibility to Web
development stakeholders and train them on good accessibility practices are required.
A suitable approach to address the aforementioned need for accessibility education seems to be the
project based learning approach (Duffy and Kirkley 2004). Project based learning (PBL) is a dynamic
approach to teaching, in which learners explore real-world problems and challenges (Bloomenfeld et al.
1991). With this type of active and engaged learning, learners are inspired to obtain a deeper
knowledge of the subjects they are studying. PBL is increasingly adopted in the context of e-learning
(Duffy and Kirkley 2004). This is due to its strong underlying theoretical basis and to the fact that “it is
an instructing-learning pattern that combines curricula, teaching methods, and assessment into one
single unit” (Lee and Tsai 2004).
PBL is characterized by a number of positive outcomes for the learner. In specific, it engages
learners in complex, real-world issues and problems, requires learners to use inquiry, research,
planning skills, critical thinking, and problem-solving skills while in the effort to complete their
assigned project (Felix 2005). It requires students to learn and apply content-specific skills and
knowledge in a variety of contexts as they work on the project. It provides opportunities for students to
learn and practice interpersonal skills as they work in cooperative teams (Barab and Duffy 2000). It
gives students the chance to practice in using the array of skills needed for their careers, such as
resources allocation, individual responsibility, interpersonal skills, and learning through experience.
PBL also triggers reflection activities that lead students to think critically about their experiences. It
usually ends with a presentation or product that demonstrates learning which is assessed.
In this paper, we present a project based learning activity mediated by an educational tool that aims
to increase awareness, motivate and educate Web development stakeholders on Web accessibility. It is
argued that such an educational approach could substantially enhance understanding and promote
learning in an effective and efficient manner. To investigate the aforementioned hypothesis, a suitable
case study was designed. The participants of the study were students of an Electrical and Computer
Engineering University Department attending a Human Computer Interaction course. The remaining of
this paper is structured as follows. First, the proposed tool-mediated, project based activity to support
accessibility learning is delineated. Next, the details of the case study that was conducted to investigate
the learning effectiveness of the proposed educational approach are described, followed by conclusions
and directions for future work.

2 Design of a tool-mediated project based activity to support accessibility learning
The goal of the proposed activity is to increase awareness and educate people on Web accessibility.
The activity is addressed to both Web practitioners and students, who currently shape their future Web
development practices. The activity is mediated by an appropriate web-based educational tool, the
Educational System to support Accessibility Learning through Paradigms (ESALP), which is described
in the following.
2.1 Educational System to support Accessibility Learning through Paradigms (ESALP)
ESALP is a web-based educational tool that was developed to support the following high-level
requirements:
1. It should expose learners to the accessibility impasses that arise when certain guidelines are
violated. In this way, they can have a clear picture and long-lasting impression of the
problematic situations that occur, and get motivated to develop good accessibility practices.
Usage of various complementary learning material (e.g. text, photo, videos) substantially
improves learning effectiveness of e-learning environments (Clark and Mayer 2008; Mayer,
2005, 2009; Psaromiligkos and Retalis 2003; van Gog et al. 2009; Wouters et al. 2008).
2. It should provide concise advice on how to avoid or resolve each problem (van Gog et al.,
2006). Long documents describing solutions in an abstract and context-independent way do not
seem to fit to the busy schedules and problem-oriented thinking of Web practitioners.
3. It should cover at least the most widely used set of available Web accessibility guidelines.
4. It should organize the provided material in a structured and flexible way so that the users of the
tool can follow their own learning paths (Tselios et al, 2008b). Additional links to external
resources should be also provided for further exploration.
An example-based learning approach accompanied by concise advice on how to avoid or remedy
the problems that arise when accessibility guidelines are violated was deemed as an appropriate scheme
to achieve the goals of ESALP (Sweller and Cooper 1985; van Gog et al. 2006, 2008, 2009). In this
way, ESALP can support “just in time learning” targeted to the assigned activity (Duffy and Kirkley
2004). Examples are often used for teaching good design practices and guidelines in both Human
Computer Interaction (HCI) and Software Engineering (SE) fields. For instance, the book by Koyani et
al. (2004) illustrates good and bad Web design practices with heavy emphasis on real-world examples.
An additional benefit of the aforementioned design choice is that educators and professors can also use
ESALP as an informal learning tool or as accompanying reference material in their accessibility
courses or training modules (Brown and Duguid 2000). Web practitioners can also use ESALP to
communicate in an easy way the need for accessibility to clients and managers. If all stakeholders
become convinced of the value of Web accessibility, then it is more likely that an accessible website
will be developed (Lazar et al. 2004).
The set of guidelines selected for ESALP was the first version of WCAG (Chisolm et al. 1999),
which is a widely used set of guidelines. The Web accessibility study (Katsanos et al. 2009) that
identified representative examples of guidelines violations for the content of the tool started on
September 2008. At this time, the second final version of the WCAG guidelines (Caldwell et al. 2008)
was not yet available. However, future work includes integrating bad design examples of this and other
sets of guidelines as well.
WCAG v1.0 includes 14 broadly-expressed guidelines, each of which has to meet a certain number
of checkpoints (Chisholm et al. 1999). Each checkpoint explains how the guideline applies in typical
content development scenarios. All 64 checkpoints are divided into three levels of priority. Since the
presented number of checkpoints is quite high, findability of related guideline examples is a mandatory
requirement in ESALP. Findability is defined as the degree to which a particular object is easy to
discover or locate and consequently the degree to which a system or environment supports navigation
and retrieval (Morville 2005). The concept of findability is of fundamental importance for the Web in
general and learning environments in particular (Tselios et al. 2008a, 2008b). From a psychological

perspective, the learner needs to consult meaningful and related links and scheme organizations with
high-quality residues, in order to proceed seamlessly, establish a flow state and not get overly frustrated
while involved in the task (Csikszentmihalyi 1990). As a result, a faceted browsing (English et al.
2002) approach was adopted, in an attempt to support learners’ personalized exploration of the learning
material according to their individual needs. The software framework developed by the Flamenco
Project
1
at the University of California at Berkeley was used to implement ESALP’s faceted browsing
interface.
The provided material is organized under five different facets: a) by WCAG number of guideline,
b) by WCAG number of checkpoint, c) by priority category of guideline, d) by elements of a webpage
and e) by type of disability. Examples of entries in each facet can be seen in Figure 1, which presents
an example of the interface of ESALP. The number of items in each facet is presented in parentheses
next to each link−option. The available options (Figure 1a) allow the learner to easily navigate the
information space by progressively narrowing the choices in each facet. Multiple filters can be applied
at the same time and each one can be easily removed with a single click. Alternatively, typical keyword
search can be used.
Figure 1 Interface of the proposed Educational Tool for Accessibility Learning through Paradigms
(ESALP): (a) Navigation mechanisms, (b) Short description of the guideline, (c) Example of its
violation, and (d) Concise advice on how to avoid or resolve the problem.
The right part of the interface of the tool presents the representative example for the selected
guideline. In specific, a short description of each guideline (Figure 1b) is provided along with
representative examples of the problematic situation that occurs when the guideline is violated in real-
world websites (Figure 1c) and concise advice on how the problem can be avoided or resolved (Figure
1d). These examples were derived from an extended accessibility evaluation study of 50 websites
(Katsanos et al. 2009). Although a single webpage can break many guidelines at the same time, each
example focuses on only one guideline violation to simplify the message and make the content easier to
understand (Kalyuga 2009). The examples contain pictures that contrast how the presented webpage is
viewed by people with and without disabilities, and include a brief textual description of the
accessibility problem (Figure 1c). Each example is accompanied by a title that communicates the
1
http://flamenco.berkeley.edu

domain (e.g. “hospital”) and the name of the actual website in order to underline the fact that these
examples refer to existing websites. In addition, concise, practical advice on how to ensure compliance
with the presented guideline and how to avoid or resolve the problem is also provided (Figure 1d). This
advice was derived from the WCAG v1.0 formal documents by summarizing the information contained
relying upon the expertise of the first author. Such a summarization was made possible by the
contextualization of the guideline in an exemplary situation. A link to additional resources about each
guideline is also provided to allow further exploration.
2.2 Scenario of the learning activity
The proposed project based learning activity starts with a short presentation that describes the objective
of the exercise, provides an overview of the learning material and software tools to be used and informs
students that they will need to produce an accessibility evaluation report of a real-world webpage at the
end of the exercise. Subsequently, learners are provided with the URL address of ESALP and they are
allowed sufficient time to explore and familiarize with its interface. Next, learners are asked to evaluate
the accessibility of one or more webpages, depending on the time allocated for the activity and the
complexity of the evaluated webpage (i.e. number of different page elements, number of violations of
each guideline etc). The evaluation process is mediated by ESALP and the Web Accessibility Toolbar
(WAT) v2.0
2
, which is a freely distributed toolbar that facilitates manual accessibility evaluation of
websites. At the end of their evaluation, learners are asked to deliver a structured accessibility
evaluation report in which they describe with sufficient detail the accessibility violations they found.
A variation of the activity, which has been proposed by a student involved in the case study
described in the following, involves creating teams of students that collaborate on the accessibility
evaluation project and at the end of the evaluation present their findings to the rest of the class for
further discussion. This variation provides opportunities for students to learn and practice interpersonal
skills as they work in cooperative teams, but requires additional time to be scheduled for the whole
activity.
3 Case study
3.1 Methodology
The within-subjects case study presented in this paper compared the learning effectiveness of
traditional academic instruction (pre-condition) with the proposed project based activity (post-
condition) in the context of a course on Human Computer Interaction. This course is offered in the 4th
year of studies at the Electrical and Computer Engineering Department of the University of Patras in
Greece. Twenty-seven students (18 male, 9 female) aged 21-30, with an average age of 22.7,
participated in the study. All students were attendants of the aforementioned course and provided
consent to participate in the study.
Initially, the students followed the traditional academic instruction on Web accessibility offered in
the aforementioned course. In specific, they attended a lecture on Web accessibility and completed a
homework assignment that asked from them to study the WCAG v1.0 standard, evaluate the
accessibility of their University’s homepage and submit a report.
Two days after the deadline for their assignment, students participated in the proposed project based
learning activity in the context of a lab exercise, which lasted 2 hours and 15 minutes. The lab exercise
started by asking students to complete a pre-test online questionnaire that evaluated their accessibility
knowledge after following the traditional academic instruction (see Appendix). The questionnaire was
divided into two parts. The first part included three questions, which asked participants to rate on a 1−5
scale their knowledge on Web accessibility (Web accessibility perceived knowledge) and on WCAG
v1.0 guidelines (WCAG v1.0 perceived knowledge), and their attitude towards the importance of Web
2
http://www.paciellogroup.com/resources/wat-ie-about.html

accessibility (Web accessibility perceived importance). The second part was an accessibility knowledge
test that included a total of 25 questions; 10 true-false statements, 10 multiple choice items and 5 open-
ended questions. The perfect score for this test was 50; 10 for true-false statements plus 20 for multiple
choice questions plus 20 for open-ended questions. Each student had 25 minutes to complete this
accessibility knowledge pre-test.
Next, the non-collaborative variation of the activity was followed as described in section 2.2. The
initial presentation of the activity lasted approximately 10 minutes, and students were allowed
approximately 10 minutes to freely explore and familiarize with the interface of ESALP. Afterwards,
students were asked to evaluate the accessibility of the homepage of a popular flight and hotel booking
website using ESALP and WAT v2.0. The accessibility of this homepage was evaluated by an expert
prior to the lab exercise in order to estimate the difficulty of the task and allow sufficient time to the
students. At the end of their evaluation, students delivered a structured report describing the
accessibility violations they had found.
Finally, each student had 15 minutes to complete a post-test online questionnaire, which was
identical to the pre-test questionnaire. There were only two differences: a) the questions and answer
choices of the Web accessibility knowledge test were presented in a different, randomized order, and b)
a new part with eight questions evaluating the usefulness of ESALP and its three most positive and
most negative characteristics was added (see Appendix, Part III).
3.2 Analysis and results
Table 1 summarizes descriptive statistics of the dependent variables used to operationalize and
compare the learning effectiveness of traditional academic instruction (pre-condition) with the
proposed project based activity (post-condition). In general, male students achieved better mean Web
accessibility knowledge pre-score (24.1 to 22.4) and post-score (31.5 to 30.7). However, female
students showed greater improvement (8.3 to 7.4), which is also reported in a relative question asking
their WCAG perceived knowledge improvement (female 0.9, male 0.6, in a scale 1 to 5).
Paired-samples t-tests using the Bonferroni adjustment criterion were conducted in order to
compare the collected dependent variables in the pre-test and post-test conditions. Given that four
comparisons were conducted, the outcome of a test was considered significant at the level of 0.0125. A
significant difference in the students’ pre-test (M=23.5, SD=6.3) and post-test (M=31.2, SD=6.8) Web
accessibility knowledge scores was found; t(26)=-6.99, p=0.000. In particular, the average
improvement of students’ Web accessibility knowledge score was 15.4% (from 23.5 to 31.2) and only
2 out of 27 students showed zero improvement (and 1 scored 2% percent worse). This result suggests
that students’ knowledge on Web accessibility improved significantly after the proposed project based
learning activity.
Table 1 Descriptive statistics of the dependent variables used in this study to operationalize and
compare the learning effectiveness of traditional academic instruction (pre-condition) with the
proposed project based activity (post-condition).
Dependent Variable
Females
(N=9)
Males
(N=18)
All
Mean
SD
Mean
SD
Mean
SD
Web accessibility knowledge
[score 0 to 50]
pre-test
22.4
5.0
24.1
6.9
23.5
6.3
post-test
30.7
5.3
31.5
7.5
31.2
6.8
Web Accessibility perceived knowledge
[scale 1 to 5]
pre-test
2.8
0.7
2.8
0.7
2.8
0.7
post-test
3.1
0.9
3.1
0.7
3.1
0.8
WCAG v1.0 perceived knowledge
[scale 1 to 5]
pre-test
2.3
0.7
2.3
0.8
2.3
0.8
post-test
3.2
0.7
2.9
0.6
3.0
0.6
Web Accessibility perceived importance
[scale 1 to 5]
pre-test
4.3
0.7
4.1
0.9
4.1
0.9
post-test
4.7
0.5
4.2
1.0
4.3
0.9

Furthermore, a significant difference in the students’ pre-test (M=2.3, SD=0.8) and post-test
(M=3.0, SD=0.6) WCAG v1.0 perceived knowledge was found (t(26)=-4.26, p=0.000). However, there
was no significant difference in the students’ pre-test (M=2.8, SD=0.7) and post-test (M=3.1, SD=0.8)
Web accessibility perceived knowledge; t(26)=-2.36, p=0.026. These results suggest that students
perceived an improvement of their knowledge on the WCAG v1.0 guidelines, but not on more general
Web accessibility issues. This is a meaningful finding, since the academic lecture students had attended
prior to the proposed project based activity addressed the general aspects of Web accessibility.
However, the lecture didn’t address in detail the WCAG guidelines, apart from two representative
examples. In addition, the evaluation of the students’ homework assignments revealed a shallow
understanding of the guidelines. This finding unveils the potential of blended, project based learning
approaches, since coupling of traditional instructional approaches with additional content covering
specific issues seems to present a broader but also a deeper understanding of designing for accessibility
(U.S. Department of Education 2009).
In addition, analysis of the collected data found no significant difference in the students’ pre-test
(M=4.2, SD=0.9) and post-test (M=4.3, SD=0.9) attitude towards Web accessibility perceived
importance; t(26)=-1.00, ns. This result suggests that students who attended the traditional academic
instruction perceived accessibility as an important issue for Web design even before being engaged in
the project based activity. The latter further strengthens our point that new and innovative approaches,
such as the one presented in this paper, are required to promote training on Web accessibility issues.
The students were also asked to evaluate their learning experience with ESALP. First, they
evaluated the suitability of ESALP as a tool to support learning of accessibility guidelines. As derived
by the results presented in Table 2, the students found ESALP useful as an educational tool (M=3.5,
SD=0.8) and they reported that they would recommend ESALP to colleagues as a means of self-
education in the context of accessible Web design (M=3.9, SD=0.8). In addition, students reported that
they would probably use ESALP to convince their employer or client to have Web accessibility as a
requirement of their project (M=3.6, SD=0.9). Table 2 also shows that students were neither satisfied
nor dissatisfied with their learning progress during the lab activity (M=2.9, SD=0.7). As Table 3 shows,
quite a few students reported that additional time for the whole activity would be beneficial. Therefore,
it is argued that students’ perceived learning effectiveness would be improved if more time is
scheduled for the lab activity; a hypothesis that will be evaluated in the next academic semester.
Table 2 Self-evaluation of students’ learning experience with ESALP.
Question
(1: strongly disagree; 5: strongly agree)
Females
(N=9)
Males (N=18)
All
Mean
SD
Mean
SD
Mean
SD
Q29: The ESALP helped me improve my
WCAG v1.0 knowledge.
3.1
0.6
3.4
0.8
3.4
0.8
Q30: I think that ESALP is useful as an
educational tool.
3.1
0.8
3.7
0.8
3.5
0.8
Q31: I would recommend ESALP to a friend
who wants to learn how to design accessible
websites.
4.0
0.5
3.8
0.9
3.9
0.8
Q32: I would use ESALP to convince an
employer/client to have Web accessibility as a
requirement.
3.4
1.1
3.8
0.8
3.6
0.9
Q33: During the activity, I am satisfied with my
learning progress and effectiveness.
3.0
0.5
2.9
0.8
2.9
0.7
Two questions required the students to identify the three most positive and negative characteristics
of ESALP respectively. The results are summarized in Table 3. The functionality offered to freely

navigate and explore the provided learning content as well as the usage of concrete examples to show
the importance and the problems posed by an accessibility guideline violation in a real-world context
were appraised. These findings provide support for the ESALP’s interaction design decisions and
content organization scheme. In addition, the inclusion of representative screenshots to further explain
accessibility guidelines violation issues was also positively mentioned, a finding that is confirmed by
numerous related studies (Clark and Mayer 2008; Mayer 2005, 2009). In contrast, some problem
descriptions and suggestions were found to be ambiguous and difficult to understand for quite a few
students (despite the fact that even more recorded the aforementioned issue as a positive characteristic).
In addition, students were asked to provide suggestions for further improvement of ESALP. The
most frequent suggestions were to provide more examples for each checkpoint (suggested by 6
students), to describe the checkpoints using simpler language (2) with less HTML usage if possible (1),
to provide better linkage across related checkpoints (1), to further standardize some checkpoint
descriptions in a form such as “rule description−problem description−ways to tackle the problem” (1),
to provide more comprehensive help (1) and to position the guideline title nearby each criterion in
order to further facilitate exploration since the numbering system requires information recall instead of
recognition.
Table 3 The most useful and annoying characteristics of ESALP as perceived by the students.
Positive characteristics
%
Negative characteristics
%
Use of concrete examples
63
Understandability of content/language
33
Content organization/taxonomy
56
Content organization/taxonomy
19
Understandability of content/language
44
Perceived usability
15
Images in Examples
44
More examples for some checkpoints
11
Perceived usability
19
Did not specify how to find problems
7
Real-world websites in examples
7
Lack of time to study the tool
7
4 Conclusions
In this paper, a project based learning activity designed to instruct Web accessibility guidelines and
good design practices was described. The activity is mediated by ESALP, a web-based educational tool
that presents real-world examples of accessibility guidelines violations accompanied by concise advice
on how to design for accessibility. In addition, a case study comparing the learning effectiveness of the
proposed activity with existing educational practices in a University course was presented.
The findings of the study supported the thesis that such a project based learning activity mediated
by appropriate tools could substantially aid development of good accessibility practices for Web
engineering students. In particular, 27 students following a traditional academic lecture-based
instruction on Web accessibility improved their post score on a knowledge assessment questionnaire
(see Appendix, Part II) by 15.4% (from 23.5/50 to 31.2/50) after being engaged in the presented project
based learning activity. Only 2 out of 27 students showed zero improvement (and 1 scored 2% percent
worse). These results illustrate that the presented project based learning activity improved students’
knowledge in a complementary way to the traditional academic lecture paradigm. This is an
encouraging finding, since despite the importance of accessibility and the availability of guidelines and
tools to support the design of accessible websites, study results indicate that a great proportion of
websites is characterized by low accessibility (Ceaparu and Shneiderman 2002; Hacket et al. 2003;
Sullivan and Matson 2000; Zaphiris and Ellis 2001; Zaphiris and Zacharia 2001). In addition, students’
perceptions were positive against adoption of such a tool-mediated educational approach, which are a
critical factor for successful technology integration in a learning process (Concannon et al. 2005).
The positive feedback obtained by the participants of the study (students of an Electrical and
Computer Engineering University Department) as well as by initial presentations of ESALP to
colleagues involved in Web development encouraged us to make the tool freely available online at

http://hci.ece.upatras.gr
3
. Our aims are to increase awareness, motivate, and educate stakeholders in
Web development on Web accessibility. Furthermore, the tool can be a valuable asset to educators
teaching, and students learning about Web accessibility. In agreement to Lazar et al. (2004), it is
argued that if more people that are involved in the development of a website become familiar and
embrace Web accessibility ideas, then it is more likely that accessible websites will be developed.
Future work involves further integration of the proposed educational approach in two University
courses that include modules on Web accessibility in order to investigate its contribution to the
educational process in the long term. In addition, further improvement of ESALP to better support the
presented project based activity is envisaged as a future direction. For instance, the tool could provide
to students the possibility to collaboratively implement their accessibility evaluation report in an
appropriate wiki space. Such an approach would be also beneficial in a distance learning course.
Furthermore, the examples of guidelines violations presented in the tool could be supplemented by
specific information on how to detect the presented problem, such as reference to the functionalities
provided by manual inspection tools (e.g. WAT v2.0). Learning, ideally, should be a function of
understanding and thinking actively about materials, their structure and relationship (Entwistle 1998),
and the aforementioned change in the design of ESALP would provide an explicit link between
informing and performing (Mayer 2007). Finally, the coverage and interrelations with sets of guidelines
other than WCAG 1.0, such as Section 508 and WCAG 2.0, is envisaged as a future direction too.
References
Barab, S., & Duffy, T. (2000). From practice fields to communities of practice. In D. Jonassen & S. M. Land
(Eds.), Theoretical foundations of learning environments (pp. 25–56). Mahwah, NJ: Lawrence Erlbaum
Associates.
Bloomenfeld, P., Soloway, E., Marx, R., Krajcik, J., Guzdial, M., & Palinczar, A. (1991). Motivating project-based
learning: Sustaining the doing, supporting the learning. Educational Psychologist, 26(3-4), 369–398.
Brown, J., & Duguid, P. (2000). The social life of information. Cambridge, MA: Harvard Business School Press.
Caldwell, B., Cooper, M., Reid, L.G., & Vanderheiden. G. (2008, December 11). Web content accessibility
guidelines 2.0. [Online]. Available: http://www.w3.org/TR/WCAG20
Ceaparu, I. & Shneiderman, B. (2002). Improving Web-based civic information access: a case study of the 50 US
States. In Proc. of the IEEE International Symposium on Technology and Society 2002, pp. 275–282.
Chisholm, W., Vanderheiden, G., & Jacobs. I. (1999, May 5). Web content accessibility guidelines 1.0. [Online].
Available: http://www.w3.org/TR/WCAG10
Clark, J. (2003). Building Accessible Websites. New Riders Press.
Clark, R. & Mayer, R. E. (2008). E-learning and the science of instruction (2nd ed). San Francisco: Jossey-Bass.
Concannon, F., Flynn, A., & Cambell, M. (2005). What campus-based students think about the quality and benefits
of e-learning, British Journal of Educational Technology, 36(3), 501–512.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY: Harper & Row
Publishers, Inc.
Duffy, T.M., & Kirkley, J.R. (2004). Learner-Centered Theory and Practice in Distance Education: Cases from
Higher Education, Lawrence Erlbaum Associates, Mahwah, NJ.
English, J., Hearst, M., Sinha, R., Swearingen, K., & Yee, K. (2002). Hierarchical faceted metadata in site search
interfaces. In Proc. of CHI '02 extended abstracts on Human factors in computing systems, Minneapolis,
Minnesota, USA, 2002, pp. 628–639.
Entwistle, N.J. (1998) Improving Teaching Through Research On Student Learning. In J.J.F.Forest (ed.)
University teaching: International Perspectives (pp. 73–112). New York: Garland.
Felix, U. (2005) E-learning Pedagogy In The Third Millennium: The Need For Combining Social And Cognitive
Constructivist Approaches. ReCall. 17(1), 85–100.
Hackett, S., Parmanto, B. & Zeng, X. (2003). Accessibility of Internet websites through time. In Proc. of 6th
International ACM/SIGCAPH Conf., Pittsburgh, pp. 32–39.
Harper, S., Yesilada, Y. (2008), WebAccessibility. Springer-Verlag London.
3
Currently, the content of ESALP is in Greek.

Henry. S. L. (2005, September). Introduction to Web accessibility [Online]. Available:
http://www.w3.org/WAI/intro/accessibility.php
Ivory, M.Y., Sinha, R.R. & Hearst, M.A., 2001. Empirically validated web page design metrics. In Proc. of CHI
'01. Seattle, Washington, United States: ACM, pp. 53–60.
Kalyuga, S. (2009). Instructional designs for the development of transferable knowledge and skills: A cognitive
load perspective, Computers in Human Behavior, 25(2), 332–338.
Katsanos, C., Tsakoumis A., & Avouris N. (2009). Web accessibility: Design of an educational system to support
guidelines learning. In Proc. of the 13th Pan-Hellenic Conference on Informatics (PCI). Corfu, Greece, 10–
12 September, 2009, pp. 155–164.
Koyani, S.J., Bailey, R.W. & Nall, J.R. (2004). Research-based web design & usability guidelines. Computer
Psychology.
Lazar, J., Dudley-Sponaugle, A. & Greenidge, K. (2004). Improving web accessibility: a study of webmaster
perceptions, Computers in Human Behavior, 20(2), 269–288.
Lee, C.I., & Tsai, F.Y. (2004). Internet project-based learning environment: the effects of thinking styles on
learning transfer, Journal of Computer Assisted Learning, 20(1), 31–39.
Mayer, R. E. (Ed). (2005). Cambridge Handbook of Multimedia Learning. New York: Cambridge University
Press.
Mayer, R. E. (2009). Multimedia learning (2nd ed). New York: Cambridge University Press.
Morville, P., 2005. Ambient findability: What we find changes who we become (1st ed.), O'Reilly Media.
Nielsen, J. (1996, October). Accessible design for users with disabilities [Online]. Available at:
http://www.useit.com/alertbox/ 9610.html
Petrie, H. & Kheir, O. (2007). The relationship between accessibility and usability of websites. In Proc of CHI ’07,
pp. 397–406.
Psaromiligkos, Y. & Retalis, S. (2003). Re-Evaluating the Effectiveness of a Web-based Learning System: A
Comparative Case Study. Journal of Educational Multimedia and Hypermedia, 12(1), 5–20.
Slatin J.M., & Rush, S. (2002). Maximum Accessibility: Making Your Web Site More Usable for Everyone.
Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA.
Sullivan, T., & Matson, R. (2000). Barriers to use: usability and content accessibility on the Web's most popular
sites. In Proc. of the 2000 conference on Universal Usability, Arlington, Virginia, USA, pp. 139–144.
Sweller, J., & Cooper, G. (1985). The use of worked examples as a substitute for problem solving in learning
algebra. Cognition and Instruction, 2(1), 59–89.
Tselios, N., Avouris, N., & Komis, V. (2008a). Towards a unified usability evaluation approach for learning
software environments: Issues and challenges. Education and Information Technologies, 13(1), 55–76.
Tselios, N., Katsanos, C., Kahrimanis, G. & Avouris, N. (2008b). The notion of Information Foraging as a design
and evaluation tool for Distance Learning Systems. In Pahl, C., Architecture Solutions for E-Learning
Systems (pp. 320–339). Hershey, PA, USA: Information Science Reference.
U.S. Department of Education, Office of Planning, Evaluation, and Policy Development, (2009). Evaluation of
Evidence-Based Practices in Online Learning A Meta-Analysis and Review of Online Learning Studies
[Online]. Available at: http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf
van Gog, T., Paas, F., & Van Merriënboer, J.J.G. (2006). Effects of process-oriented worked examples on
troubleshooting transfer performance. Learning and Instruction, 16(2), 154–164.
van Gog, T., Paas, F., & Van Merriënboer, J.J.G. (2008). Effects of studying sequences of process-oriented and
product-oriented worked examples on troubleshooting transfer efficiency. Learning and Instruction, 18(3),
211–222.
van Gog, T., Kester, L., Nievelstein, F., Giesbers, B., & Paas, F. (2009). Uncovering cognitive processes:
Different techniques that can contribute to cognitive load research and instruction. Computers in Human
Behavior, 25(2), 325–331.
Wouters, P., Paas, F., & Van Merriënboer, J.J.G. (2008). How to optimize learning from animated models? A
review of guidelines based on cognitive load. Review of Educational Research, 78(3), 645–675.
Zaphiris, P., & Ellis, R.D. (2001). Website usability and content accessibility of the top USA Universities. In
Proc. of WebNet 2001, Orlando, FL, USA.
Zaphiris, P., & Zacharia, G. (2001). Website content accessibility of 30,000 Cypriot Web sites. In Proc. of the 8th
Panhellenic Conference on Informatics. Nicosia, Cyprus, pp. 128–136.

Appendix : Online Questionnaire for Accessibility Knowledge Assessment
PART I: Perceived knowledge and attitude towards Web accessibility (both pre- and post-test)
Q1: How would you rate your level of knowledge on the subject of Web accessibility?
VERY LOW 1 2 3 4 5 VERY HIGH
Q2: How important do you think it is to ensure the accessibility of websites?
NOT AT ALL 1 2 3 4 5 EXTREMELLY
Q3: How would you rate your level of familiarity with the WCAG v1.0 standard?
VERY LOW 1 2 3 4 5 VERY HIGH
PART II: Accessibility knowledge test (both pre- and post-test)
Q4: A screen reader can render the text of a webpage in an incomprehensible way if the changes in
natural language are not specified in the source code?
a) True
b) False
Q5: Embedding text in an image can create accessibility problems to people with low vision.
a) True
b) False
Q6: An accessible webpage allows interaction with all of its elements though keyboard without
requiring usage of mouse.
a) True
b) False
Q7: A webpage with lots of images cannot be accessible to people who are blind.
a) True
b) False
Q8: Auto-refresh of a webpage can result in accessibility impasses under certain circumstances.
a) True
b) False
Q9: It is necessary to provide an alternative text-only version for each webpage of a website in
order to make it accessible to people with disabilities.
a) True
b) False
Q10: Webpages using frame elements are inaccessible to screen readers.
a) True
b) False
Q11: Server-side image maps are more accessible than client-side image maps.
a) True
b) False
Q12: Flashing content can trigger a seizure to people with photosensitive epilepsy.
a) True
b) False
Q13: Ensuring conformance to accessibility guidelines can result in an increase of its usability for
people without disabilities.
a) True
b) False

Q14: People who are blind can get an idea of a webpage’s content by relying on:
a) Text equivalents for every non-text element.
b) Headings.
c) Clear and simple language of the content.
d) Tables that transform gracefully.
Q15: The following sentences concern usage of tables in webpages. Specify the correct one.
a) For data tables, an alternative version that presents its data in an accessible way should be
provided.
b) For data tables, row and column headers should be identified.
c) Tables should not be used in webpages, since they create accessibility problems that
cannot be resolved.
d) Both (a) and (b) are correct.
Q16: What one should do in order to make a client-side image map accessible?
a) Provide redundant text links for each active region of the client-side image map.
b) Embed a textual description of each active region of the client-side image map using image
processing software (e.g. Photoshop).
c) Replace the client-side image map with a server-side one.
d) Client-side image maps are anyways accessible and thus no specific action is required.
Q17: When does a hyperlink with a label “Click here” meet the accessibility requirements of WCAG
v1.0?
a) When it is accompanied by an appropriate auditory description.
b) When it provides an extended description of its label via a tooltip.
c) When the sentence or paragraph it relates to is visually close and thus the link can be easily
associated with its context.
d) None of the above.
Q18: How can a complex graph in a webpage be accessible to people who are blind?
a) Such a graph cannot be accessible to people who are blind.
b) The graph should have a text equivalent.
c) The graph should be accompanied by a table presenting the same data.
d) Both (b) and (c) are necessary.
Q19: Specify in which of the following cases an accessibility problem would arise for a person who
uses a special keyboard due to motor disabilities in order to browse an online newspaper.
a) There is a “+” symbol next to each article, which is triggered with an “onMouseClick” and
presents a summary of its content.
b) The webpages of the online newspaper use Javascript.
c) The TAB button does not select the visually next link to an article.
d) Both in (a) and (c).
Q20: In order to ensure accessibility of a video in a webpage one should:
a) Provide synchronized subtitles with the video.
b) Provide an auditory description of the important information of the video.
c) Not use video in webpages.
d) Both (a) and (b).
Q21: Why is it good practice to include non-link, printable characters between adjacent links?
a) Because this is the only way to ensure a logical tab order between adjacent links.
b) Because this is the only way to ensure that a screen reader would be able to read aloud
each link separately.
c) Because this is the only way to ensure that the webpage containing the links will remain
accessible when style sheets are disabled.
d) Because this is the only way to ensure that the links will be comprehensible when the font
size is changed.

Q22: Usage of blinking content can be problematic for:
a) People who are blind and rely on screen readers to interact with a website.
b) People with cognitive disabilities, such as dyslexia.
c) People without any type of disability.
d) All the above people.
Q23: Why is it important to ensure that a webpage validates to published formal grammars, such as
HTML 4.0?
a) Because the webpage can then present the associated validation logo (e.g. “W3C HTML
4.0 verified”).
b) Because the webpage can then transform gracefully when style sheets are disabled.
c) Because the webpage is more likely to be accessible though assistive technologies.
d) Because the webpage will be surely accessible if it validates to published formal
grammars.
Q24: Briefly describe the accessibility problem that arises for people who are blind when titles are
not used for the frames of a webpage.
Q25: Which 3 requirements must be met in order to ensure that the image maps of a website are fully
accessible to all people?
Q26: Which, in your opinion, is the most important guideline to achieve accessibility of Web forms?
Q27: What steps should be taken to make an image in a webpage accessible to people who are blind?
Briefly explain the problem that arises and how it can be avoided.
Q28: You are developing an e-shop for a client. Your client has specifically asked for a homepage
that presents in red color the price of the goods with discount. Briefly explain to him the
accessibility problem that would arise by such a choice and propose an alternative design
solution.
PART III: Evaluation of ESALP usefulness (only post-test)
Q29: The ESALP helped me improve my WCAG v1.0 knowledge.
STRONGLY 1 2 3 4 5 STRONGLY
DISAGREE AGREE
Q30: I think that ESALP is useful as an educational tool.
STRONGLY 1 2 3 4 5 STRONGLY
DISAGREE AGREE
Q31: I would recommend ESALP to a friend who wants to learn how to design accessible
websites.
STRONGLY 1 2 3 4 5 STRONGLY
DISAGREE AGREE

Q32: I would use ESALP to convince an employer/client to have Web accessibility as a
requirement.
STRONGLY 1 2 3 4 5 STRONGLY
DISAGREE AGREE
Q33: During the activity, I am satisfied with my learning progress and effectiveness.
websites.
STRONGLY 1 2 3 4 5 STRONGLY
DISAGREE AGREE
Q34: Which, in your opinion, are the three most positive characteristics of ESALP?
1st:
2nd:
3rd:
Q35: Which, in your opinion, are the three most negative characteristics of ESALP?
1st:
2nd:
3rd:
Q36: What would be your proposals to improve ESALP? Any other comments?
Note1: The questionnaire was presented to students in Greek which is their mother language. In this
appendix, it was translated in English for presentation purposes.
Note2: In this appendix, the correct answers are presented in italics.