Conference PaperPDF Available

A Method for Accessibility Testing of Web Applications in Agile Environments

Authors:

Abstract

Currently, more than one billion people live with some form of disability. A person's environment has a huge impact on the extent and impact of their disability, e.g. inaccessible environments create disability by creating barriers, while accessible environments diminish disability and enable full participation and inclusion. Web applications are often the only means available for people to access certain services or to certain information, e.g. healthcare information, public services, banking, education, and entertainment. Web accessibility is the property of a web application to support the same level of effectiveness for users with disabilities as it does for users without disabilities. In this study, we propose the use of automated tools, simulators, expert-based testing, and user-based testing in the context of a comprehensive method for accessibility testing of web applications in agile environments. The proposed method consist of five stages, as defined by the International Software Testing Qualifications Board: test planning and control; test analysis and design; test implementation and execution; evaluating exit criteria and reporting; and test closure activities. For each of these stages, the method details specific tasks to perform accessibility testing of web applications in the context of agile developments.
A Method for Accessibility Testing of Web Applications in
Agile Environments
Sandra Sanchez-Gordon1, Sergio Luján-Mora2
1 Department of Informatics and Computer Science, Escuela Politécnica Nacional
Quito, Ecuador
sandra.sanchez@epn.edu.ec
2 Department of Software and Computing Systems, University of Alicante
Alicante, Spain
sergio.lujan@ua.es
Abstract. Currently, more than one billion people live with some form of disability. A
person’s environment has a huge impact on the extent and impact of their disability, e.g.
inaccessible environments create disability by creating barriers, while accessible
environments diminish disability and enable full participation and inclusion. Web
applications are often the only means available for people to access certain services or
to certain information, e.g. healthcare information, public services, banking, education,
and entertainment. Web accessibility is the property of a web application to support the
same level of effectiveness for users with disabilities as it does for users without
disabilities. In this study, we propose the use of automated tools, simulators, expert-
based testing, and user-based testing in the context of a comprehensive method for
accessibility testing of web applications in agile environments. The proposed method
consist of five stages, as defined by the International Software Testing Qualifications
Board: test planning and control; test analysis and design; test implementation and
execution; evaluating exit criteria and reporting; and test closure activities. For each of
these stages, the method details specific tasks to perform accessibility testing of web
applications in the context of agile developments.
Keywords: Disability, Web Accessibility, WCAG, ISO/IEC 25010, Software Testing
Process, ISTQB, Agile.
1 Introduction
Web applications are manifestly not accessible. This is a big problem today because web applications
are often the only means available for people to access several services or to certain information, e.g.
healthcare information, public services, banking, education, and entertainment. This study explains
the concepts of disability, accessibility, web accessibility, and software testing. It also presents the
principles of the Web Content Accessibility Guidelines (WCAG 2.0) and the software product quality
characteristics of the standard ISO-25010 Systems and software engineering Systems and
software quality requirements and evaluation System and software quality models. Then, it includes
a review of relevant published research in the field of accessibility testing of web applications. Finally,
it proposes a method to perform accessibility testing of web applications in agile environments. The
method consists of five stages, as proposed by ISTQB: test planning and control, test analysis and
design, test implementation and execution, evaluating exit criteria and reporting, and test closure
activities.
The rest of this paper is organized as follows. Section 2 presents the theoretical foundation.
Section 3 describes accessibility testing tools and simulators. Section 4 describes the proposed
method. Section 5 presents conclusions and future work.
A Method for Accessibility Testing of Web Applications in Agile Environments
2 Theoretical Foundation
2.1 Disability and Accessibility
According to the World Health Organization, disability is part of the human condition (25). Disability
results from the interaction between persons with certain conditions and environmental barriers that
hinder their participation in society on an equal basis with others. Hence, a disability is not an attribute
of the person but depends on the barriers that persons with disabilities encounter in their day to day
lives.
Currently, more than one billion people live with some form of disability. This is about 15% of the
world's total population, i.e. the world´s largest minority group (25). Furthermore, the number of
persons with disabilities increases appreciably when taking into account not only permanent
disabilities but also people with temporary disabilities due to illnesses or accidents.
We can analyze accessibility issues from two perspectives. First, personal disabilities which are
those associated to body or mental impairments of the human being that can be of birth or acquired at
any point in a person's life, e.g. vision, hearing, speech, motor, cognitive, and psychosocial. Second,
non-personal disabilities which are those associated to situations in the environment surrounding the
human being that can occur at any point in a person's life and are usually temporary, e.g. cognitive
issues due to language, religion, or cultural barriers, environmental conditions, internet availability,
and technology availability. A person’s environment has a huge impact on the extent of their disability.
Inaccessible environments create disability by creating barriers. On the contrary, accessible
environments diminish disability and enable full participation and inclusion. The International
Organization for Standardization (ISO) defines accessibility as “the usability of a product, service,
environment or facility by people with the widest range of capabilities (9).
2.2 Web Accessibility
Web accessibility is the property of a web application or website to support the same level of
effectiveness for people with disabilities as it does for people without disabilities (16). Besides, a web
application or website that is accessible for users with different needs, skills, and situations also
benefits people without disabilities (19).
The Word Wide Consortium (W3C) created the Web Accessibility Initiative (WAI) with the aim of
studying the problems of accessibility in the web, develop guidelines and provide resources. The WAI
is recognized as an international authority on web accessibility. In 1999, WAI published the first
version of the web content accessibility guidelines WCAG 1.0. In 2008, WAI published the current
version, WCAG 2.0. The WCAG defines how to make web content accessible to disabled persons.
WCAG establishes four principles that give the foundation for web content accessibility: perceivable,
operable, understandable, and robust. Perceivable means that the information and components of the
user interface should be presented to users so they can perceive them. Operable means that the
components of the user interface and navigation must be operable. Understandable means that the
information and manipulation of the user interface must be understandable by the users. Robust
means that content must be robust enough to be reliably interpreted by a wide variety of user agents,
including browsers and assistive technology. WCAG define three levels of conformance: A (low), AA
(medium) and AAA (high) (20).
2.3 Software Testing Process
Software testing is a process applied in a software development project with the goal of assuring the
quality of the software product. The standard ISO/IEC 25010 Systems and software engineering --
Systems and software quality requirements and evaluation -- System and software quality models
defines eight quality characteristics for software products: functional suitability, performance
efficiency, compatibility, usability, reliability, security, maintainability, and portability. According to ISO,
accessibility is a sub-characteristic of usability (8).
The International Software Testing Qualifications Board (ISTQB) defines the fundamental software
testing process with five generic stages: planning and control; analysis and design; implementation
A Method for Accessibility Testing of Web Applications in Agile Environments
and execution; evaluating exit criteria and reporting; and test closure activities (5). Each stage
involves a set of tasks and sometimes, two or more tasks need to be done in parallel, e.g. time
pressure may mean that test execution starts before all tests have been designed.
2.4 Review of Published Research
To the best of our knowledge, there are few relevant published research on the field of accessibility
testing of software products: (1), (7), (3), (17), (2), (26), (12), (11), (4), (10), (14), (15).
Brajnik (2) presents a taxonomy of accessibility evaluation methods, review existing methods such
as WCAG, and proposes the use of automated tests, screening techniques, subjective assessments,
barrier walkthroughs and user testing. Herramhof et al. (7) presents two tools to support test case
management for accessibility test suites. The first one creates test suites for WCAG 2.0. The second
one allows the edition of test description files using XML. Goncalves de Branco et al. (4) presents a
CASE tool that allows traceability of accessibility requirements from conception to coding, giving
developers useful information for the construction of accessible software products. Sanchez-Gordon
and Moreno (14) presents a review of proposals to incorporate accessibility requirements and
evaluation tools, including the Accessibility Development Lifecycle proposed by Microsoft. Sanchez-
Gordon et al. (15) presents a proposal for developing accessible software based on ISO/IEC 29110.
3 Tools for Accessibility Testing
While accessibility testing is not fully automatable, tools can significantly assist software testers and
contribute to more effective testing and debugging of web applications in agile environments. There
are two types of tools for accessibility testing: validators and simulators.
3.1 Accessibility Testing with Validators
Automated accessibility validators are software applications, browser plug-ins, or online services that
help determine if a web application or website meets accessibility requirements, such as WCAG 2.0.
These tools are a useful resource to identify accessibility issues. They are best exploited when used
by testers familiar with web accessibility. Table 1 presents a selection of some of the most popular
evaluation tools among the listed by (23).
Name
Description
Accessibility
Developer
Tools
Adds to Chrome Developer Tools, an Accessibility audit with 17 rules and an
Accessibility sidebar pane in the Elements tab that provides extra debugging
information. https://chrome.google.com/webstore/detail/accessibility-developer-
t/fpkknkljclfencbdbgkenhalefipecmb?hl=en
AChecker
Interactive, international, customizable, web content accessibility checker. Allows
testers to create their own guidelines, and author their own accessibility checks.
http://achecker.ca
Photosensiti
-ve Epilepsy
Analysis
Tool
Identify seizure risks in web content and software. The evaluation is based on an
analysis engine developed specifically for web and computer applications.
http://trace.wisc.edu/PEAT
Readability
Grader
Check whether a web content is easy-to-read. It generates 7 different scores.
https://jellymetrics.com/readability-grader/
Tenon
It is an API which can be seamlessly integrated into an existing toolset. It identifies
WCAG 2.0 issues. http://www.tenon.io
Total
Validator
Includes a (X) HTML validator, an accessibility validator (WCAG), a CSS validator,
a spell checker, and a broken links checker. http://www.totalvalidator.com
WAVE
Provides a visual representation of accessibility issues within a web page.
http://wave.webaim.org/
A Method for Accessibility Testing of Web Applications in Agile Environments
Table 1. Selected accessibility evaluation tools.
3.2 Accessibility Testing with Simulators
Simulators tools are software applications, browser plug-ins, or online services that simulates how
users with different kinds of visual disabilities will perceive the software. Table 2 presents of some of
the most popular simulation tools among the listed by (23).
Name
Description
Accessibility
Color Wheel
Simulates three kinds of color blindness and it shows the result of W3C
algorithms that reveal if a color pair (text/background) to use in a web page is
accessible.
http://gmazzocato.altervista.org/colorwheel/wheel.php
Color Oracle
Color blindness simulator that shows in real time what people with common color
vision impairments will see.
http://colororacle.org
NoCoffee
Vision simulator for Chrome that is helpful for understanding the problems faced
by people with slight to extreme vision problems.
https://chrome.google.com/webstore/detail/nocoffee/jjeeggmbnhckmgdhmgdckei
gabjfbddl
Table 2. Selected simulation tools.
3.3. Accessibility testing with experts and users with disabilities
Once the testers have used the simulators and automated testing tools, it is important that
accessibility experts perform heuristic testing using standards and personas (6). Also, user-based
testing is necessary. It should involve potential users with disabilities, including users with aging-
related disabilities and foreign language users. These users may help to identify additional
accessibility barriers that are not easily discovered by automated tools, simulators, and expert
evaluation alone (21).
4 Proposed Method
In this section, we present the proposed method for accessibility testing in agile environments. The
method consists of five stages, as proposed by ISTQB: test planning and control, test analysis and
design, test implementation and execution, evaluating exit criteria and reporting, and test closure
activities. For each of these stages, the proposed method includes different types of activities and
details engineering tasks including specialized tasks to perform accessibility testing of web
applications in the context of agile developments. Figure 1 shows the relationships among the five
stages and the main accessibility-related tasks.
Figure 1. Method for accessibility testing in agile environments.
A Method for Accessibility Testing of Web Applications in Agile Environments
The initial input is the list of requirements for the web application, including the accessibility
requirements, that are used in the planning part of the first stage to define a testable DoD (Definition
of Done) for each item in the product backlog. The control part of the first stage will be using the
testable DoD to control, monitor and evaluate the execution of the tasks of the following stages.
In the test analysis and design stage, the methods and tools for accessibility testing are selected.
This second stage provides feedback for the planning and control tasks. The third stage is the test
implementation and execution. In this phase the testing environment is configured and the test cases
are executed following the test procedures. This stage provides feedback for the previous stage of
test analysis and design. The four stage is the evaluation of the exit criteria and reporting. In this
stage, findings and non-conformances are listed in the testing report. This stage provides feedback
for the two previous stages: test analysis and design and test implementation and execution. Finally,
the five stage is the test closure activities, that includes the storage of the testing assets for future
reference and the identification of lessons learned.
4.1 Test Planning and Control
This stage prepares the agile team for the rest of stages and it selects the tools to accomplish the
testing process. Table 3 shows the tasks included.
ID
TASKS
TPC1
Determine the scope of the tests, risks, objectives and strategies.
TPC2
Determine the resources of the necessary tests.
TPC3
Implement testing strategies.
TPC4
Create a schedule for the analysis and design of the tests.
TPC5
Create a schedule for the implementation and execution of the tests.
TPC6
Determine the exit criteria of the tests.
TPC7
Measure and analyze the results.
TPC8
Monitor and document the progress, coverage and exit criteria of the tests.
TPC9
Initiate corrective actions.
TPC10
Take decisions.
TPC11
Sensitize the agile team through the observation of users with disabilities
interacting with software products.
TPC12
Define a testable DoD for each accessibility requirement in the product
backlog.
Table 3. Test planning and control tasks.
4.2 Test Analysis and Design
In this stage the test environment is designed and the tools are selected. Table 4 shows the tasks
included.
ID
TASKS
ACTIVITY
TAD1
Review the evidence base.
Analysis
TAD2
Identify and prioritize test conditions, test requirements or test objectives,
and test data required.
TAD3
Evaluate the testability of the requirements and the system.
TAD4
Design specific combinations of test data, actions and expected results to
cover major quality risks.
Design
TAD5
Identify the test data required for the conditions and test cases
TAD6
Design the test environment.
TAD7
Identify some infrastructure and some necessary tools.
TAD8
Select accessibility evaluation tools, e.g. WAVE (24).
Accessibility
TAD9
Select HTML and CSS checkers, e.g. W3C HTML Validator (22).
TAD11
Select simulators for different types of visual disabilities, e.g. NoCoffee.
TAD11
Select assistive technologies, e.g. NVDA screen reader (13).
TAD12
Select simulations aids for testing purposes, e.g. blindfolds, ear defenders.
Table 4. Test analysis and design tasks.
A Method for Accessibility Testing of Web Applications in Agile Environments
4.3 Test Implementation and Execution
In this stage the test cases and procedures are developed and run. Table 5 shows the tasks included.
ID
TASKS
ACTIVITY
TIE1
Develop, implement and prioritize test cases, create test data and write
test procedures.
Implementa-
tion
TIE2
Prepare test harnesses and write automated test scripts.
TIE3
Organize test sets and sequences of test procedures for the efficient
execution of tests, taking into account the various constraints that could
determine the order in which tests are to be performed.
TIE4
Verify that the test environment has been successfully installed.
TIE5
Run both manual and automated test cases.
Execution
TIE6
Record test results, including versions of the software being tested, test
tools and testware.
TIE7
Compare the actual and expected results, which may require the
identification of anomalies where the actual and expected results do not
match.
TIE8
The investigation of anomalies can result in the creation of reports and the
analysis of incidents.
TIE9
Repeat the corrected or updated tests where necessary.
TIE10
Run regression tests, when the new test versions arrive.
TIE11
Review the design architecture, software components and interfaces for
traceability with accessibility requirements.
Accessibility
TIE12
Use accessibility checklists, e.g. WebAIM´s WCAG 2.0 Checklist (24).
Table 5. Test implementation and execution tasks.
4.4 Evaluating Exit Criteria and Reporting
The evaluation of the exit criteria and the creation of reports of the test results are strongly overlapped
with the execution of the tests. Table 6 shows the tasks included.
ID
TASKS
ACTIVITY
ECR1
Check the test records against the exit criteria of the tests specified during
test planning.
Evaluation of
exit criteria
ECR2
Evaluate whether further testing is required or whether the specified exit
criteria should be modified.
ECR3
Write a test summary report for business stakeholders.
Reporting
ECR4
Evaluate whether further accessibility testing is required.
Accessibility
Table 6. Evaluating test criteria and reporting.
4.5 Test Closure Activities
As the execution of the tests reaches a close, the exit criteria have been fulfilled and the final reports
of the results of the tests are compiled, the activities of the closure begin to occur. Table 7 shows the
tasks included.
5 Conclusions
Automated tools and simulators do not necessarily produce reliable results since not all the
accessibility problems can be automatically detected. Besides, a tool can produce fail positives and
fail negatives, up to 33% and 35% respectively according to Brajnik (2). These fail positives and fail
negatives need to be discarded by using a combination of tools (18) and including expert-based
A Method for Accessibility Testing of Web Applications in Agile Environments
evaluation and user-based testing. Hence, automated tools provide good support to software testers
but they should be used in the context of a comprehensive method.
The method presented in this study is based on the five stages defined by ISTQB and adds specific
activities related to accessibility testing of web applications in agile contexts.
Debuggers have to solve the accessibility issues found during accessibility testing by making
changes on the web application to improve accessibility based on the evaluations results.
In the future, the definition of further requirements for each type of disability is necessary, as well
as to propose mechanisms to overcome the challenges associated with the implementation, testing,
and debugging of these requirements.
ID
TASKS
ACTIVITY
TCA1
Confirm test deliverables, final resolution or postponement of defect reports
and acceptance of the system by receiving parties.
Closure
TCA2
Finalize and archive testware, test environment, and test infrastructure for
later use during maintenance.
TCA3
Deliver the testware and the possibility of additional items to the
maintenance organization
TCA4
Conduct a retrospective study to take into account improvements for future
versions, projects and testing processes.
TCA5
Store accessibility testing assets in the software configuration repository.
Accessibility
Table 7. Test closure activities.
References
(1) Brajnik, G. (2006). Web accessibility testing: when the method is the culprit. In Proceedings of the
International Conference on Computers for Helping People with Special Need (ICCHP), pp. 156-
163.
(2) Brajnik, G. (2008). Beyond conformance: the role of accessibility evaluation methods. In
Proceedings of the Web Information Systems Engineering Conference (WISE), pp. 6380.
(3) Freire A. P., Goularte R., and Mattos-Fortes R. P. (2007). Techniques for developing more
accessible web applications. In Proceedings of the 25th ACM International Conference on Design
of Communication (SIGDOC), pp. 162-167.
(4) Goncalves de Branco, R., Cagnin, M. I., Barroso Paiva, D. M. (2014). AccTrace: Accessibility in
phases of requirements engineering, design, and coding software. In Proceedings of the 14th
International Conference on Computational Science and Its Applications (ICCSA), pp. 225-228.
(5) Hambling, B. (2015). Software Testing: An ISTQB-BCS Certified Tester Foundation Guide 3rd
Edition. Swindon: BCS.
(6) Henka, A. and Zimmermann, G. (2014). Persona based accessibility testing. In Proceedings of
the International Conference on Human-Computer Interaction, pp. 226-231.
(7) Herramhof, S., Petrie, H., Strobbe, C., Vlachogiannis, E., Weimann, K., Weber, G., & Velasco, C.
A. (2006). Test case management tools for accessibility testing. In Proceedings of the 10th
International Conference on Computers for Helping People with Special Needs (ICCHP), pp. 215-
222.
(8) ISO. (2011). ISO/IEC 25010 Systems and software engineering -- Systems and software Quality
Requirements and Evaluation (SQuaRE) -- System and software quality models. Retrieved from
http://www.iso.org/iso/catalogue_detail.htm?csnumber=35733
(9) ISO. (2012). ISO 9241-171 Ergonomics of human-system interaction Guidance on software
accessibility. Retrieved from
https://www.iso.org/obp/ui/#iso:std:iso:9241:-171:ed-1:v1:en
(10) Keates, S., Looms, P. O. (2014). The role of simulation in designing for universal access. In
Proceedings of 8th International Conference of Universal Access in Human-Computer Interaction
(UAHCI), pp. 54-63.
(11) Luján-Mora, S. and Masri, F. (2012). Integration of Web Accessibility into Agile Methods. In
Proceedings of the 14th International Conference on Enterprise Information Systems (ICEIS), 3,
pp. 123-127.
A Method for Accessibility Testing of Web Applications in Agile Environments
(12) Masri, F. and Luján-Mora, S. (2011). A Combined Agile Methodology for the Evaluation of Web
Accessibility. In Proceedings of the IADIS International Conference Interfaces and Human
Computer Interaction 2011 (IHCI), pp. 423-428.
(13) Nvaccess. (2015). NVDA Screen Reader. Retrieved from
http://www.nvaccess.org/
(14) Sánchez-Gordón, M-L., Moreno, L. (2014). Toward an integration of web accessibility into testing
processes. In Procedia Computer Science, 27, pp. 281-291.
(15) Sanchez-Gordon, S., Sánchez-Gordón, M-L., Luján-Mora, S. (2016).Towards an engineering
process for developing accessible software in very small entities. In Proceedings of the 11th
International Conference on Evaluation of Novel Software Approaches to Software Engineering
(ENASE), p.241-246.
(16) Slatin, J., Rush, S. (2003). Maximum accessibility: making your web site more usable for
everyone. Addison-Wesley Professional.
(17) Torkey. F., Keshk, A., Hamza, T., and Ibrahim. A. (2007). A new methodology for web testing. In
¨Proceedings of the 5th International Conference on Information and Communications
Technology, pp. 7783.
(18) Vigo, M., Brown, J. and Conway, V. (2013). Benchmarking Web Accessibility Evaluation Tools:
Measuring the Harm of Sole Reliance on Automated Tests. In Proceedings of the 10th
International Cross-Disciplinary Conference on Web Accessibility, pp. 1-10.
(19) W3C. (2005). Introduction to web accessibility. Retrieved from
http://www.w3.org/WAI/intro/accessibility.php
(20) W3C. (2008). Web content accessibility guidelines WCAG 2.0. Retrieved from
http://www.w3.org/TR/WCAG20/
(21) W3C. (2010). Involving users in evaluating web accessibility. Retrieved from
https://www.w3.org/WAI/eval/users.html
(22) W3C. (2013). Markup Validation Service. Retrieved from
http://validator.w3.org.
(23) W3C. (2016). Web accessibility evaluation tools list. Retrieved from
https://www.w3.org/WAI/ER/tools/
(24) WebAIM. (2015). WAVE Chrome Extension. Retrieved from
http://wave.webaim.org/extension/
(25) WHO. (2011). World report of disability. Retrieved from
http://www.who.int/disabilities/world_report/2011/report.pdf.
(26) Zimmermann, G., and Vanderheiden, G. (2008). Accessible design and testing in the application
development process: considerations for an integrated approach. Universal Access in the
Information Society, 7(1-2), pp. 117-128.
... Giovanna et al. [62] developed an open support accessibility evaluation tool to improve automatic accessible support following accessibility conformance testing (ACT) rules. Sanchez-Gordon and Luján-Mora [64] proposed an agile environment-based accessibility evaluation framework to improve evaluation results based on automated tools, simulators, and expert and user-based testing. In further evaluation, Song et al. [65] addressed the complexity of accessibility evaluation methods and the shortage of experts in this field. ...
... (FD 2 .) Assets for web accessibility evaluation: (1) design a cost-effective crowdsourcing framework for web accessibility evaluation considering 25 checkpoints and 5 conformance levels [59]; (2) proposed a framework in order to evaluate the well-known automatic accessibility tools in terms of webpage accessibility through their proposed measurement metrics [60]; (3) a crowdsourcing framework for web accessibility evaluation against web accessibility content guidelines checkpoints [61]; (4) an open and flexible accessibility testing tool to support single and multi-page validation [62]; (5) WUAM is a framework for websites usability and accessibility evaluation to improve website performance [63]; (6) proposed a framework for web accessibility improvement following ISTQB in agile contexts [64]; (7) proposed a crowdsourcing framework for website accessibility evaluation to identify the accessibility barriers and determine the overall accessibility level [65]; (8) proposed an API based website accessibility testing tool following ADA guidelines to identify the potential errors and violations, even without prior knowledge [43]; (9) proposed a framework for website accessibility barrier measurement according to several variable magnitude techniques [50]; 10) proposed a heuristic method to determine the level of accessibility of high ranked websites [66]. ...
Article
Full-text available
Several works of literature contributed to the web evaluation process in recent years to promote digital inclusion by addressing several accessibility guidelines, methods, processes, and techniques. Researchers have investigated how the web evaluation process could be facilitated by including accessibility issues to obtain an inclusive and accessible solution to improve the user experience and increase user satisfaction. Three systematic literature reviews (SLRs) have been conducted in the context of past research, considering such research focuses. This paper presents a new SLR approach concerning accessibility in the web evaluation process, considering the period from 2010 to 2021. The review of 92 primary studies showed the contribution of publications on different phases of the web evaluation process mainly by highlighting the significant studies in the framework design and testing process. To the best of our knowledge, this is the first study focused on the web accessibility literature reporting the engineering assets for evaluation of new accessible and inclusive web-based solutions (e.g., websites). Besides, in this study, we aim to provide a new direction to the web designers and developers with an updated view of process, methods, techniques, tools, and other crucial aspects to contribute to the accessible process enrichment, as well as depict the gaps and challenges that may be worthy to be investigated in the future. The findings of this SLR introduce a new dimension in web accessibility research on determining and mitigating the research gap of web accessibility issues for web designers, developers, and other practitioners.
... Accessibility should be considered throughout the whole software development life cycle. Most importantly, accessibility that is not well managed (e.g., remaining implicit or becoming late requirements) can lead to severe software development issues [37]. Software engineering plays a fundamental role in developing accessible applications since it promotes the integration between methodologies and specific accessibility techniques and activities in the software development process [33]. ...
... In addition, two interviewees mentioned that in their companies, accessibility is usually considered only at a late stage of development or during system refactoring and maintenance. Given the increasing demand for more accessible software [20,37,42], practitioners need to put more emphasis on accessibility understanding and the ability to appreciate trade-offs. The benefit-cost was repeatedly mentioned by interviewees, and a certain number of companies consider accessibility would be an extra cost for them to incorporate into development. ...
Preprint
Full-text available
Being able to access software in daily life is vital for everyone, and thus accessibility is a fundamental challenge for software development. However, given the number of accessibility issues reported by many users, e.g., in app reviews, it is not clear if accessibility is widely integrated into current software projects and how software projects address accessibility issues. In this paper, we report a study of the critical challenges and benefits of incorporating accessibility into software development and design. We applied a mixed qualitative and quantitative approach for gathering data from 15 interviews and 365 survey respondents from 26 countries across five continents to understand how practitioners perceive accessibility development and design in practice. We got 44 statements grouped into eight topics on accessibility from practitioners' viewpoints and different software development stages. Our statistical analysis reveals substantial gaps between groups, e.g., practitioners have Direct v.s. Indirect accessibility relevant work experience when they reviewed the summarized statements. These gaps might hinder the quality of accessibility development and design, and we use our findings to establish a set of guidelines to help practitioners be aware of accessibility challenges and benefit factors. We also propose some remedies to resolve the gaps and to highlight key future research directions.
Article
Being able to access software in daily life is vital for everyone, and thus accessibility is a fundamental challenge for software development. However, given the number of accessibility issues reported by many users, e.g., in app reviews, it is not clear if accessibility is widely integrated into current software projects and how software projects address accessibility issues. In this paper, we report a study of the critical challenges and benefits of incorporating accessibility into software development and design. We applied a mixed qualitative and quantitative approach for gathering data from 15 interviews and 365 survey respondents from 26 countries across five continents to understand how practitioners perceive accessibility development and design in practice. We got 44 statements grouped into eight topics on accessibility from practitioners’ viewpoints and different software development stages. Our statistical analysis reveals substantial gaps between groups, e.g., practitioners have Direct v.s. Indirect accessibility relevant work experience when they reviewed the summarized statements. These gaps might hinder the quality of accessibility development and design, and we use our findings to establish a set of guidelines to help practitioners be aware of accessibility challenges and benefit factors. We suggest development teams put accessibility as a first-class consideration throughout the software development process, and we also propose some remedies to resolve the gaps between groups and to highlight key future research directions to incorporate accessibility into software design and development.
Book
Jutaan bisnis menggunakan internet sebagai channel komunikasi yang efektif. Internet memudahkan mereka bertukar informasi dengan target market mereka dan transaksi bisa dilakukan secara cepat dan aman. Namun komunikasi yang efektif hanya dimungkinkan ketika bisnis tersebut dapat menangkap dan menyimpan data-data yang dibutuhkan serta memiliki tujuan untuk memproses informasi ini dan mempresentasikan hasilnya kepada user.
Article
Full-text available
De acordo com o 14º Relatório do Estado do Agile (Versionone, 2020), a Gestão de Projetos Ágeis (GPA) apresenta resultados positivos para melhoria da produtividade, alinhamento, visibilidade e priorização das demandas. Embora seja aplicada na indústria de desenvolvimento de software, outros setores buscam sua adoção a fim de obter os resultados conhecidos pela metodologia, como nos segmentos financeiro, comercial, marketing e industrial. Por meio de uma abordagem qualitativa, o presente artigo teve como objetivo compreender como os valores do Manifesto Ágil e das ferramentas scrum poderiam gerar resultados positivos de eficiência na rotina de uma indústria de mineração brasileira, que desconhecia completamente a GPA e apresentava falhas de gestão. Como protocolo de pesquisa, a intervenção ocorreu por meio de um programa de treinamento (presencial) com 14 funcionários da organização de diversas áreas, no qual foi medida a variação do ganho de eficiência após a aplicação da GPA na rotina. Como resultados, foi possível observar que houve variações positivas no relacionamento entre as pessoas (colaboração), maior proximidade do cliente no processo de entrega de valores (percepção do valor do produto), melhor comunicação e interatividade entre os membros da equipe, melhor priorização de tarefas (viabilidade), geração de novos produtos (inovação) e adoção de uma reunião retrospectiva ao final de cada ciclo de entrega como procedimento formal de melhoria contínua.
Chapter
ISO/IEC/IEEE 29119 is an internationally agreed set of standards for software testing that must be adopted during all software development processes and incorporated by every software company when conducting every software testing development. This article presents a study on ISO/IEC/IEEE 29119 standard, focusing on the description of a three-layer process model that covers: (1) organizational test specifications; (2) test management and (3) dynamic testing. Therefore, a comprehensive survey has been done to summarise and analyse the intended purpose of the implementation and adoption of this software testing standard. Furthermore, this paper also states the added value related to the implementation of this standard for any organisation in terms of software quality assurance. The adoption of software testing processes is an essential part of the software development that must be included in every development project, in particular, considering the artificial intelligence software development. The costs associated are relatively high. However, the cost associated with the treatment of bugs and software problems is even higher.
Article
Full-text available
Agile methodologies were born as a more flexible response to older methods of software development, that were more focused on planning than on interactions. In parallel, there is an evolution of the quality of the developed software, considering systems requirements such as user interaction, usability and accessibility. In this way, a systematic literature review was carried out aiming to identify the attention given to accessibility in software development in writings about agile methods, identifying how accessibility has been worked with agile software development methods. The research could find some studies in several areas of the software life cycle, such as development processes, accessibility assessment methods and accessibility refactoring, which show that although research in this area is still incipient, it has evolved considerably in the last decade.
Conference Paper
Full-text available
In a short period of time, the World Wide Web has had a huge impact on our society and lives. In web sites and web applications, accessibility and usability are essential key requirements. Unfortunately, most web sites are inaccessible to many disabled people and fail to satisfy the most basic standards for accessibility. Many of the barriers people with disabilities face on the Web are completely avoidable and the disadvantage associated with disability can be entirely overcome. To support the accessibility of web sites, different accessibility guidelines and standards have been introduced for the last ten years. Nevertheless, a web site can meet accessibility standards, but it can still difficult for people with disabilities to use it. Moreover, web accessibility has been often an afterthought in the development process of web sites. In many cases, web developers provide an adaptation or a fix to the interface of a web site after it has been released to the public. In this paper, we argue that the adoption of agile software development methods can help to improve the accessibility of web projects. Besides, the integration of accessibility into agile methods is proposed.
Conference Paper
Full-text available
The use of web accessibility evaluation tools is a widespread practice. Evaluation tools are heavily employed as they help in reducing the burden of identifying accessibility barriers. However, an over-reliance on automated tests often leads to setting aside further testing that entails expert evaluation and user tests. In this paper we empirically show the capabilities of current automated evaluation tools. To do so, we investigate the effectiveness of 6 state-of-the-art tools by analysing their coverage, completeness and correctness with regard to WCAG 2.0 conformance. We corroborate that relying on automated tests alone has negative effects and can have undesirable consequences. Coverage is very narrow as, at most, 50% of the success criteria are covered. Similarly, completeness ranges between 14% and 38%; however, some of the tools that exhibit higher completeness scores produce lower correctness scores (66-71%) due to the fact that catching as many violations as possible can lead to an increase in false positives. Therefore, relying on just automated tests entails that 1 of 2 success criteria will not even be analysed and among those analysed, only 4 out of 10 will be caught at the further risk of generating false positives.
Conference Paper
Full-text available
The topic I want to address is the role that accessibility evaluation methods can play in helping the transition from accessibility viewed as standard conformance, to a user-centered accessibility. As we will see, this change sets additional requirements on how evaluations of websites should be carried out. This paper first discusses different problems that occur while dealing with accessibility. We will see that different people have radically different views of accessibility and how it should be assessed. The first requirement is a clear definition of what accessibility is and how it should be assessed. The accessibility model discussed in Section 2.1 has precisely this role. Several existing evaluation methods are then reviewed and discussed, a simple taxonomy is presented, and differences that occur when evaluating accessibility rather than usability are pinpointed.
Conference Paper
Full-text available
Two tools are presented which support test case management for accessibility test suites. Creating test suites for the Web Content Accessibility Guidelines 2.0 is one major objective of the EU-funded project BenToWeb. Parsifal is a desktop application which easily allows editing test description files. Test description files compose an XML layer containing descriptive information about the particular test cases. Amfortas is a web application which allows controlled evaluation of the test suites by users. Controlled in that sense means, that Amfortas not only stores the evaluation results, but also is aware of the physical and technical condition of the evaluator.
Article
The goal of this paper is to review the literature in order to understand the implications of accessibility testing processes with the objective to detect potential improvements and developments in the field. Thus, a brief review is presented of the fundamental test processes proposed by the International Software Testing Qualification Board (ISTQB) and the currently available literature about testing processes for evaluating the accessibility of web applications. The result of the review reflects an array of proposals to incorporate accessibility requirements and evaluation tools, but they do not describe a comprehensive testing process at each phase of the development lifecycle of accessible web applications.
Conference Paper
This study presents the results of a web accessibility evaluation performed on a sample of six software products developed by small software enterprises of two countries. According to the International Standard Organization (ISO), an enterprise, organization, department or project with up to 25 people is considered small. All the products evaluated presented accessibility issues, mainly lack of HTML labels, alternative texts, and color contrast errors. These results showed there is a need in small software enterprises of an engineering development process that, taking into account their constraints of staff and budget, includes activities for improving the accessibility of their software. We present the current state of an ongoing work to define such process based on ISO/IEC 29110 that includes accessibility-related task in each of the following activities: initiation, analysis, design, construction, integration and test, and delivery.
Conference Paper
It is known that the adoption of user-centred design processes can lead to more universally accessible products and services. However, the most frequently cited approach to user-centred design, i.e. participatory design, can be both problematic and expensive to implement., particularly over the difficulty of finding and recruiting suitable participants. Simulation aids offer a potentially cost-effective replacement or complement to participatory design. This paper examines a number of the issues associated with the use of simulation aids when designing for Universal Access. It concludes that simulation aids can play an effective role, but need to be used with due consideration over what insights they provide.
Conference Paper
Web authors have a hard time understanding and applying accessibility guidelines. The guidelines are considered too technical, without providing sufficient support for problem solving. This results in bad usability of Web applications for people who rely on accessibility. In the field of designing Web applications and interfaces, the concept of personas, as a representation of the target audience, is well established. Personas are typically used to describe the user on a personal level, with their needs, preferences and habits. In this poster, we illustrate a new workflow approach for accessibility evaluations. We propose persona-based representations of accessibility guidelines for acceptance tests of Web applications, for web authors to gain understanding on the needs of people with disabilities and thus improve the accessibility of Web applications.
Article
Accessible design principles should permeate virtually all phases of the application development cycle, using existing “best practices of software engineering” for accessibility purposes. This paper proposes a methodology for accessible design and testing that includes proven tools of software engineering, namely use cases and scenarios, to capture functional requirements. Guidelines developed through user testing and heuristics are made real using personas to exemplify accessibility requirements, reflecting a diversity of user capabilities and use contexts. For implementation and testing, test cases containing accessibility checkpoints are generated, based on the guidelines. Complementary to this methodology, expert reviews and user testing should be conducted for evaluation of the developed products and further refinement of the development process.