Content uploaded by Garreth W. Tigwell
Author content
All content in this area was uploaded by Garreth W. Tigwell on Jul 25, 2022
Content may be subject to copyright.
Author Reflections on Creating Accessible Academic Papers
RACHEL MENZIES, University of Dundee, Scotland.
GARRETH W. TIGWELL, School of Information, Rochester Institute of Technology, USA.
MICHAEL CRABB, University of Dundee, Scotland.
Academic papers demonstrate inaccessibility despite accessible writing resources made available by SIGACCESS and others.
The move from accessibility guidance to accessibility implementation is challenging for authors. Our work focuses on
understanding what challenges authors of academic papers face in creating content elements (e.g., tables, charts, images)
to better understand how to improve accessibility. We classied 3866 content elements from 330 papers covering a 10-year
sample of academic work from ASSETS to understand the variety used. We also reected on the design choices that make
the content elements inaccessible. We then conducted interviews with 13 academic authors from PhD student through to
Professor Emeritus that publish within top-tier accessibility and HCI venues to understand the challenges faced in creating
accessible content. We found critical issues in how academics understand and implement accessibility while also balancing
the visual design of the paper. We provide recommendations for improving accessibility in the academic paper writing process
and focus on steps that can be taken by authors, publishers, researchers, and universities.
CCS Concepts: • Human-centered computing → Accessibility.
Additional Key Words and Phrases: Accessibility, Content Elements, Charts, Tables, Images
1 INTRODUCTION
Academic publications are a key part of knowledge dissemination. However, the papers are often not accessible [
7
],
meaning that people with disabilities are going to face barriers when trying to access the information. Recently, a
large-scale analysis of 11,397 PDFs sourced from dierent research areas (e.g., Biology, Computer Science, Physics,
Sociology) found that 97.6% had accessibility issues [
79
]. Prior work found similar PDF issues for HCI papers (e.g.,
only 23.3% of the 459 CHI 2014 Papers and Notes used heading tags) making them inaccessible to screen reader
users [
9
], even though CHI 2014 promoted the importance of creating accessible PDFs [
37
]. However, those
studies are mostly focused on screen reader and magnier accessibility, and we need to consider PDF accessibility
broadly for many types of impairments. Addressing paper accessibility is one important step toward meeting the
goal of making computing more inclusive [37].
In recent years, there has been advocation for accessibility becoming integral to creating academic content [
13
].
SIGCHI has already committed to making sure the academic papers from its many conferences are accessi-
ble. The CHI conference strongly encourages authors to make their papers accessible and provides guidance
(sigchi.org/conferences/author-resources/accessibility-guide), which has built upon the work of the SIGACCESS
community [
2
,
3
,
27
,
75
]. Furthermore, there has been a recent call to action for the HCI community to create
inclusive data visualisations for people with disabilities [
43
]. Data visualisations found in research papers can
range from simple bar charts to more complex and custom images.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Authors’ addresses: Rachel Menzies, University of Dundee, Dundee, Scotland., r.menzies@dundee.ac.uk; Garreth W. Tigwell, School of
Information, Rochester Institute of Technology, Rochester, NY, USA., garreth.w.tigwell@rit.edu; Michael Crabb, University of Dundee,
Dundee, Scotland., m.z.crabb@dundee.ac.uk.
© 2022 Association for Computing Machinery.
This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The denitive Version of Record was
published in ACM Transactions on Accessible Computing, https://doi.org/10.1145/3546195.
2 • R. Menzies, G. W. Tigwell and M. Crabb.
Although prior research has investigated this issue from the reader’s experience with inaccessible content,
we want to focus on understanding what challenges authors of academic papers
1
face that is contributing to
inaccessible papers.
Academic work often consists of many dierent content elements, such as tables, charts, images, and other
labelled content, which are chosen with the intent to best facilitate the reader in understanding the written work
and arguments being put forth [
6
,
35
,
50
,
53
,
58
]. However, authors need to be cognisant of the varying abilities of
the readers, and strive to ensure all readers have an equitable experience in accessing the information—especially
because academic work is often funded by public money [63].
Fig. 1. Timeline showing four parts of the research process toward publication: 1) Planning; 2) Conducting, 3) Writing; and 4)
Publishing.
Although accessible writing resources are available, the evidence of inaccessible papers from prior research
suggest that the move from guidance to implementation is challenging for authors.
We conducted a study that examined the frequency of specic content elements appearing in 330 ASSETS PDF
papers published between 2011-2020. We focused on ASSETS because ASSETS has a long history for requesting
accessible papers, and we can also review full proceedings across multiple years. Conducting this review helped
us to plan for our interviews discussing content element creation and to understand the variability in content
elements over time. We identied a total of 3866 content elements and classied them into 21 separate element
types (e.g., Annotated Screenshot). We also reected on the design choices that make the content elements
inaccessible.
Next, we interviewed 13 academic authors from PhD student through to Professor Emeritus who have published
academic work within top-tier HCI and accessibility venues.
Our interviewees discussed their experience with the writing and content element creation process (including
wider research team contributions), the motivation behind including content elements within their papers, how
the accessibility of content elements is considered, and what challenges may aect the process.
We found that authors carry out a balancing act between their preferred design of content elements and
ensuring the information is accessible. Creating accessible content elements is a multi-faceted issue that involves
consideration of all accessibility domains, but authors often do not have this specic expertise. Implementing
accessible content elements requires specic knowledge and skills, with guidance required on many granular
levels.
We discuss the implications of our ndings across the dimensions of content creator expertise, confer-
ences/publisher guidance, opportunities for researchers/industry to design new academic publishing tools
that support the creation of accessible content elements, and the role university/funders can play in supporting
positive change.
Paper Contributions: We make the following contributions: 1) We contribute quantitative data summarising
the frequency of use of content elements (e.g., tables, charts, images) over 10 years of paper proceedings, which
helps inform trends within accessibility and HCI research, as well as where to focus eorts when designing new
1
We will use the term authors for simplicity in our paper. However, we want to acknowledge that other people in a research team may help
to create content appearing in published work but who do not get authorship recognition.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
3 Author Reflections on Creating Accessible Academic Papers •
academic publishing tools that support the creation of accessible content elements; 2) We provide a thematic
analysis of 13 interviews with academic authors who describe how they consider accessibility when writing
accessible academic papers and the challenge they face achieving this; 3) We contribute recommendations on
how to improve the academic publishing process to support authors in making accessible content elements.
2 RELATED WORK
A CHI 2021 paper surveying accessibility papers published in CHI and ASSETS found that from 1994 to 2019
focus on accessibility research had considerably increased [
42
], indicating a growing interest from the wider
HCI community toward accessibility research—albeit one that disproportionately focuses on specic types of
disabilities. Studies that reect on HCI’s progress are essential for us to continue making positive change, and we
were inspired to inspect how the HCI publication process mediates the accessibility of papers.
2.1 Publication Process
Academic publication is complex and varied, with many dierent venues and formats used to share academic
outcomes. For example, an academic publication may include a PDF document, an HTML version of the paper, a
video, a talk, and supplementary materials such as data sets or code libraries. The primary publication is the paper,
which is typically presented as a PDF document. For many publishers, an HTML version of the paper may also be
available. The other ‘parts’ of a publication are expected to provide an introduction to the material (e.g. a teaser
video at CHI), a summary of the material (e.g. a conference talk), or to provide further detail (e.g. a data set that
can be downloaded and inspected). However, it is common that the additional elements will refer users back to
the paper in either PDF or HTML format. Furthermore, PDFs still remain the most convenient method for sharing
the detailed insights outside of an academic community. For instance, although the HTML versions of papers
are very accessible, they are behind a paywall and currently authors do not have an author-accepted HTML
manuscript that they can share. In this paper, we are focusing on the development of the written presentation
of academic publications as a PDF document, since this format is included at multiple stages of the publication
process, i.e. during peer review and as a nal publication.
The publication of academic work comprises many stages, from the conception of a research idea to a publication.
An example timeline is shown in Figure 1 and can be summarised as (1) Planning; (2) Conducting, (3) Writing
and (4) Publishing. Our timeline is not to be taken as denitive, we acknowledge that at each stage we can expect
some variation based on preferences and requirements of both authors and publishers, but we are highlighting
a typical standard process. In some cases, research is conducted by an individual and in some cases by a team
which can lead to variations such as allocation of tasks. The writing process also varies depending on preferred
tools and writing styles, as well as conference requirements. The submission of research papers is particularly
varied across dierent conferences. For example, ASSETS has a rebuttal phase
2
, CHI has moved to a revise and
resubmit model for full papers3, and NordiCHI has an accept or decline model4.
Considering the many variations that are possible within the publication timeline, we expect this is going to
aect the extent that authors can make a paper accessible. Furthermore, each author’s abilities and experience will
factor into this process. For example, academic authors with vision impairments face many challenges working
with sighted collaborators and particular collaborative writing tools [
15
]. Next we focus on prior work that details
the content presented within academic papers.
2https://assets21.sigaccess.org/call_for_papers.html
3https://chi2022.acm.org/for-authors/presenting/papers/
4https://www.nordichi.eu/2020/research-papers.html#/
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
4 • R. Menzies, G. W. Tigwell and M. Crabb.
2.2 Types of Content Elements
An academic paper presented as a PDF document contains many content elements. In this paper, we focus on
visual content elements such as tables, charts, and images that can be used in academic papers to convey complex
ideas in a small amount of space [
48
] and considered an abstract entity with semantics that subsequently convey
meaning [
23
]. It is the development of this meaning that assists in creating structure between content elements
and the academic work as a whole [
70
]. Visual data allows readers to consider the information collected and
encourages the comparison of multiple pieces of data [
77
], can assist in making arguments more persuasive [
61
]
and provide a rich way to assist in the storytelling aspect of a narrative [26].
Content elements should be accessible for all readers to interpret [
38
]. The process of achieving accessibility
rst requires that content elements are semantically correct, which is not always the case. Pareddy et al [
54
]
conducted an analysis of images in HCI conferences and CS ArXiV. Reviewing papers across one year, they
identied content being misrepresented as an image, e.g. a screenshot of a table being included in a paper. In
addition, achieving accessibility involves making sure that elements are visually accessible (e.g., using patterns
instead of colours in charts) and also that an additional text-based alternative is present for usage when content
is not consumed visually (e.g., image alt text for screen readers). This process includes making sure that elements
can be perceived and understood, while also allowing creativity to occur with forgiveness, where misunderstands
in data presentation will still lead readers to the correct nal outcomes [39].
One method of presenting data in a paper is to use tables that enables the reader to view precise numbers whilst
also referring to summary values [
45
]. Tables are generally structured into rows that represent variables and
columns that represent records [
32
]. Within a table, creators should consider the clarity of data being presented,
consistency in design, the use of abbreviations, and logical order of row and column data [
51
]. Authors also need
to consider the available space (sometimes limited by paper format) when constructing tabular data [69].
Second, data can be presented visually in charts [
24
] to create a graphical representation of categorical (i.e.
nominal and ordinal) [
25
] and numerical (i.e. interval and ratio) data [
32
]. Using charts can support readers in
understanding data that might not translate well in summary information alone. Such visualisations can reveal
information that is ‘hidden by computed statistics’ [
52
] and is seen as a method to discover and reason through
data.
Third, images can be used to help the reader to understand specic points being made and also to illustrate
ideas that the author may have [
12
]. They are traditionally used as a graphical representation of an object or a
scene that can be used to show the reader contextual information about a specic moment in time or to provide a
broader overview of an event [
44
]. Photographs, in particular, provide the opportunity to place a lens through
which readers can view dierent lifestyles and cultures that may be important to understand for a given research
topic [
57
]. However, care must be taken when including images within work as they can be received as ller
content that is seen as a way to increase paper length [46].
When images are used within academic work, the overall composition should increase reader focus on the
highlighted aspects [
40
]. If possible, a clear white background should be used to reduce image clutter and increase
image attention [73]. Images of scenes should be captured in good lighting and with sucient detail [31].
2.3 Understanding Accessibility and Why Accessible Content Elements Are Important
Creating accessible content is important but requires an understanding of the process. We have seen that formal
education rarely prepares people with the expertise needed to avoid creating accessibility barriers (e.g., software
engineers [
55
]), and this is likely due to accessibility topics being less developed than other Computer Science
topics [
4
]. The result is a general lack of understanding of how a person with a disability uses technology, and
impedes the way that developers create digital resources [14].
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
5 Author Reflections on Creating Accessible Academic Papers •
Inaccessible visual information can impact the ability of people to understand content. Accessibility can be
categorised into the general areas of visual, cognitive, physical, communication, emotional, and intersectional
areas [
14
]; content elements can impact all of these areas. For example, poorly made charts can cause cognitive
accessibility challenges if the information is dicult to understand [33], but the inclusion of items such as ow
charts to explain procedures can increase overall accessibility by providing information in a dierent manner
5
.
Additionally, content elements that are too small (e.g. condensed legends in charts, or highly detailed images) can
pose visual challenges that can be solved by zooming in digital formats, however, this also increases dexterity
demands which may isolate other readers [76].
Visual data should be presented to increase the accessibility for all readers [
21
,
30
]. As part of this, the selection
of colour within visualisations is an important aspect to consider. Colours should be easily understood by people
with impaired colour vision [
11
], print well in grey-scale, and also remain pleasant to look at [
8
]. As a requirement,
the level of detail between colour levels (luminance contrast) should be as high as possible to allow adequate
dierences to be present [
80
]. In addition, the presentation of supplemental information to assist in understanding
charts (e.g. legends, labels) should be of sucient size for the reader to easily see [5].
Visual data can be remapped to provide an alternative accessible format. The challenge here is to present the
data in both a visual and non-visual manner. This dual presentation of information presentation is not new; when
visual content was becoming commonplace within academic work the Royal Society asked for it to additionally
be reduced to writing so that a deeper understanding of the data can be obtained [
28
]. It is now common to
expect ‘alt-text’ to be presented for every piece of visual content within a piece of work
6
as a method of providing
accessible content to all readers. Alt text is important for screen reader accessibility, but people do not often
create well dened alt text. For example, Sharif et al [
64
] identied that alt text is frequently lacking for screen
reader users or does not contain eective descriptions such as overall trends for charts, which renders data
visualizations inaccessible for screen reader users. They showed that screen reader users extract information
61.48% less accurately and spend 210.96% more time interacting with online data visualizations compared to
people who do not use screen readers. In addressing this challenge, Mack et al. has created an alt text authoring
and feedback tool [41].
Once we understand where HCI authors are struggling in the paper writing process, it will provide us with
insights on where new accessibility tools for content elements should be developed.
Despite the large body of prior work that has focused on understanding what the accessible format of data
may look like, very little has focused on the processes that authors and designers must go through in order to
reach this level. Accessibility does not constrain visual design [
56
], yet a mixture of creative forgiveness [
39
] and
complex accessibility requirements makes it very dicult to produce a method to easily assess and implement
accessibility associated with content elements within academic work. It has been discussed that less than 25% of
author-supplied alternative text is sucient enough to understand visual data contents [7].
2.3.1 Accessible Content Guidance. When discussing the accessibility of online content, it is common for the
W3C Web Content Accessibility Guidelines (WCAG) [
34
] to be one of the primary resources that is used. The
main purpose of WCAG is to improve the accessibility web content by providing guidance that assists in making
material perceivable, operable, understandable, and robust. These four guiding principles are used to categorize
guidelines that can then be met via a number of success criteria.
Of interest for this work is success criteria that relate to non-visual text
7
and PDF documents
8
. Yet, a challenge
arises in creating accessible scientic documents due to the complexity of images used, the form factor of the
5https://www.w3.org/TR/coga-user-research/
6https://authors.acm.org/journals/how-to-write-alt-text-and-why
7https://www.w3.org/TR/UNDERSTANDING-WCAG20/text-equiv-all.html
8https://www.w3.org/TR/WCAG20-TECHS/pdf.html#pdf_notes
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
6 • R. Menzies, G. W. Tigwell and M. Crabb.
documents themselves (i.e. PDF), and a lack of technical guidance on how to create suitable alternatives. The
W3C Web Accessibility Initiative provides tutorials on providing alternative text for complex images
9
, but success
criteria linked to this are either based on general implementation or technical implementation for HTML. Prior
work has also mentioned data visualization practitioners fo not have sucient evidence-based guidelines and
tools to support them in creating accessible content [43].
An alternative to WCAG when focusing on the accessibility of digital documents is ISO 14289:1
10
, commonly
referred to as PDF/UA (Universal Accessibility). This international standard focuses on the technical implemen-
tation of accessibility inside of PDF documents with a focus on correct methods of tagging content inside of
a document itself. Similar to WCAG, PDF/UA provides general guidance on how individual elements can be
made accessible and the subsequent technical implementation of these methods within the PDF le format. The
challenge in both cases is in the overall granularity of the information that is provided, and how authors use
these sources (and others) in the development of accessible scientic papers.
2.4 Summary and Research estion
The previous work we report outlines variations in the publication process, types of content elements used in
papers, understanding accessibility, and why accessible content elements are important. However, since academic
papers are often inaccessible, it seems there is still a challenge with creating accessible content for academic
paper and suggests that the move from guidance to implementation is challenging for authors. To address this we
need to better understand the challenges that authors face in making their papers accessible—not only for screen
readers, but to accommodate many dierent impairments. Our work is therefore motivated by our RQ: What
challenges exist when implementing accessible content within academic writing?
3 QUANTIFYING CONTENT ELEMENTS
Inaccessible papers are a known issue within the research community [
7
,
9
]. However, the focus of our research
is on understanding what challenges authors face that is contributing to inaccessible content elements. The
primary focus of our work is qualitative research with academic authors who publish academic work within
top-tier HCI and accessibility venues, but in order to prepare for our interviews (see sections 4 and 5) we rst
want to understand the extent and variability of content elements added to research papers in our eld, and how
trends might have changed in recent years. In addition, our quantication of content elements can be used to
prioritise the development for future tools to support content creation (see section 6.4).
3.1 Method
We base our content elements quantication method on prior work from ACM ITiCSE [
68
]. We quantied all
content elements within full papers from the ASSETS conference over a ten year period (2011-2020).
We chose ten years of ASSETS papers for several reasons. First, ASSETS is an accessibility based conference,
with a community that has close ties to larger conferences such as CHI, and it has a long history of encouraging
authors to make their work accessible (i.e., we expect ASSETS papers to demonstrate best practices). Second,
ASSETS 2011 follows a year after the conference rst promoted the importance of creating accessible PDFs,
so organisers had time to rene anything after the its introduction if authors needed more guidance
11
. Our
examination of a ten year sample allowed us to include papers that have gone through ve format changes
of varying page lengths and reference limit. Third, conferences (including CHI) regularly signpost to ASSETS
9https://www.w3.org/WAI/tutorials/images/complex/
10https://www.iso.org/standard/64599.html
11http://assets10.sigaccess.org/accessible_pdfs.html
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
7 Author Reflections on Creating Accessible Academic Papers •
materials for creating accessible papers and publications from ASSETS is reected in this guidance, meaning that
the HCI community recognizes ASSETS as the top-tier conferences for leading accessibility eorts.
Our code book contains 21 codes within four broad categories: Table, Chart, Images, Other. The full code book
can be found in Appendix A, with this being created after pilot work examining papers from ASSETS 2018. We
applied the code book to the ten years of papers, such that two authors were assigned to each year. Each author
coded papers individually and then discussed disagreements. The purpose of this initial study was not specically
to identify accessibility challenges in the papers as this is considered in previous work (e.g., [
1
,
7
,
9
,
79
]). The
primary purpose was to identify the types and frequency of content elements in order to better inform the design
of future tools and applications to support their development and inclusion in academic work.
However, we did reect on the accessibility of each element based on our own previous experience (a combined
33 years in accessibility research). Each author kept notes during this process on any good practice that was
identied and any accessibility challenges encountered. These notes form the basis of insights presented in
section 3.3. Initial agreement over all conference years was 75.9%, with the remaining author consulting when a
consensus could not be reached.
We did not make decisions based on the appropriateness of gures used, but on the content of the gure. For
example, where charts were used inappropriately, we coded the intention of the author, e.g. a line chart that
incorrectly uses categorical data will be coded as a line chart (even though the more appropriate chart to present
the data would have been a bar chart).
For single gures with multiple component parts, e.g. two charts that are labelled or dened in the gure
caption as parts (a) and (b) were coded as individual instances. Where a gure consisted of multiple parts that were
not labelled or dened these were split whenever each part contains specic dierent information components.
In some cases we encountered content that was not captioned as a gure or equivalent. When data was
presented, e.g. as a table, without a caption it was coded, but for equations presented within a block of text, these
were not coded because they were not called out as a distinct content element for readers.
3.2 Results
We reviewed 330 papers, and these contained 3866 content elements (tables and gures). Table 1 summarises the
total count of each broad-category content element type per year. The average number of elements in each paper
was 12 (min=0; max=88). Six papers had no content elements.
3.2.1 Tables. 76% of all papers contained at least one table of any type. The average number per paper was
2 (min=0, median=2.5, max=10). Of the tables presented, 94% were basic presentations of data using text and
numbers, with varied template compliance.
Some papers (n=41) contained more complex tables that used colour or images as part of the data. We also
found examples of tables being used to appear visually as charts. In one paper we even noted a table that contained
animations embedded into the PDF le itself—but the animations would only work if viewed in Adobe Acrobat.
3.2.2 Charts. The average number of charts per paper was 2.4 (min=0, median=1, max=16) and 54% of papers
contained at least one chart. The types of charts presented most frequently were bar charts (35% of all charts),
“other” charts (28%) and line charts (21%). “Other” charts included complex presentations of data, e.g. with multiple
axes or stacked bar charts. Pie charts, histograms and scatter plots were considerably less popular (see Table 1).
3.2.3 Images. The average number of images per paper was 6.7 (min=0, median=5, max=69) and 86% of papers
contained at least one image. We recreated an example of each type of image (see Figure 2). The images presented
most frequently were screenshots (24% of all images), diagrams (19%) and photo scenes (18%).
Screenshots comprised high-delity screen mock-ups as well as capturing applications on a range of dierent
devices, such as smartwatches, mobile phones, tablets, desktop computers, Augmented Reality (AR) headsets and
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
8 • R. Menzies, G. W. Tigwell and M. Crabb.
Table 1. Frequency of each broad-category of visual element identified by proceedings (number of papers each year).
Additional data tables for specific content element frequency are in Appendix B (see Section 7 for a reflection on the dierent
table and chart designs we tried before seling on a simple table).
Papers Year Tables Charts Images “Other” Total
27 2011 56 73 172 15 316
24 2012 37 64 186 37 324
29 2013 65 80 171 19 335
29 2014 62 70 203 1 336
31 2015 61 41 184 9 295
28 2016 43 81 205 11 340
36 2017 82 75 249 12 418
31 2018 65 84 176 11 336
45 2019 91 101 332 14 538
50 2020 100 126 347 55 628
– Total 662 795 2225 184 3866
Fig. 2. Our recreation (to preserve anonymity) of (a) a photo object, (b) an annotated photo object, (c) a photo scene, (d) an
annotated photo scene, (e) a screenshot, (f) an annotated screenshot, (g) a drawing, (h) a diagram.
Virtual Reality (VR) headsets. Annotated screenshots typically included labelling dierent parts of the screenshot
to aid in further discussion within the paper, but were much less common, being only 4% of all images.
Diagrams typically included owcharts to explain experimental or algorithmic processes, room setups for
experiments and representations of dierent interface interactions.
Photo scenes presented assistive technology being used in context or a research session in progress. Annotated
photo scenes, e.g. to label dierent aspects of the scene, represented 7% of all images.
3.2.4 Other. The average number of “other” content elements per paper was 0.6 (min=0, max=18) and 14% of
papers contained at least one “other” visual element. These are listed in order of percentage of papers that contain
at least one of these: equation (7%), text (5%), pseudocode (1.5%) and code (1.5%). Text as a gure was found in 16
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
9 Author Reflections on Creating Accessible Academic Papers •
papers. Examples included presenting audio transcripts or screen reader output alongside screenshots, as well as
highlighting design prompts used in focus groups, such as method cards.
3.2.5 Comparison to other selected work. We compared our results with those from Simon et al. [
68
], on which
our methodology was based. We found that content elements were more prevalent in ASSETS papers compared
to computing education papers at ITiCSE: 76% of the papers contained tables compared to 61%, 54% of the papers
contained charts compared to 49%, and 86% of the papers contained images compared to 69%.
In addition, other work from within HCI, e.g. [
54
] has identied a number of issues that relate to the creation
of dierent content elements. Analysis of images in HCI conferences and CS ArXiV from 2018 were reviewed to
determine the content elements within a sub-set of these papers.
Images were the most popular content elements across both ASSETS and iTICSE conferences, but the types of
images diered: in the accessibility papers, photos were more popular with a particular emphasis on annotated
photo objects. This is consistent with the prevalence of ASSETS papers that investigate new assistive technologies
since many images detailed the construction and use of assistive technologies, and is also common for papers
published within the accessibility sub-committee at CHI. Of interest, Pareddy et al [
54
] conrmed that the variety
of content elements seen in ASSETS proceedings are also found across other HCI conferences. Hybrid images
(gures containing more than one image) were less common in ASSETS than other conferences.
Despite variations in the prevalence of dierent content elements, it is clear that the dierent categories exist
across at least two dierent subject areas and that the order of popularity is consistent: images are the most
popular, followed by tables and then charts. Therefore, it is likely that content elements can result in challenges
in multiple research areas. With this in mind, it is clear that if we are to prioritise the development of accessibility
design tools to support implementing accessibility in academic writing, then tools for making accessible images
is a priority across the accessibility/HCI and educational computing domains, followed by table creation, and
then chart creation. However, to inform the development of such tools, we still need to understand how and why
authors create the content elements they do and where there are issues in the writing process.
3.3 Accessibility Challenges Encountered
During our audit of papers, we identied inaccessible content elements that were present in the papers. Insights
recognised during the audit were discussed by the authors on completion of the audit, and are outlined here.
Prior work discusses the need to consider cognitive, motor, and visual impairments when creating inclusive
data visualisations for people with disabilities [
43
], although for our work we want to have a broader outlook
by considering all aspects of paper content, as well as situational impairment when reading. The purpose of
sharing these details is to emphasise that even with guidance for authors to follow, papers are being published
with various inaccessible content elements. Reecting on these observations also helped us with planning for our
interviews to nd out why inaccessible content occurs.
3.3.1 Complexity of Information. We found evidence of rather complex tables being used within the papers.
Examples of this include: headings with subheadings, merged table cells resulting in headings spanning multiple
columns, multiple values contained in a single cell, the inclusion of images and animations within cells (see
Figure 3.b), as well as other information “baked” into the visual design of the values (e.g., bold or underline styles
to indicate statistical signicance, table cell background colours with assigned meaning, and symbols). It is no
longer a simple task to read from the top heading down or left column across. Sometimes tables include a new
top-level heading or commentary midway down the table and it often seems like those decisions are a result of
trying to save space rather than opting to present multiple tables, which would simplify the table design. As a
result, the reading ow is easily interrupted.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
10 • R. Menzies, G. W. Tigwell and M. Crabb.
Fig. 3. Our recreation of challenges with presentation of tables: (a) a table being used to create a chart, (b) a tables containing
images.
Fig. 4. Challenges with presentation of charts: (a) a stacked bar chart, (b) a 3D bar chart, (c) a bar chart using only colour to
determine categories.
We found examples of many dierent charts, and, while one type of chart may be used, the data sometimes
could have been presented in another format with the added benet of being more accessible. For example, two
types of problematic charts we want to highlight are the stacked bar charts and 3D charts (see Figure 4.a and
Figure 4.b). Prior work has warned about the diculty of interpreting those styles of chart. Specically, stacked
bar charts can be dicult to compare adjacent bars in a stack [
72
], while the use of 3D eects can negatively
impact readability by aecting a person’s ability to judge bar height accurately and in the context of surrounding
bars [
82
]. Using a standard, at bar chart design with patterns to distinguish between bars would likely improve
readability. Furthermore, the use of colour and symbols in charts will also add to their visual complexity (see
Section 3.3.2 for a discussion on colour accessibility).
Finally, the various gures included in an academic paper can range from a simple photo of a device to a
complicated diagram made up of screenshots, arrows, labels, and icons with dierent meanings. Images can
include multiple layers of meaning that require specic scientic or cultural knowledge and these should be
appropriately explained.
3.3.2 Use of Colour and Symbols. Colour often plays a role within society and in dierent cultures to signify
information [
18
,
19
]. For many people, colour seems like a natural way in which to represent or highlight
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 11
information. Yet, using colour as the sole means of representing information is problematic [
11
] and the approach
is discouraged by the Web Content Accessibility Guidelines (WCAG) [34].
We are often advised to consider colour blind people who have reduced colour dierentiation abilities compared
to people with typical colour vision, however, this view is restrictive. It is more appropriate to discuss the idea of
impaired colour vision (ICV) and when doing so, it is clear how important the consideration of colour is on a large-
scale. ICV can result from three non-mutually exclusive factors: congenital ICV (e.g., aected X-chromosome [
49
]),
acquired ICV (e.g., solvent abuse [
17
], brain damage [
83
], aging [
16
]) and situational ICV (e.g., printing papers in
greyscale, ICV caused by room lighting or tinted lenses [22, 74]).
We often found colour was used to represent information without any other visual indicator. Sometimes
the chosen colours lacked sucient distinction and at other times were easy to distinguish but only when not
experiencing any form of ICV. Red, orange, yellow, and green were common colours found within various charts
(see Figure 4.c) and gures, and although their chromatic dierence may be distinguishable for some, these colour
are dicult to distinguish for the most common type of congenital ICV [
65
]. Sometimes the brightness of the
colour is varied, which should help since contrast detection is often not a problem, but this relies on the authors
applying a sucient level of contrast between the colours. One example of insucient contrast being applied
was to a stacked bar chart using a single hue and seven shades of a single colour. The dierence between the
darkest and lightest shades was a contrast ratio of 5.1:1, but the dierence between the darkest colour and next
shade lighter was only 1.26:1 dierence, and even the data point two shades lighter still failed to meet WCAG’s
minimum recommendation of 3:1 [34].
In one case, we found a table with two background colours that only had a contrast ratio of 1.12:1. Colour was
also used to emphasise the dierence in the size of numbers, and although this approach is ne when the number
itself is present to represent the data in another form, an issue arises when the background colour and text colour
converge so that they become indistinguishable. When measuring an example of this, we found a contrast ratio
of 2.02:1 for the darkest background shade with black font.
This is not to say we found no evidence of good practice for using colour with visual element. Some papers
utilised patterns, shape, size, and labels to represent information in another way so that if the colour was used,
it was not the only method for displaying important information. However, it is easy to make errors, and even
within papers that demonstrated good practice, this was not always consistently applied.
In some cases, patterns or symbols were used in place of colour. Sometimes the dierence between dashed lines
or symbols was not distinct enough and could result in confusion. For charts, there may be some expectation the
reader will match the order of the chart key to the order of the bars in the chart, but the reader has to make an
assumption here, and we know that mistakes can make it through to publication (see Section 3.3.4 for discussion).
3.3.3 Image ality. Image quality is an important part of visual elements within research papers. Papers can
include example gures that are not meant to provide specic information beyond a high-level overview. For
example, one paper provided an example of dierent chart graphics and clearly stated as such with the caption.
Usually, these examples are not given much space in a paper and so the ner details are lost. This should be
acceptable if the reader is explicitly made aware that the gures purpose is not to give detailed information,
otherwise, an argument could be made that unnecessary information should be removed for simplicity (see
Section 3.3.1).
Ideally, authors should seek to include text and equations as an embedded part of the document and not part
of images, but we found this not always to be the case. For example, we found an image of an equation—it was
slightly squished and stretched, which made for an odd reading experience. Furthermore, if the reader needs to
adjust the font for improved readability, such as to accommodate dyslexia [
59
], then this important element will
not have the reader’s preferences applied to it.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
12 • R. Menzies, G. W. Tigwell and M. Crabb.
3.3.4 Misleading Information and Unconventional Formaing. We found evidence of mislabelling that can create
confusion for readers, such as with the numbering of tables and gures. For example, ‘Table 2’ listed in a paper
without any existing ‘Table 1’ or two tables both labelled ‘Table 1’. The organisation layout for a paper is also
important to consider and we found the numbering of visual elements could sometimes fall out of the typical
reading order (e.g., ‘Table 3’ appearing before ‘Table 1’ and ‘Table 2’). We also found an example of no in-text
reference to a gure, but the gure did include a detailed caption. However, the question is at what point is the
reader supposed to focus on the gure or can it be ignored entirely, and if so, is there a need to include the gure?
Captions help to explain the content shown in tables and gures but they can be a source of confusion.
Challenges we found included mentioning something in the caption that is not included in an image
12
, captions
lacking in sucient detail, and omitting captions entirely. In the case of vague or no captions, these instance will
causing the reader to switch their attention between the relevant body text and the visual element position, which
may not always be in close proximity. Captions are also an opportunity to highlight when charts may be using a
particular type of scale such as a logarithmic scale. In doing so, authors can use captions as an opportunity to
emphasise this to the reader who may otherwise misinterpret the chart by assuming something else like the use
of a linear scale.
We previously discussed the challenge presented by complex formatting of visual elements, such as tables with
many headings, sub-headings, merged cells, images or animations (see Section 3.3.1), yet another aspect worth
discussing is opting for a visual element that may be unconventional or unnecessary. For example, the use of a
table to list information rather than using a bullet point list or using a dierent design for each table in a paper,
thus lacking consistency. Further, we often saw tables listed as gures, and even a chart styled to appear as a
chart in a similar way that one could inappropriately use a table for layout purposes in HTML (see Figure 3.a).
Finally, we found visual elements with parts missing, either during the writing process or when the publishers
format the nal document. We found an example of this in a paper using radar plots. The radar plots were missing
the axis, grid, and lines connecting the data points, but the authors had also included the same data in a table and
so readers are still able to access accurate results information.
3.4 Summary
Overall, we coded 3866 content elements within 330 papers. The most common type of visual element used was
images. Tables were also a common way to visually display information. The use of charts was varied, with
bar charts being the most popular. Only six papers (2% of the total) did not contain any content elements. We
therefore ran interviews to further understand the content element creation process.
We also found many dierent types of issues related to content elements in the research papers, reinforcing
suggestions from prior research that the move from guidance to implementation of accessible content is challenging
for authors. Our discussion of the problems we identied primarily isolated the issues so that we could discuss
the implications of each with more clarity. However, we want to stress that these issues can appear in conjunction
with other issues in the same visual element (e.g., a low-resolution image that relies on colour-coded annotations
and abbreviations), as well as a single paper having multiple visual elements with problems.
It is clear that authors value the inclusion of content elements to assist in demonstrating aspects of their
work that cannot be easily explained by text alone—this nding likely generalises across HCI when we consider
the popularity of system building and evaluating at conferences such as ACM CHI and UIST. However, the
challenges faced by authors when making content accessible to the wider scientic community remains present.
Implementing accessibility can be a challenging task with a large number of areas that have to be considered.
In order to investigate this further, we carried out semi-structured interviews with academics from the HCI
12The object was likely removed during the writing process and the caption was not updated.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 13
community to focus on the implementation of accessibility within academic work, and to identify what solutions
can address this problem in the future.
4 INTERVIEW METHOD
Our review of 330 academic papers provided us with insights into the frequent use of a variety of content elements
over time, as well as the presence of accessibility issues, and the wide variety of ways HCI authors present
information in papers. In order to understand how, and why, these content elements were created, and to see
if authors can reect on why accessibility challenges are occurring, we conducted interviews with authors to
answer our RQ: What challenges exist when implementing accessible content within academic writing?
4.1 Procedure
After obtaining ethical approval from our IRB, we conducted semi-structured one-to-one interviews using video
conferencing tools. The rst and second authors conducted all interviews using a shared interview guide. Both
independently conducted a pilot interview and compared approaches to ensure consistency, and to review
the interview structure and questions. Our participants were recruited via social media and the ACM access-
announcements mailing list. The mean interview time was 54 minutes (range: 35-66 minutes).
4.2 Participants
We present only minimal professional experience details and omit personal data such as age and gender in
order to preserve author anonymity, especially to reduce risk of our disabled authors being identied from
within the relatively small academic accessibility community; a practice followed by other authors in the area of
accessibility [66, 67].
We interviewed 13 participants who have published academic work both within top-tier HCI (e.g., ACM CHI,
CSCW, DIS, IDC, ISS, TOCHI, UIST) and accessibility (e.g., ACM ASSETS, TACCESS) venues. We did not target
authors from within any particular conference, but instead focused on authors with experience of publishing
papers related to accessibility topics. Published papers had coverage of seven contribution types within HCI [
81
]:
empirical, artefact, methodological, theoretical, dataset, survey and opinion.
Our participant sample represented diversity with regards to the current stage they are at in their careers: from
PhD student through to Professor Emeritus. Our participants had varying levels of publishing experience, which
was reective of how long they had each worked within academia. Publication output was as low as three to over
250 published works (median=44). When we focus specically on their experience with publishing accessibility
research, the median self-reported number of years was ve (min=1; max=23) and median self-reported number
of published accessibility papers number was nine (min=1; max=200).
Participants were not asked to disclose if they had a disability during the interview. However, some disclosures
were made to give context to the participant’s answers: Three participants disclosed that they were blind or had
a signicant visual impairment, and one participant disclosed that they had a movement disorder.
4.3 Analysis
We analysed our interview transcripts using thematic analysis [
10
] with all interviews being transcribed by the
third author. Codes were generated by the rst author using a data-driven approach then collated and collapsed.
All authors then reviewed the nal coding and identied similarities to allow thematic grouping by creating
an initial thematic map. We did not conduct inter-rater reliability because it is not part of Braun and Clarke’s
checklist for good thematic analysis [10].
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
5 INTERVIEW RESULTS
14 • R. Menzies, G. W. Tigwell and M. Crabb.
Our thematic analysis identied four themes (Figure 5): 1) Inclusion of content elements is based on visual design
and presentation, 2) Motivation for accessible content elements is grounded in awareness of others and self, 3)
Implementation of accessible content elements is heterogeneous, and 4) There is a balance between accessibility
and visual presentation of content elements.
Fig. 5. Final thematic map of four main themes and their sub-themes: 1) Inclusion of content elements is based on visual
design and presentation, 2) Motivation for accessible content elements is grounded in awareness of others and self, 3)
Implementation of accessible content elements is heterogeneous, and 4) There is a balance between accessibility and visual
presentation of content elements.
Unsurprisingly, participants approached paper writing in a variety of ways which reects the dierent writing
styles within academia. For example, participants made use of their preferred templates at dierent stages of
their writing process, and reported a range of writing strategies, e.g. starting with headings and bullet points or
writing linearly from beginning to end of the paper.
5.1 Inclusion of content elements is based on visual design and presentation
The inclusion of content elements within the paper is primarily based on visual design and presentation. The
visual aesthetic of papers was important to participants, with some noting the impact of images on the reader
and the importance of the visual presentation within the paper. For example, a teaser image was considered to be
a key element of the visual presentation of the paper:
P1: “If you don’t have a [teaser] image, they won’t read past the abstract and I always thought this was very sad, but
it is kind of true.”
However, the emphasis of visual formatting was a concern, due to the additional impact on accessibility:
P6: “If you distinguish content from presentation, we indulge ourselves in a world that is presentation oriented, and
then we have to do all this work to repair the deciencies of that particular presentation.”
Information was presented in content elements in a variety of ways, e.g. tables for demographics or raw data,
gures to show a user journey or to showcase a prototype interface. Overall, there were many positive reasons
for including these, such as enhancing the understanding of the paper, particularly for dierent audiences:
P9: “We use that visual representation to help non-technical people understand how the data worked in the...algorithms.”
Enhancing this understanding was achieved through adding context in several dierent ways. For example,
our participants discussed how visuals in the form of charts, diagrams, gures, images, and tables. Images can be
useful to “to demonstrate members of the community engaging with technology” (P12), whereas tables can provide
a “concise and visual way [...] to provide contextualised information on the participants” (P4), and diagrams to place
the work within the context of the author’s wider research contributions. This matches our observations when
we reect on our rst study evaluating ASSETS proceedings over 10 years. We saw how authors meticulously
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 15
annotated photos and screenshots—sometime with intricate detail—while breaking what might be viewed as
classic convention to creatively share information albeit at the cost of accessibility (e.g., rather than tables being
reserved for reporting basic text and numerical info, we saw complex tables with nested information, colour
coding, and even animations that only work if viewed with Adobe Acrobat). Some of this provision of context
was considered to be required by “convention” (P2) and was “standard practice” (P7), which suggests that there
are tensions surrounding expected author practice in how to eectively disseminate research in an accessible
way and sharing information in creative and immersive ways.
Content elements were considered to be of value to the reader, not only for the content that they provide. For
example, P11 commented that they make use of images for “mixing up the medium a little bit, giving people a
break from the text”.
Despite the ways that the content elements added value, participants agreed that they are supplementary to
the textual content and tended to favour sighted readers in particular:
P8: “Someone with a visual impairment who is using a screen reader would still have access to the data, but, for
sighted people, it just adds that little extra bit.”
5.2 Motivations for accessible content elements is grounded in awareness of others and self
Participants have a range of motivations for creating accessible content elements in their papers based on their
knowledge and awareness of abilities and disabilities of authors, reviewers and readers. P4 stated that they “want
to reach more people”, while P10 indicates that accessible papers have wider career benets:
P10: “It’s also a bit selsh because it increases your citations.”
Initially, participants focus on making their papers accessible for reviewers in order to aid their acceptance for
publication.
P7: “I always like to submit accessible versions for review because I assume that my reviewer is going to be a screen
reader user.”
However, this was not always the case for participants at an earlier stage in their careers, with participants
becoming more aware of reviewers with disabilities later on:
P4: “I think it’s a huge hindsight of mine that I didn’t think about reviewers who might have certain kinds of
impairment.”
5.2.1 Reviewers can influence paper content. One prominent discussion around the role of the reviewer was
requests from the reviewer for changes to the paper. In one case, a participant was asked to remove external links
from their paper, due to concerns about readers not having access to appropriate libraries to access accessible
charts online, leading to a conict between providing alternative formats and readers having access to those
alternative formats:
P5: “I think the reviewer said: [he] would probably remove [links to accessible charts] because, unless you have
various packages installed, it doesn’t really work, and it could lead to frustration in querying the data and querying
the work”.
In contrast, reviewers have also asked for additional content elements to be provided, such as adding tables
to show data. Given the anonymity of the review process, there is no way for a reviewer to know whether the
submitting author(s) have disabilities, which can lead to tension when these requests are made:
P11: “Being that I’m blind, I have gotten more than one review saying that maybe I should have more visuals in my
papers. . . which I’m just kind of always annoyed about!”
Many of our participants were also reviewers for HCI and accessibility conferences and reported a lack of
requirements to consider accessibility as a reviewer:
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
16 • R. Menzies, G. W. Tigwell and M. Crabb.
P10: “There weren’t any guidelines to check accessibility, so, I didn’t really do it, I was more focused on the content at
that point.”
5.2.2 Awareness of the reader can be limited. When considering the abilities of the reader, participants focus mainly
on screen reader users, with some consideration of low vision users. There was a consensus that accessibility
requirements for screen reader users were relatively simple to achieve. However, participants typically had
no method for verifying the accuracy or usefulness of their alt text, even though it may have been developed
iteratively over time as the gure was developed:
P9: “I literally can’t remember any discussion about what we would want in the alt text.”
Beyond accessibility for blind and low vision users, this leaves a gap with little consideration of the wider
disabled academic community or indeed beyond the academic community.
P6: “When people say accessibility, they mean accessible to blind readers, and, I’ll mention, the main thing that
leaves out is a constituency and area that I write for sometimes, and that’s people with cognitive limitations.”
P12: “I think that we often focus on visual disabilities, which I think is really good. I’m also like kind of interested
to see how you would make like all of our papers more accessible to a much wider audience beyond the academic
community, and also in our end communities, but, also people with disabilities in the academic community.”
5.2.3 People have abilities and disabilities. People with disabilities within the academic community face additional
challenges when writing papers and creating accessible content elements (see [15, 29, 67] for a wider reection
on academic challenges). For example, inaccessible tools can delay or stop progress and remove independence for
authors. In one case, a blind participant was able to create a diagram using an accessible tool, but was unable to
verify the output:
P5: “The output [from a javascript diagram library] is not accessible, it’s an image. But to create them, that’s a game
changer for me.”
This lived experience was prominent in our participants who self-disclosed a disability, with this aecting how
they approached the generation of visual content:
P11: “My questions about the design tend to be like utilitarian in terms of does it t the design of the paper, and does
it communicate the message that we’re hoping to communicate, but as far as like the lower-level intricacies of font
and spacing and style, I probably don’t know. No clue.”
In addition to being aware of this varied approach in light of their own disability, P11 also noted the need to
foster an awareness of the wider needs of co-authors or readers, and considering what accessibility might mean
for them.
P11: “Now I’m thinking a little bit more about like, how do I work with dierence? Instead of just saying this is what
I need, like, how do I think about, like, what do other people need as well?”
This awareness of others was reported more generally when participants reected on working with authors
with disabilities and the benets that interactions brings to them when creating accessible content.
P8: “As I started being around more researchers within accessibility, it really kind of helped establish empathy and
understanding why it’s important, and I think that’s a big motivator for doing it now.”
5.2.4 Senior academics have responsibility for creating an accessibility ethos. Reections of the participants show
a clear desire for an accessible ethos within research to drive change. We found that more established researchers
were condent to speak out about issues related to accessibility, even if they do not have much local support or
resources. This contrasts with the experience of early career researchers who lack condence to lead on matters
of accessibility and look to more senior colleagues to address systemic issues.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 17
P10: “I don’t think I’ve got the condence to do that ... I think that would be more for senior academics to raise [a
lack of accessibility review requirements] ... I wouldn’t stand up at a conference and say that you should consider
accessibility.”
More broadly, participants reected on who had responsibility for accessibility and the role of universities in
this infrastructure. This reects various levels of support and education for authors.
P4: I would really love for the supervisor, for the Department, and even for the University as a whole, to put accessibility
education in the curriculum.
5.3 The implementation of accessible content elements is heterogeneous
5.3.1 Implementation is based on individual skill and knowledge. We found that authors considered accessibility
at dierent stages of the writing process. Participant discussions around this topic highlighted focusing on
accessibility early: “Accessibility is front and centre from the beginning” (P7) and this could be motivated by the
desire to avoid challenges of implementation later on:
P5: “It doesn’t matter whether it’s a document or website to retrot. Accessibility is actually harder than baking it
into being.”
In other cases, our participants took a more iterative approach to including accessibility in the writing process
(i.e., accessibility is going to be revisited throughout):
P12: “We normally write the paper in LaTeX, so what we normally do is we—as we add an image into the paper—we
write an alt text caption for it, which kind of gets iterated as part of the process [...] but normally it’s done kind of as
we iterate the paper, so, if we change, perhaps the caption of the gure, we will update the alt text. Or, if we change
some wording, or whatever it is, we’ll keep the alt text kind of in the loop.”
However, accessibility is part of the “last step” (P4) in the process, and this could be “because it’s just complicated”
(P1) to do it any earlier than when making the nal PDF. Consideration of the type of content to include in a
paper would be prioritised.
Our participants explained how they would collate accessibility resources to supplement their writing process
with additional support. For example, bookmarking resources for future use (P4) and backing up of conference
writing guides such as SIGACCESS in the event the website becomes unavailable (P10), although the saved
resources may end up being infrequently used (P2).
We also found that the participants had dierent opinions on who is responsible for accessibility. One perspective
was that the lead author was responsible:
P13: “Usually, we make a rule that I think 2 hours or 3 hours before that deadline we make a nal accessibility check
so that we could make it everything ne, so that responsibility goes to the main author of the paper, so [the person]
who is leading the writing process.”
However, there are many factors making this a more complicated situation. Others on the project have a
responsibility to check, but after the author responsible for including that content in the paper (P8). A research
team may need to share responsibility, but this is not necessarily evenly distributed on account of authors having
varied skills and abilities. For example:
P9: “In my team its normally [name], he knows more about the technical aspects of accessibility.”
There was also additional pressure put on disabled authors in maintaining their integrity with advocating for
and producing accessible papers:
P11 “It’s not something I always do as well as I want, and that is, that’s really upsetting to me, particularly on the
collaborations when I’m not the leader and then I see something go out into the world and I’m like ‘that wasn’t
accessible like now that has my name on it you need to x that right now.’”
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
18 • R. Menzies, G. W. Tigwell and M. Crabb.
With deeper reections on who is responsible, some participants raised the question of whether responsibility
falls outside of the research team and onto universities or even the publisher. For example:
P6: “I left the consideration of things like alt text... I’m embarrassed to say that I haven’t checked to see how inclusive
or accessible the nished book is. I kind of left that to the publisher and gured that they would have some sort of
process for that, and I now think that maybe they didn’t.”
P07: “I would say rst the author themselves, and 2nd, do you wanna extend the University who is sponsoring this
author? Because the University, you know it’s the prestige of the University. So, the University, it’s on the, on their
best interests for the works to be accessible. I wonder the publishers. Ah, I would say yes. They are also responsible,
and as I was telling you earlier, it’s very disappointing to me to see that even if I send them an accessible PDF, they
butcher my accessibility features in the, in the nal published version. So I’d say the three of them are are responsible,
to dierent degree of course.”
There needs to be more support built into the whole publication process so that paper accessibility is also
taken on by the publisher and does not end with the authors.
5.3.2 Challenges of publication processes exacerbate issues. The publication pipeline can be a factor that introduces
challenges such as the requirements for the nal format of the paper. Since proprietary software (Adobe Acrobat)
is required to create accessible PDF documents, this can lead to inaccessibility and inequality, e.g. participants
noted that “the PDF editor from Adobe is expensive” (P3). Indeed, some participants were forced to share licenses
or download repeated free trials to access it, which can add stress to the nal stages of the publication process:
P10: “Our university doesn’t have a licence for Adobe... Every time that I’ve needed it, I’ve downloaded free trials
essentially on dierent laptops, which is a bit of a pain because I’m running out of laptops to download free trials
on... Even my supervisor got a discounted licence for it, he’s only got 2 licences and he’s got a group of 7 or 8 PhD
students. We have to switch the licence between us all, so, sometimes we don’t have access to that software.”
The PDF format itself was viewed as problematic, with repeated eorts needed ensure the accessibility of
content elements:
P7: “I am disheartened to see that we still rely on PDFs. And to see that even if I craft an accessible PDF, a lot of the
times when it goes to copy editing the alt-text or the images is not, is not there once the nal version makes it to the
to the ACM Digital Library. I’m happier now that there’s a bigger push for HTML versions.”
Although, P7’s comment positively viewed the publisher’s openness for alternative formats, there is still an
issue whereby the steps in the process from the publisher side is undoing authors’ accessibility work. This had
led to a lack in trust that the publishers will honour the accessibility eorts of the authors or even identify issues
(e.g., P1: “I think the last version didn’t have alt-text because I had sent it before and just forgot about it.”).
Many tools are used to support the paper writing process, but, unfortunately, those tools can result in
inaccessibility:
P5: “The facility to actually navigate charts again in raw Excel, you can get some information from a chart, but to
actually embed that in a Word document is not something I’ve had much success with. I’ve tried.”
In addition to issues within tools an author may chose to use themselves, part of the publication process can
dictate what tools need to be used when writing a paper:
P4: “It is much easier to create accessible document from HTML based document than from word because you don’t
know what’s happening in the automated process, but, with overleaf, it’s easier to control the process.”
5.4 There is a balance between accessibility and visual presentation of content elements
A key consideration shared by participants was the balance between including images to aid in comprehension
and how they are also avoided due to accessibility constraints. Participants described that for “empirical research
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 19
I tend not to use that many pictures because it is... such an accessibility nightmare” (P1), and the recognition that
“textual presentation of ideas is more inclusive” (P6). Participants also noted situations where they would “keep the
images to the minimum because of issues with screen readers” (P7) and that there are times where they “can’t see
how to do proper captions” (P1) with the result being the removal of content elements, despite their inherent value.
One of the main constraints of this process is related to “the hours it takes to actually check through [element
accessibility]” (P9). Participants also discussed the time constraints present in learning how to make use of tools
and techniques that would assist in making accessible content elements:
P11: “It takes a long time to develop knowledge that’s beyond checklists.”
As an example, learning to eectively use screen reader software was discussed by participants to be a key skill
in ensuring accessibility, however some participants noted that “I just have never had the time to learn it” (P12).
A further consideration on the inclusion of content elements within academic work is related to space
constraints, which is something we observed when reviewing the collection of ASSETS papers. When publications
have a strict page limit this impacts on the number of visuals that authors can include. Participants discussed this
aspect and while they may have “wanted to include more” (P2), this may not be possible as content elements may
take up “too much space and conveys too few, too little information” (P4), for example:
P12: “In the old days of the old templates where they had to be a nite size, I found myself kind of squishing stu
onto a page to simply meet the page limit.”
Conversely, non-textual elements can also be used to save space within a paper where “sharing that through
text form like through a paragraph or something is really verbose” (P8). Our participants mentioned that aspects
such as participant background tables are “the most concise and visual way to put together [participant data]” (P4).
Our participants commented that images inside of academic work “takes up a lot of space” (P9) and that there is
often a compromise required when there is a page limit. This was particularly true for qualitative work, meaning
that some contextual information presented in the content elements cannot be provided in the most appropriate
or accessible way.
P10: I had to severely reduce the image size just because I would have been over the page by a couple of lines.
Participants commented that when including content elements in academic work it is important to create
a direct link between the elements that are included and the main body of the text. It is important to ensure
that there is a “a relationship between the rst mention of the image and the image itself” (P7) and to also “add
explanatory text in the surrounding paragraphs” (P5) in order to give context to images.
When designing the elements themselves, participants discussed multiple reasons for design choices, with
factors such as colour schemes being used because they are “the default for Excel” (P8), elements being “visible in
grayscale” (P7), and also in giving “a nod” (P1) to methodological roots within colour choice. Participants also
discussed imaging editing techniques, such as “removing backgrounds” (P12), “image scaling” (P10), and making
sure an “image is grayscale” (P7) all being used.
5.4.1 Current guidelines can be barriers. Many guidelines and processes exist that have the end goal of improving
accessibility within the academic publication process. Our participants showed an awareness of these and
discussed versions such as the “ACM guidelines on how to write accessible papers” (P7), WC3 Web Guidelines
as “the go to” (P8) reference, and venue specic guidance for “venues like CHI and ASSETS” (P11). However,
participants had varied opinions about their expectations of accessibility guidance. P2 commented that they “have
lots of questions [but]...no answers like how to do that”, with this sentiment echoed by others:
P4: “I can’t really comment on the CHI guideline because I don’t know what a perfectly accessible paper would look
like at that time. So, if I, if there is a good example that maybe we could see the discrepancy, but then again [it] goes
back to the general awareness and education.”
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
20 • R. Menzies, G. W. Tigwell and M. Crabb.
P7: “I feel like they could give practical tips with screenshots, and giving you step by step instructions on how to do
this, instead of just giving you like guidelines.”
Overall, “more systematic best practice” (P4) is required and “video resources on how to actually do these things”
(P7), may be benecial.
Our participants also commented that there are a variety of tools that assist in improving document accessibility
that are available as part of the writing process with this ranging from the “accessibility validation of Microsoft
Oce” (P3), to “Adobe Acrobat to make a PDF accessible” (P7). However, participants voiced heavy concerns about
the tools themselves as “tagging a PDF is actually an inaccessible process as a blind person” (P5). When accessibility
features are stripped out of a PDF as part of the publication pipeline, participants discussed that they “can’t go
and remedy that because the tools are not accessible” (P11). Participants also commented that the use of tools
themselves requires skill and that sometimes it can be challenging to determine if a process has been successful:
P12: “When I upload a paper, it’s kind of made accessible by me or by my colleagues, but I’m never convinced that a
person with a screen reader can access it because I’ve been shown in Adobe in a really visual way, with all the little
blue boxes, and all the reading order stu, but I’m not fully convinced that when it ends up on somebody’s screen
reader they’re actually going to be reading it the way I want them to.”
6 DISCUSSION
RQ: What challenges exist when implementing accessible content within academic writing?
Creating accessible publications is central in enabling access to scientic work for all people. However, the
complexity required to illustrate certain aspects of our work creates challenging accessibility problems that
authors must understand. Our interviews highlighted that authors carry out a balancing act between presenting
information visually and making sure that this information is accessible. We also highlighted that creating
accessible content elements is a multi-faceted issue that involves considering all accessibility domains. Finally,
our participants discussed the challenges they have faced in implementing accessible gures, describing the
specic skills and guidance required on many granular levels. We discuss these points below.
6.1 Balancing Accessibility and Visual Design
Authors consider the inclusion of graphical content as a core part of the academic paper writing process. Our
initial quantication of content elements coded 3866 content elements within 330 papers. Images were the most
common type of element that was used. Implementing accessibility within paper writing does not necessarily
mean that no graphical content is included, and our interviews were designed to understand the process that
authors go through when creating papers and making content accessible.
Our participants described that content elements are used within their work in order to provide a broader
understanding of their work, but that they can also face challenges in presenting this in an accessible way, which
supports the observations we made of the many dierent inaccessible content elements. Contemporary guidance
on creating charts suggest that they should tell a specic story and assist in leading the reader to an intended
understanding [
77
]. As a contrast, alternative text for charts remove this step of reader-led understanding and
presents raw data in a form that is challenging to draw conclusions from [
30
]. In short, the visual representation
of gures is often dierent from its accessible description.
A further challenge exists in interpreting the formatting guidelines that are attached to presenting work at
academic venues. It is still common to see xed page limits for conference venues, and for paper style guides
to be viewed as compulsory (example style guides packaged as templates can be accessed at www.acm.org/
publications/proceedings-template). Style guides can create friction between creating a nal paper that is visually
pleasant and functionally accessible, with a preference directed towards the former.
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.
Author Reflections on Creating Accessible Academic Papers • 21
Participants described that content elements are mostly used as supplemental content in their work and can
be used to present existing information in an alternative format. The decisions made by participants in how to
include content elements within their paper are varied, and are seen elsewhere in the literature [
54
], e.g. personal
preferences and time pressures played a key role. Ensuring that the PDF is accessible is generally thought of
as a last step in the paper writing process and is considered very close to paper submission. This approach is
very similar to that of waterfall software development life cycle [
62
], with accessibility seen as a nal process
that must be implemented without altering the visual presentation of the work. We suggest that an iterative
and incremental approach [36] to accessibility throughout the paper writing process would be benecial and is
something that should be embraced and more readily facilitated.
6.2 Accessibility is Multi-Faceted
Accessibility is often viewed as a niche research domain [60], but it is one that contains a complex mix of areas
that can intersect with each other. The resultant complexity makes it challenging to develop broad expertise in
how to create fully accessible artefacts. Common techniques that are used to develop an awareness of accessibility
challenges involve education, empathising with others, and personal experience [
14
]. These methods, whilst
benecial, individually require experience in order for them to be of value.
Our participants commented that having general accessibility knowledge when creating academic papers is
important for all members of the research team. They added that it is also important for team members to have
more specic areas of expertise with this being related to either individual accessibility areas or individual aspects
of papers. Our participants gave examples of team members splitting up responsibility for the implementation of
accessibility for dierent parts of their publications based on team member experience, expertise in a given area,
and also as a method to provide training to new members.
In carrying out this work, we acknowledge that the challenges that our participants highlighted when creating
accessible academic documents is very similar in nature to the challenges that are faced by the software develop-
ment community as a whole [
14
]. While it is unsurprising that this may be the case, it highlights a systemic issue
in accessibility training within the very broad computing domain.
6.3 Creating Accessible Content
It is not enough for academics to understand the specic accessibility challenges that can arise when creating
publishable material. There is also a need to understand how these challenges can be overcome and what
techniques and skills are required in order to achieve this. Common methods that are used include altering (e.g.
colour palette changes), and adding content (e.g. alt-text, descriptive labels).
Our participants commented on the technical challenges that are associated with creating accessible elements
within academic work. They discussed that available guidance for creating accessible elements can be challenging
as advice is often oered at a very high level. Our participants’ overall opinions on implementing this guidance
was mixed. Participants commented that implementing screen reader accessibility can be simple to achieve, but
the simplicity of this process is directly linked to the overall complexity of the paper that is being created.
We want to reiterate that our current work is focusing more broadly on the visual accessibility of content
elements that appear in a paper and the overall paper pipeline from initial project idea to publication, rather
than on PDF accessibility for screen readers, which prior work has demonstrated is a signicant issue [
7
,
9
,
79
].
However, it is worth reecting on the tools that are used to write academic papers and the eects this can have
on the nal PDF accessibility, since there are parallels with our ndings for the challenges of creating accessible
content for paper in terms of resources available. It is generally accepted that using Microsoft Word is a better
starting point than using LaTeX if the intended outcome is to produce an accessible PDF. Microsoft Word and
LaTeX both support necessary features for screen reader users such as a dened heading structures, but Microsoft
ACM Trans. Access. Comput., Vol. 1, No. 1, Article . Publication date: July 2022.