Article

Readability formulas have even more limitations than Klare discusses

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

A literature review reveals many technical weaknesses of readability formulas (when compared to direct usability testing with typical readers): they were developed for children s school books, not adult technical documentation;they ignore between-reader differences and the effects of content, layout, and retrieval aids on text usefulness; they emphasize countable features at the expense of more subtle contributors to text comprehension.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... The studies in [15], [17], and [19] are our motivation towards having additional evaluation attributes other than the ones used in traditional formulas. The following three paragraphs put a light to these papers. ...
... There are certain limitations concerned with the readability formulas like the grade-level formulas were designed for school books only, and no formula was designed for technical materials. Another major problem with readability formulas is that they consider all readers to be alike [15]. ...
... The function would say first one "short" and the second one "Tall" just because of 1 inch. [15] [15] However, the applications of fuzzy sets to this scenario make a huge real-time applicable difference. This way the approach of using fuzzy sets for tall men can leverage the functionalities of basic sets for an improved representation of the tallness of a person. ...
... This formula is similar to the Flesch reading formula in a sense that it uses two variables, average sentence length and a percentage of difficult words. However, it was designed to overcome some flaws of the FRE formula (i.e., affix counts and personal references contained within an earlier version of the FRE metric) by assessing word complexity based on a large list of frequent words (n = 3000), at least 80 percent of which are familiar to fourth-grade students [15], [17]- [19], [26]. ...
... Second, neither version of the Dale-Chall formula was created for measuring technical materials so it is questionable whether this formula gives an accurate score for technical and legal documents. In addition,, the developers of the Dale-Chall metric did not aim to provide any theoretical insight into readability [17], [26]. ...
... Our results across the traditional readability measures revealed that none of the selected traditional formulae give identical scores nor match human rating. The differences in rating the same piece of text are not surprising given previous studies [20], [26]. The aforementioned comparison between the Strathclyde readability results and human comprehension test results revealed that the majority of texts" results were matched in human ranking and SRM. ...
Conference Paper
Full-text available
Researchers have designed a number of software readability metrics that evaluate how difficult a passage is to comprehend; yet, little is known about the impact of readability on the interpretation of information security policies (ISPs) and whether experiment of readability may prove to be a useful factor. This paper examines and compares eight ISP documents on nine mechanical readability formula results with outcomes from a human-based comprehension test. The primary focus is to identify if we might rely on a software readability measure for assessing the difficulty of a text document in the domain of Information Security Policies. Our results reveal that traditional readability metrics are ineffective in predicting the human estimation. Nevertheless, readability, as measured using a bespoke readability metric, may yield useful insight upon the likely difficulty that end-users face in comprehending an ISP document. Thereby, our study aims to provide a means to enhance the comprehensibility of ISPs.
... Though there exist other factors that influence the course of action, apart from readability, the level of academic performance as well as student retention capabilities goes down as textbooks become difficult to read. According to Redish, (2000) and Rush, (1985) readability formulae may not necessarily define the actual readability and therefore, should not be used unguardedly because actual readability is not just a simple function that can be objectively used to measure properties like a word or the length of a sentence. The length and grammatical complexities of a sentence (syntax) and the level of difficulty of words, measured in several syllables (semantics), in most cases, are used in calculating the level of indices of readability. ...
... The length and grammatical complexities of a sentence (syntax) and the level of difficulty of words, measured in several syllables (semantics), in most cases, are used in calculating the level of indices of readability. However, the calculated indices of readability take no explanation of the various text aspects as well as the characteristics of the reader like skill, level of motivation, and experience (Redish, 2000;Rush, 1985;Bargate 2012). ...
... It is essential to note that given limitations are mainly linked with readability indexes. Redish, (2000) Klare, (1976, pp. 136-151), andDuBay, (2004) point out that the readers' motivation and familiarity with a number of vocabularies are not determined by readability formulas. ...
Article
Full-text available
In recent years, students in the high schools in Ghana continue to perform poorly in English Language examinations, especially for students at the West African Senior School Certificate Examination (WASSCE). Various stakeholders in the education sector continue to deliberate to find answers to this unfortunate development. Some scholars have suggested that students' failure in the English Language examination is as a result of the readability and comprehensibility levels of the comprehension passages presented to students in their examinations. English language examination questions that cannot be read easily cannot be well understood. Determining the level of readability is of great importance if comprehension is aimed at. The authors made use of two Readability Indexes which include the Flesch-Kincaid Reading Ease and Gunning Fog Scale to carry out document analysis of the reading ease of seven comprehension passages presented to candidates from 2014 to 2020 in the WASSCE, to carry out the readability analysis and how that may have affected their Comprehensibility. The key findings from this study indicate that the readability of the comprehension passages analyzed between 2014 and 2020 was either very difficult or difficult for the students to comprehend. In conclusion, the researchers believe that the high level of readability of the comprehension passages has partly contributed to the low pass rate of the student in the English language at the WASSCE. Keywords: Readability, comprehensibility, students, stakeholders, Ghana
... Early on, the American plain language movement was associated with readability formulas: most notably the Flesch-Kincaid formula (still available in Microsoft Word), the similar Dale-Chall formula, and the Gunning fog index (Klare, 1963). But since the late 1970s, plain language advocates have adopted usability testing and emphasized design considerations beyond words and sentences: that is, information, document, and visual design (Redish, 2000;Schriver, 2017). We enumerate the main limitations of existing readability formulas below. ...
... Most of all, for modern proponents of plain writing, readability formulas fail to take into account all of the non-prose elements of websites and other documents. These include the organization of information, the use of headings, tables of contents, layout and formatting, visuals, and so on (Redish, 2000;Redish and Selzer, 1985). Therefore, although readability measures have in the past been used to evaluate the accessibility of government documents, such measures are not mentioned in either the Plain Writing Act of 2010 or the SEC's Plain English Rule of 1998. ...
... The methods are also vulnerable to gaming (Redish, 2000). For example, Flesch-Kincaid readability scores can be raised by replacing lengthy words with acronyms. ...
... Readability formulae are mathematical equations whose results provide interpretations for the description of a text from 'very easy' to 'very difficult'. They are meant to help predict or indicate reading ability (in terms of school grade level) required to read and comprehend a particular written text (Redish 2000;DuBay 2004). The formulae are based on research findings that identify vocabulary difficulty (which is measured in word familiarity or difficulty, or word length) and sentence structure (measured in sentence length) as two reliable variables in the study of text complexity and difficulty (DuBay 2004;Hulden 2004). ...
... The notional complexity of the text was corroborated by results of the readability formula analysis, which indicate that on average, 85% of the texts of the various newspapers were difficult and readable by readers with university educational attainment. Although readability formulae have been criticised, especially in their use to guide the writing and revision of documents (Redish and Selzer 1985;Redish 2000), they remain reliable tools in determining levels of language complexity (DuBay 2004). Their relevance in this instance is the fact that this study was not meant to revise or guide the writing of newspaper stories but just to indicate readability levels. ...
Research
Full-text available
The Ghana Journal of Linguistics is a double-blind peer-reviewed scholarly journal appearing twice a year, published by the Linguistics Association of Ghana.
... Over time, these formulae have been criticized on several counts. These criticisms included their unsuitability for technical writing analysis (Redish (2000)), their inability to accurately reflect a user's comprehension of the content (Fuchs et al. (1983)) and the inherent drawback of relying on solely quantitative measures to gauge readability (Redish (2000)). However, naïve as they are, they are still widely used for gauging text readability as stated by Guo et al. (2011). ...
... Over time, these formulae have been criticized on several counts. These criticisms included their unsuitability for technical writing analysis (Redish (2000)), their inability to accurately reflect a user's comprehension of the content (Fuchs et al. (1983)) and the inherent drawback of relying on solely quantitative measures to gauge readability (Redish (2000)). However, naïve as they are, they are still widely used for gauging text readability as stated by Guo et al. (2011). ...
Preprint
Full-text available
Ensuring that the readability of school textbooks is appropriate for the prescribed grade is the cornerstone of any successful education system. With globalization spreading like wildfire, it is of the utmost importance that readability standards should be upheld across the world to ensure the seamless progress of education. Thus I decided to investigate the research question "Is there a difference in the readability of textbooks, written in the English language, in L1 and L2 Anglophone countries?" The USA was chosen as the L1 Anglophone country, and India was chosen as the L2 Anglophone country. The textbooks for different subjects were chosen from Grade 1 to Grade 10 thereby encompassing the full range of the school education system. Various readability metrics such as the Flesch Reading Ease score, the Gunning-Fog Index, Lexical and Content Diversity were then used for comparing textbook readability. Textbooks for both countries displayed a gradual decrease in reading ease, i.e. an increase in complexity with each increasing grade. Considering the fact that it was expected for the Indian textbooks to be more readable, in general, than their US counterparts(given that they are intended for non-native speakers); the readability indices showed similar characteristics for all the metrics. This similarity in behaviour bears out the truth of the hypothesis that the readability level of textbooks in L1 English speaking countries should be comparable to textbooks written in the English language in L2 Anglophone countries for a particular age group.
... Readability formulas have numerous limitations such as considering jargon, concept difficulty and density, logic, coherence and any type of formatting or visual support for the reader. Nevertheless, the formulas convince through their automation capability, simplicity, and independence from the readers (Bayerlein & Davidson, 2011;Bruce et al., 1981;Crossley et al., 2019;Redish, 2000). We believe that for readability studies, the absolute result of one single text is of less interest than the influence of time or other factors on the score of a bigger sample of texts (Thoms et al., 2020). ...
... Readability formulas also form the primary tool of readability research of written company disclosures (Adelberg, 1979;Courtis, 2004;Jones, 1988;Li, 2008;Rutherford, 2003;Smith & Taffler, 1992;Subramanian et al., 1993;Thoms et al., 2020). Alternative procedures for assessing readability include usability tests with representative readers (Redish, 2000), asking readers to assess text where every nth word is deleted 1 (Smith & Taffler, 1992;Taylor, 1953) and other, more modern and multi-dimensional computational linguistics analysis tools such as the Coh-Metrix (Chang & Stone, 2019;Crossley et al., 2019). They all have in common that they are more costly to process. ...
Article
Full-text available
This research examines the relation between annual report readability and company performance in a German-speaking country, Austria. The incomplete revelation hypothesis, management obfuscation hypothesis and agency theory assume that firms with lower performance strategically use readability in their disclosures to obfuscate negative results. For investors, reading, analysing, and interpreting data becomes a costly affair; this weakens the negative effect of such data on a firm's reputation and share price. We use LIX and Flesch formulas to measure the readability of letters to the shareholders and/or interviews with the board in annual reports. The sample consists of 37 companies that are listed on the Prime Market of the Vienna Stock Exchange and their data from the year 2009 to 2020. Company performance is measured by the change in turnover, profit, and share price. The analysed sections mostly show high to very high levels of difficulty. During the observation period, readability levels do not change significantly. We find that the annual reports of firms with lower performance are not harder to read and, therefore, cannot confirm the management obfuscation hypothesis. A significant influence of change in profit/loss on readability is minutely observed. Possible reasons for this observation could be characteristics of the German language, statistical outliers, the long observation period, more professional investor relations offices, and changing communication channels between companies and stakeholders. The last point, changing communication channels, also puts the obfuscation hypothesis and its application to readability up for discussion again. Implications for Central European audience: Our study shows that also Central European countries are confronted with low levels of readability in annual reports. Nevertheless, we cannot see a clear tendency towards obfuscation in corporate disclosures.
... Various measures can be used to determine readability levels, including the Flesch Reading Ease Scale and the Gunning Fog Index. The Flesch Reading Ease Scale awards a score to a Bereitgestellt von | De Gruyter / TCS Angemeldet Heruntergeladen am | 09.11.18 14:54 document using a formula based on the number of words, number of syllables per word, and number of sentences (Redish 2000). Documents with high Flesch scores are considered easier to read; documents with low scores are considered more difficult. ...
... There has been considerable debate about the usefulness of readability formulae as a document is not automatically readable if it contains few words, each with a small number of syllables; other factors such as writing style and relevance to the target audience must also be taken into account. See Redish (2000) for a discussion of some limitations of readability formulae. ...
Chapter
Although for centuries people have been writing instructions to explain technological advancements, technical communication only developed as a recognised occupational field in the twentieth century. The expansion of software industries in the 1980s, driven by the exponential increase in business computing and home computer ownership, led to a need for people who could explain technology to lay users. Thus, technical communication became a more recognised occupation. This chapter outlines the features and functions of contemporary technical communication and technical documentation contexts. Although technical communication is a field characterised by diversity, and therefore evades easy definition, our chapter delineates central concepts. In addition to explaining documentation types and writing processes, we also outline traditional skillsets and competencies, including writing, information design, and interviewing. We then explore evolving competencies of technical communicators, such as structured authoring, usability, and knowledge of web 2.0 technologies. The chapter concludes that, despite constant technological shifts, the ability to communicate clearly is central to this field.
... Readability formulas do not allow any real judgment about how comprehensible the meaning of a text really is. They do not provide any indication of prior knowledge required or coherence of a text and cannot judge mood or tone (Redish, 2000). The wide variation of readability measures is the reason that several measures were examined in this study for the purpose of robustness. ...
Preprint
Full-text available
This study examines how the readability of scientific discourses changes over time and to what extent readability can explain scientific impact in terms of citation counts. The basis is a representative dataset of 135,502 abstracts from academic research papers pertaining to twelve technologies of different maturity. Using three different measures of readability, it is found that the language of the abstracts has become more complex over time. Across all technologies, less easily readable texts are more likely to receive at least one citation, while the effects are most pronounced for comparatively immature research streams. Among the more mature or larger discourses, the abstracts of the top 10% and 1% of the most often cited articles are significantly less readable. It remains open to what extent readability actually influences future citations and how much of the relationship is causal. If readability indeed drives citations, the results imply that scientists have an incentive to (artificially) reduce the readability of their abstracts in order to signal quality and competence to readers-both to get noticed at all and to attract more citations. This may mean a prisoner dilemma in academic (abstract) writing, where authors intentionally but unnecessarily complicate the way in which they communicate their work.
... However Klare [1] revealed that there are many technical weaknesses in readability formula. As these formulas are basically made for children's academic books and it ignores the reader difference, type of text, its effectiveness. ...
Conference Paper
Full-text available
Web Readability plays an important role in web usability design. Readability means to identify, perceive and understand any text or data. Poor readability scares readers away from the content. There are a number of factors affecting web readability like hierarchy, contrast, line height, line length, consistency, typeface, size, understanding of vocabulary, plain text etc. Web readability is one of the main issues in lower-literate people, a product designer or company who want to do their business online must not neglect this mass portion of society. Different accessibility and legibility guidelines have many factors that can play a role in determining the best conditions for legibility. Some web-based readability challenges do not take into consideration while designing a web-based application which has a great role in usability. The aim of this study is to identify those readability challenges which are often ignored, to evaluate different aspects of readability for lower-literate people and the choice of best readability guidelines for these caders of web users with respect of (WCAG 2.0, IBM Web 5.1 accessibility checklist, Section 508 §1194.22).
... For example, Smith and Smith (1971) In spite of its popularity, Tefki (1987:270) reports that some studies have found that the Dale-Chall formula tends to rate specialized materials as more difficult than they are based on the percentage of words not on the list that might be common words in certain fields. Redish (2000) argues that this flaw has led some researchers to adapt the list by arbitrarily adding words supposedly known to their audiences, which may alter the final score. Redish also points out that the word list does not account for polysemy, since it is not possible to know which meaning of polysemic words is on the list, and that the vocabulary knowledge of the 4 th -graders from many generations ago upon which the list was built probably differs from that of present-day 4 thgraders (133-134). ...
Thesis
Full-text available
Readability analysis through the use of standardized formulas has been largely applied in different fields since the first widely-accepted formulas were published in the mid-20th century. More recently new models based on corpus linguistics and natural language processing have tested different deep-level text features in order to find new systems that are more reliable and suitable for different text formats and genres. Additionally, the European Union’s amendment of its general data protection regulation requires privacy policies to be drafted in plain language. This paper aims to measure the readability of the privacy policy notices of some of the most popular social networking websites. Five traditional readability formulas, along with Lexile text scores and four deep-level text features were computed for 19 privacy policy notices of social networking websites and applications. Results suggest that the privacy policies notices in our corpus are difficult to read for most users. Key words: readability, privacy policy, social networks, readability formulas, GDPR.
... Communicating in the 21st Century 3.8 constituent sentences or clauses, even though a readability score of the linked sentences would suggest that a reader would have less, rather than more, understanding (see also Graesser et al. 2004;Meyer 2003). ■ ■ Redish (2000) argues that readability scores do not take account of the actual behaviour of readers, and that usability testing (see chapter 2 -the actual observation of readers using texts, and then testing the readers for what actually helped or hindered understanding -is far superior. She links this to the history of readability testing and scoring, pointing out anomalies: ...
Chapter
Full-text available
... We calculate the readability scores to assess the quality of the comments, a notion first introduced by Klare (Klare and others 1963). However, the validity of the original readability score formulation by Klare is controversial (Redish 2000). Therefore, in this paper, we use two other readability scores to look into the comments from different angles. ...
Preprint
Full-text available
Crowdwork often entails tackling cognitively-demanding and time-consuming tasks. Crowdsourcing can be used for complex annotation tasks, from medical imaging to geospatial data, and such data powers sensitive applications, such as health diagnostics or autonomous driving. However, the existence and prevalence of underperforming crowdworkers is well-recognized, and can pose a threat to the validity of crowdsourcing. In this study, we propose the use of a computational framework to identify clusters of underperforming workers using clickstream trajectories. We focus on crowdsourced geopolitical forecasting. The framework can reveal different types of underperformers, such as workers with forecasts whose accuracy is far from the consensus of the crowd, those who provide low-quality explanations for their forecasts, and those who simply copy-paste their forecasts from other users. Our study suggests that clickstream clustering and analysis are fundamental tools to diagnose the performance of crowdworkers in platforms leveraging the wisdom of crowds.
... By the turn of the century, various researchers, eg Redish (2000) and Shriver (2000), were ...
Book
This book is an endeavour to probe into the areas of Hindi syntax which have been rather under-explored in generative literature. It investigates the syntax and semantics of Hindi verbs and their argument structure alternations within the minimalist framework. In the course of this exploration it examines unaccusativity, unergativity, transitive, causative alternations and passives in Hindi. The book will be of interest to theoretical linguists and computational linguists, as well as to Hindi syntax specialists.
... The conclusions of this study should be considered in the context of its limitations. Readability formulas have been challenged in the literature as valid measures of reading difficulty (Chambers, 1983;Fuchs et al., 1982;Redish, 2000;Stokes, 1978;Sydes & Hartley, 1997;Templeton et al., 1981). The main argument against their use is that most formulas, including the ones used in this study, presume that sentences with longer words in terms of syllables and characters are harder to read and comprehend. ...
Article
Adaptive behavior rating scales are widely used assessments that ask teachers and parents to rate how well students care for themselves, interact with others, and exhibit practical and academic skills. While adaptive rating scales have yielded reliable and valid results, it is unclear how the readability of the rating forms may impact the results, especially for parents who may have lower literacy levels. Using three readability indices—the Flesch Reading Ease for English, the Fernández Huerta for Spanish, and the Automated Reading Index for both English and Spanish—the readability of the English and Spanish parent/caregiver forms of the Vineland‐3 and ABAS‐3 was assessed. Results indicated that the reading grade level equivalent of the English and Spanish versions of the ABAS‐3 was on average greater than the recommended sixth grade level. In contrast, the readability of the Spanish and English versions of the Vineland‐3 was on average sixth grade level and below. Based on these results, when using parent/caregiver adaptive behavior rating forms, school psychologists should take precautions to ensure valid results by ascertaining parent/caregiver reading proficiency, providing additional explanations as permitted by the assessment protocols, and using alternative interview forms provided by publishers as needed.
... Okunabilirliğin hesaplanmasında kullanılan diğer yöntem olan okunabilirlik formülü, belirli bir düz yazı parçasını anlamak için gereken okuma yeteneği seviyesini tahmin etmek için tasarlanmış matematiksel bir denklemdir (Redish, 2000). Okunabilirliğin sayısal verilere dayandırılarak ölçülmesi ile ilgili ilk çalışmalar arasında Thorndike tarafından 1921 yılında yayımlanan The Teacher's Word Book kitabı göze çarpmaktadır. ...
... This study has several limitations. First, our data collection focused on information available on government websites only, and it is possible that information was missed if it was distributed through alternative channels including social media or through traveller registration apps (48)(49)(50). Second, the SMOG and Flesch Kincaid reading ease tools used in this study were not speci cally designed to assess online information as has been discussed previously, however they are the most frequently used tools in assessments of online information readability due to a lack of validated alternatives (11,22,26,48). ...
Preprint
Full-text available
Background: In response to the continuing threat of COVID-19, many countries have implemented some form of border restriction. A repercussion of these restrictions has been that some travellers have been stranded abroad unable to return to their country of residence, and in need for government support. Our analysis explores the COVID-19-related information and support options provided by 11 countries to their citizens stranded overseas due to travel restrictions. We also examined the quality (i.e., readability, accessibility, and useability) of the information that was available from selected governments’ web-based resources. Methods: Between June 18 to June 30, 2021, COVID-19-related webpages from 11 countries (Australia, New Zealand, Fiji, Canada, United States of America (USA), United Kingdom (UK), France, Spain, Japan, Singapore, and Thailand) were reviewed and content relating to information and support for citizens stuck overseas analysed. Government assistance-related data from each webpage was extracted and coded for the following themes: travel arrangements, health and wellbeing, finance and accommodation, information needs, and sources. Readability was examined using the Simplified Measure of Gobbledygook (SMOG) and the Flesch Kincaid readability tests; content ‘accessibility’ was measured using the Web Content Accessibility Guidelines (WCAG) Version 2.1; and content ‘usability’ assessed using the usability heuristics for website design tool. Results: Ninety-eight webpages from 34 websites were evaluated. No country assessed covered all themes analysed. Most provided information and some level of support regarding repatriation options; border control and re-entry measures; medical assistance; and traveller registration. Only three countries provided information or support for emergency housing while abroad, and six provided some form of mental health support for their citizens. Our analysis of the quality of COVID-19-related information available on a subset of four countries’ websites found poor readability and multiple accessibility and usability issues. Conclusion: With large variance in the information and services available across the countries analysed, our results highlight gaps, inconsistencies, and potential inequities in support available, and raise issues pertinent to the quality, accessibility, and usability of information. This study will assist policymakers plan and communicate comprehensive support packages for citizens stuck abroad due to the COVID-19 situation and design future efforts to prepare for global public health emergencies.
... The most commonly used were the Simple Measure of Gobbledygook grading [27], Gunning Fog index [26], and Flesch-Kincaid grade level [20,28]. It is important to use a readability tool only to determine the reading level of the text and not for assessing the overall suitability of online material, because readability tools fail to take into account comprehension and the role of the reader [44,45]. Regarding the writing style and the sentence construction, information on the website should be conveyed in an active voice in small sentences and complex information should be broken down. ...
Article
Full-text available
Background Online health information, if evidence-based and unbiased, can improve patients’ and caregivers’ health knowledge and assist them in disease management and health care decision-making. Objective To identify standards for the development of health information resources on the internet for patients. Methods We searched in MEDLINE, CINAHL, Scopus, Web of Science, and Google Scholar for publications describing evaluation instruments for websites providing health information. Eligible instruments were identified by three independent reviewers and disagreements resolved by consensus. Items reported were extracted and categorized into seven domains (accuracy, completeness and comprehensiveness, technical elements, design and aesthetics, usability, accessibility, and readability) that were previously thought to be a minimum requirement for websites. Results One hundred eleven articles met inclusion criteria, reporting 92 evaluation instruments (1609 items). We found 74 unique items that we grouped into the seven domains. For the accuracy domain, one item evaluated information provided in concordance with current guidelines. For completeness and comprehensiveness, 18 items described the disease with respect to various topics such as etiology or therapy, among others. For technical elements, 27 items evaluated disclosure of authorship, sponsorship, affiliation, editorial process, feedback process, privacy, and data protection. For design and aesthetics, 10 items evaluated consistent layout and relevant graphics and images. For usability, 10 items evaluated ease of navigation and functionality of internal search engines. For accessibility, five items evaluated the availability of websites to people with audiovisual disabilities. For readability, three items evaluated conversational writing style and use of a readability tool to determine the reading level of the text. Conclusion We identified standards for the development of online patient health information. This proposed instrument can serve as a guideline to develop and improve how health information is presented on the internet.
... This study has several limitations. First, our data collection focused on information available on government websites only, and it is possible that information was missed if it was distributed through alternative channels including social media or through traveller registration apps [49][50][51]. Second, the SMOG and Flesch Kincaid reading ease tools used in this study were not specifically designed to assess online information as has been discussed in other research, however they are the most frequently used tools in assessments of online information readability due to a lack of validated alternatives [12,23,27,49]. ...
Article
Full-text available
Background In response to the continuing threat of importing novel coronavirus disease (COVID-19), many countries have implemented some form of border restriction. A repercussion of these restrictions has been that some travellers have found themselves stranded abroad unable to return to their country of residence, and in need for government support. Our analysis explores the COVID-19-related information and support options provided by 11 countries to their citizens stranded overseas due to travel restrictions. We also examined the quality (i.e., readability, accessibility, and useability) of the information that was available from selected governments’ web-based resources. Methods Between June 18 to June 30, 2021, COVID-19-related webpages from 11 countries (Australia, New Zealand, Fiji, Canada, United States of America (USA), United Kingdom (UK), France, Spain, Japan, Singapore, and Thailand) were reviewed and content relating to information and support for citizens stuck overseas analysed. Government assistance-related data from each webpage was extracted and coded for the following themes: travel arrangements, health and wellbeing, finance and accommodation, information needs, and sources. Readability was examined using the Simplified Measure of Gobbledygook (SMOG) and the Flesch Kincaid readability tests; content ‘accessibility’ was measured using the Web Content Accessibility Guidelines (WCAG) Version 2.1; and content ‘usability’ assessed using the usability heuristics for website design tool. Results Ninety-eight webpages from 34 websites were evaluated. No country assessed covered all themes analysed. Most provided information and some level of support regarding repatriation options; border control and re-entry measures; medical assistance; and traveller registration. Only three countries provided information or support for emergency housing while abroad, and six provided some form of mental health support for their citizens. Our analysis of the quality of COVID-19-related information available on a subset of four countries’ websites found poor readability and multiple accessibility and usability issues. Conclusion This study uniquely analyses government support for citizens stuck abroad during the COVID-19 pandemic. With large variance in the information and services available across the countries analysed, our results highlight gaps, inconsistencies, and potential inequities in support available, and raise issues pertinent to the quality, accessibility, and usability of information. This study will assist policymakers plan and communicate comprehensive support packages for citizens stuck abroad due to the COVID-19 situation and design future efforts to prepare for global public health emergencies.
... Traditional or surface-based features are predictors that were used by experts for their old readability formulas for Filipino such as sentence and word counts in Guevarra (2011). Despite the claims that these features insufficiently measures deeper text properties for readability assessment (Redish, 2000), since this is the pioneering study for Cebuano, we still considered these features for our baseline model development. In this study, we adapted the seven features of traditional features from existing works in Filipino (Imperial and Ong, 2020, 2021a,b) such as number of unique words, number of words, average word length, average number of syllables, total number of sentences, average sentence length and number of polysyllable words. ...
... In this work, we considered measuring readability as a ranking task, where the relative difficulty of documents is compared. Readability in the health domain is often measured with formulas developed to ensure that school textbooks are appropriate for children at a particular school grade level [16]. However, obtaining a grade level often is not the ultimate goal. ...
Article
Full-text available
Background: The use of electronic health record (EHR) systems with patient engagement capabilities, including viewing, downloading, and transmitting health information, has recently grown tremendously. However, using these resources to engage patients in managing their own health remains challenging due to the complex and technical nature of the EHR narratives. Objective: Our objective was to develop a machine learning-based system to assess readability levels of complex documents such as EHR notes. Methods: We collected difficulty ratings of EHR notes and Wikipedia articles using crowdsourcing from 90 readers. We built a supervised model to assess readability based on relative orders of text difficulty using both surface text features and word embeddings. We evaluated system performance using the Kendall coefficient of concordance against human ratings. Results: Our system achieved significantly higher concordance (.734) with human annotators than did a baseline using the Flesch-Kincaid Grade Level, a widely adopted readability formula (.531). The improvement was also consistent across different disease topics. This method's concordance with an individual human user's ratings was also higher than the concordance between different human annotators (.658). Conclusions: We explored methods to automatically assess the readability levels of clinical narratives. Our ranking-based system using simple textual features and easy-to-learn word embeddings outperformed a widely used readability formula. Our ranking-based method can predict relative difficulties of medical documents. It is not constrained to a predefined set of readability levels, a common design in many machine learning-based systems. Furthermore, the feature set does not rely on complex processing of the documents. One potential application of our readability ranking is personalization, allowing patients to better accommodate their own background knowledge.
... This is different to what is found for words in a non-scientific context where word length is indicative of complexity [53]. The findings of this study suggest that the conventional readability metrics may need to be further improved for use within a regulatory context and it echos the limitations summarised by Redish [54]. The techniques that were applied are actually used in many fields as the default measures of complexity. ...
Article
Full-text available
Medical device regulations are dynamic, as they need to cover an ever changing landscape. In Europe this has led to a new set of regulations (both for Medical Devices and In Vitro Diagnostics), which replaced the old rules. This study is interested in how the complexity of these medical regulations changed over time and if additional time-based metrics can be associated with any of the complexity metrics. Complexity is defined in terms of readability of the text and it is computed using established linguistic measures, as well as Halstead complexity scores. It was shown that the regulatory complexity of new EU medical device regulations was higher than their predecessors, especially when Halstead complexity measures were considered. The complexity metrics obtained for the new regulations were subsequently associated with the time it took to consider these regulations. Only very weak Pearson’s correlation coefficients were found between the complexity scores and the obtained response times for the new regulations. This could indicate that there are issues with how complexity is perceived by those that need to apply these regulations. Taking the complexity of regulations into account can greatly help with the development of more user friendly regulations. The results from the data-driven methods that are applied in this research indicate that governments could benefit from focusing on making regulations more accessible and utilitarian. This would improve the stakeholder adherence and facilitate effective implementation. This work also highlighted the need to develop more suitable methods to analyse regulatory text to further inform the wider research community.
... Traditional or surface-based features are predictors that were used by experts for their old readability formulas for Filipino such as sentence and word counts in Guevarra (2011). Despite the claims that these features insufficiently measures deeper text properties for readability assessment (Redish, 2000), since this is the pioneering study for Cebuano, we still considered these features for our baseline model development. In this study, we adapted the seven features of traditional features from worsk in Filipino (Imperial and Ong, 2020, 2021a,b) such as number of unique words, number of words, average word length, average number of syllables, total number of sentences, average sentence length and number of polysyllable words. ...
Preprint
Full-text available
In this study, we developed the first baseline readability model for the Cebuano language. Cebuano is the second most-used native language in the Philippines with about 27.5 million speakers. As the baseline, we extracted traditional or surface-based features, syllable patterns based from Cebuano's documented orthography, and neural embeddings from the multilingual BERT model. Results show that the use of the first two handcrafted linguistic features obtained the best performance trained on an optimized Random Forest model with approximately 84\% across all metrics. The feature sets and algorithm used also is similar to previous results in readability assessment for the Filipino language showing potential of crosslingual application. To encourage more work for readability assessment in Philippine languages such as Cebuano, we open-sourced both code and data.
... The limitations of our study include the potential inability of readability statistics to predict true comprehension due to jargon, syntax, or other linguistic features [18,19]. This study also does not consider whether some vaccine recipients desire additional information beyond what is offered [20]. ...
Article
Background More than 130 million individuals in the United States have now received at least one dose of a COVID-19 vaccine. Currently, all adults in the Unites States now have access to one of three COVID-19 vaccines. As part of the vaccination procedure, Emergency Use Authorization (EUA) fact sheets, which contain information regarding the vaccine, are provided. The purpose of this study was to analyze the ease of reading (i.e., readability) of the EUA-approved fact sheets for the vaccines currently available in the United States, the V-Safe adverse event survey script, and the Centers for Disease Control and Prevention (CDC) website information on COVID-19 vaccines designed for the general public in the United States. Methods We acquired the Pfizer, Moderna, and Janssen EUA fact sheets, as well as the V-Safe survey script and the CDC website information regarding COVID-19 vaccines. These documents were analyzed for their complexity regarding the following readability factors: average length of paragraphs, sentences, and words; font size and style; use of passive voice; the Gunning-Fog index; the Flesch Reading Ease index; and the Flesch-Kincaid Grade Level index. Results Only the V-Safe adverse-event survey script met readability standards for adequate comprehension. The mean readability scores of the EUA fact sheets and the CDC website were as follows: Flesch Reading Ease score (44.35 avg); Flesch-Kincaid Grade Level (10.48 avg); and Gunning-Fog index (11.8 avg). These scores indicate that at least a 10th-grade level education would be required to understand these reading materials. Conclusion The average person in the United States would have difficulty understanding the information provided in the EUA fact sheets and CDC COVID-19 vaccine website documents; however, the V-Safe survey was written at an adequate reading level. To ensure that the general public fully understands information regarding COVID-19 vaccines, greater care and effort should be given to the development of simplified information material.
... The formulas have been criticized a lot by some researchers in the field of reading comprehension studies, as well as teaching writing, particularly because most of these formulas are often based on only two variables (i.e., semantic and syntactic difficulty) which are measured by word length or frequency and sentence length. These two features only, may not be, as these researchers believe, sufficient predictors of language difficulty in a reading passage (to cite, for example, Chambers, 1983;Redish, 2000;Baker, 2020 b). ...
... The formulas have been criticized a lot by some researchers in the field of reading comprehension studies, as well as teaching writing, particularly because most of these formulas are often based on only two variables (i.e., semantic and syntactic difficulty) which are measured by word length or frequency and sentence length. These two features only, may not be, as these researchers believe, sufficient predictors of language difficulty in a reading passage (to cite, for example, Chambers, 1983;Redish, 2000;Baker, 2020 b). ...
... 28 Aside from English, by far the most widely spoken language is 29 Spanish (40 million speakers), followed by Chinese (3.3 million), 30 Tagalog (1.7 million), Vietnamese (1.5 million), and French 31 (1.2 million) (13). The quality of many health resources 32 for non-English users, however, is far from ideal (15, 16), 33 contributing to health disparities nationwide and amplifying 34 structural inequalities accentuated by the pandemic (17-19). 35 U.S. residents receive information about COVID-19 from a 36 diverse selection of official sources, ranging from federal agen-37 cies such as the Centers for Disease Control and Prevention 38 (CDC) and Food and Drug Administration (FDA) to state, 39 county, and local health departments. ...
Preprint
Lack of high-quality multilingual resources can contribute to disparities in the availability of medical and public health information. The COVID-19 pandemic has required rapid dissemination of essential guidance to diverse audiences and therefore provides an ideal context in which to study linguistic fairness in the U.S. Here we report a cross-sectional study of official non-English information about COVID-19 from the Centers for Disease Control and Prevention, the Food and Drug Administration, and the health departments of all 50 U.S. states. We find that multilingual information is limited in many states, such that almost half of all individuals not proficient in English or Spanish lack access to state-specific COVID-19 guidance in their primary language. Although Spanish-language information is widely available, we show using automated readability formulas that most materials do not follow standard recommendations for clear communication in medicine and public health. In combination, our results provide a snapshot of linguistic unfairness across the U.S. and highlight an urgent need for the creation of plain language, multilingual resources about COVID-19.
Article
Background Veterinarians often provide supplemental healthcare information to horse owners via newsletters and website articles. However, articles written above the reading level of the intended audience contributes to misunderstanding. To ensure that the text in equine healthcare articles and brochures is consistent with the literacy of clients, veterinarians can adopt guidelines set forth by the National Institutes of Health (NIH) and the American Medical Association (AMA) for a target 6th grade readability level. Objectives The goal of this study was to evaluate the readability levels of a sample of American Association of Equine Practitioners’ (AAEP) newsletter and website articles prepared expressly for veterinarians to download and share with their clients. Examples from these handouts are used to illustrate principles and techniques veterinary professionals can use to align their writing with the literacy of their intended audience. Study design Software‐based readability analysis of 17 AAEP newsletter and website articles. Methods A free online readability calculator was used to generate a consensus grade level readability score for 17 downloadable AAEP veterinary newsletter and website articles. Results Sixteen of 17 articles were written above the recommended 6th grade reading level. Main limitations We propose that a 6th grade readability level, as set forth by the AMA, is a reasonable target for the diverse population that makes up veterinary clients; however, there is currently no research that establishes this target for veterinary clients and the American Veterinary Medical Association (AVMA) has yet to issue a consensus statement on the subject. Conclusions Awareness of the issue of client literacy and use of tools such as readability analysis software can help veterinarians provide clients with ‘easy‐to‐read’ written materials that deliver a message that clients can comprehend, thus improving their health literacy and empowering them as partners in the veterinary‐client relationship.
Article
Purpose: The purpose of this study was to examine the readability of published patient-reported outcome measures (PROMs) designed for use in adult audiologic rehabilitation. The readability results were compared with the readability levels recommended for health information by health literacy experts. Method: Reading grade levels were calculated using the Flesch-Kincaid Grade Level Formula (Flesch, 1948), Gunning Fog Index (Gunning, 1952), Simple Measure of Gobbledygook (McLaughlin, 1969), and FORCAST (Caylor, Sticht, Fox, & Ford, 1973) readability formulas for 10 published PROMs. Descriptive statistics were computed across the different PROM sections: instructions, items, response scale, and overall contents of the measure directed toward respondents. Results: The majority of the PROM sections exceeded the 6th grade reading level recommended by health literacy experts, regardless of the formula applied. All PROM sections exceeded the 6th grade reading level when calculated according to the FORCAST formula, the most appropriate readability formula for use with a nonnarrative text format, such as PROMs. Conclusions: When developing or reevaluating PROMs designed for use in adult audiologic rehabilitation, researchers should consider ways to improve the readability of their measure, as poor readability may affect the validity of the empirical data collected using the PROM. Additionally, the adequate readability of audiologic PROMs is required if patient/family-centered care values are to be adhered to within the field of adult audiologic rehabilitation.
Article
Full-text available
Under the Consumer Rights Act 2015 consumer contracts and consumer notices are required to be expressed in plain and intelligible language. This is a difficult concept to capture. Determining whether a contract is expressed in plain and intelligible language involves resource intensive work by regulators and difficult adjudications by courts. This paper explores whether reading scores present a viable alternative. Can a simple computer program tell a consumer, a business, a regulator or the court that a particular contract is not expressed in plain and intelligible language? The paper begins by exploring the concept and role of plain and intelligible language in the Consumer Rights Act, before considering the ways that reading scores have developed and been used in legal contexts. We then report on the findings of an experimental examination of insurance contracts using a basket of reading scores, using our findings to draw conclusions about the utility of reading scores in determining whether a contract is expressed in plain and intelligible language. We find that reading scores can play a role in such determinations, but that further work is needed to provide appropriate tools for business, regulators and courts to use in assessing plain and intelligible language.
Article
Text readability assessment is a challenging interdisciplinary endeavor with rich practical implications. It has long drawn the attention of researchers internationally, and the readability models since developed have been widely applied to various fields. Previous readability models have only made use of linguistic features employed for general text analysis and have not been sufficiently accurate when used to gauge domain-specific texts. In view of this, this study proposes a latent-semantic-analysis (LSA)-constructed hierarchical conceptual space that can be used to train a readability model to accurately assess domain-specific texts. Compared with a baseline reference using a traditional model, the new model improves by 13.88% to achieve 68.98% of accuracy when leveling social science texts, and by 24.61% to achieve 73.96% of accuracy when assessing natural science texts. We then combine the readability features developed for the current study with general linguistic features, and the accuracy of leveling social science texts improves by an even higher degree of 31.58% to achieve 86.68%, and that of natural science texts by 26.56% to achieve 75.91%. These results indicate that the readability features developed in this study can be used both to train a readability model for leveling domain-specific texts and also in combination with the more common linguistic features to enhance the efficacy of the model. Future research can expand the generalizability of the model by assessing texts from different fields and grade levels using the proposed method, thus enhancing the practical applications of this new method.
Chapter
People with cognitive disabilities are gaining in a long struggle for recognition of their right to control their lives. In the information society, access to the Web is essential to this control. Cognitive barriers to this access are diverse, reflecting the complexity of human cognitive faculties. Advances in supporting configurable presentation and interaction methods, and in representing the meaning as well as the form of information, will yield progress. This progress is being accelerated by increasing international awareness of the importance of cognitive access.
Chapter
Access to the Web for people with cognitive, language, and learning differences and limitations is widely recognized as important and becoming more important as Web content and Apps become ubiquitous. Yet progress has been slow, as indicated by limited support for cognitive accessibility within the Web Content Accessibility Guidelines process. How can this progress be accelerated, and how can research contribute to the increase in cognitive accessibility that is needed?
Article
Text readability assessment is a well-known problem that has acquired even more importance in today’s information-rich world. In this paper, we survey various approaches to measuring and assessing the readability of texts. Our specific goal is to provide a perspective on the state of the art in readability assessment research for Arabic, which differs significantly from other languages on which readability studies have tended to focus. We provide background on readability assessment research and tools for English, for which readability studies are the most advanced. We then survey approaches adopted for Arabic, both classical formula-based approaches and studies that combine Machine Learning (ML) with Natural Language Processing (NLP) techniques. The works we cover target text corpora for different audiences: school-age first language readers (L1), foreign language learners (L2), and adult readers in non-academic contexts. Therefore, we explore differences between reading in L1 and L2 and consider how they play out specifically in Arabic after describing language characteristics that may impact readability. Finally, we highlight challenges for Arabic readability research and propose multiple future directions to improve readability assessment and related applications that would benefit from more attention.
Article
Full-text available
From a long time, readability formulas have remained favorite and trustworthy in analyzing the readability, however, researches have now made their trust worthiness a bit shaky as they lack usability components in them. In this paper, the readability formulas are analyzed against usability factors. For this purpose, a group of 40 readers belonging to different age groups, gender, educational backgrounds and variant skills, were selected to analyze the effect of color contrast on readability. The readability measures in terms of usability constraints are put on the scrutiny for the validity and effectiveness of the formulas. The usability factors that can be met by emending or modifying the readability formulas are put forth to get the desired success ratio for ease of reading. Among the identified factors, the color contrast of foreground and background is picked and a survey was conducted that ensured the strong effect of color contrast on readability. The color contrast factor is adjusted in the readability formula that resulted better and enhanced readability analysis.
Chapter
Readability is referred to as the degree of difficulty to which an given text (article) can be understood by readers. When readers are reading a text with high readability, they will achieve better comprehension and learning retention. However, it has been a long-standing critical challenge to develop effective readability prediction models that can automatically and accurately assess the readability of a given text. When building readability prediction models for the Chinese language, word segmentation ambiguity is often a knotty problem that will inevitably happen in the pre-processing of texts. In view of this, we present in this paper a novel readability prediction approach for the Chinese language, building on a recently proposed, so-called Bidirectional Encoder Representation from Transformers (BERT) model that can capture both syntactic and semantic information of a text directly from its character-level representation. With the BERT-based readability prediction model that takes consecutive character-level representations as its input, we effectively assess the readability of a given text without the need of performing error-prone word segmentation. We empirically evaluate the performance of our BERT-based readability prediction model on a benchmark task, by comparing it with a strong baseline that utilizes a celebrated classification model (named fastText) in conjunction with word-level presentations. The results demonstrate that the BERT-based model with character-level representations can perform on par with the fastText-based model with word-level representations, yielding the accuracy of 78.45% on average. This finding also offers the promise of conducting readability assessment of a text in Chinese directly based on character-level representations.
Article
This study examines the strategic use of readability to obfuscate negative news in a German financial communication context. Combining a manual and an automated content analysis, the authors assess the tone and readability of three parts (chairman’s address, share-price development, and development in the fiscal year) of the 2014 annual reports of the 30 companies listed in the German stock index DAX. The results indicate that positively connoted passages in annual reports are not necessarily easier to read than negatively connoted passages. Furthermore, the readability of the annual report varies depending on the part and its function within the report.
Article
Machine learning is the process of training a computer program to reproduce and generalize the relationship between some kind of input and some kind of output. High profile successes for machine learning are attracting interest and investment. Can this technology be applied in ways that benefit people with cognitive disabilities and those who support them? What are the current limitations in the technology that affect these applications, and what are the prospects for overcoming them? What actions can maximize the benefits for people with cognitive disabilities?
Article
Full-text available
zet Yapay Zekâya dayalı bir okunabilirlik formülü ile yalnızca bir metnin okunabilirliğini değil aynı zamanda anlaşılabilirliğini de aynı anda tespit etmek mümkündür. Geleneksel okunabilirlik formüllerinde yalnızca metine dayalı özelliklerden hareketle düzey tespiti yapılırken yapay zekâya dayalı formüllerde metne dayalı daha fazla değişkenin yanı sıra okuyucu özelliklerinin de dâhil olduğu bir düzey ataması yapılmaktadır. Türkçe için en son geliştirilen formül 2010 yılında yapılmıştır. Aradan geçen 10 boyunca yeni bir formül geliştirilmemiştir. Bununla birlikte yapay zekâya dayalı formül çalışmaları dünyada 2009 yılı itibariyle geliştirilmeye başlamıştır. Bu araştırmada parmakla, elle ya da gözle sayım yapılan okunabilirlik formüllerinden, Türkçe yapay zekâya dayalı okunabilirlik formülünün oluşmasında ilk basamağı oluşturmaya dönük Türkçenin yapısal özelliklerini ortaya koymak amaçlanmıştır. Araştırma, tarama modelinde hazırlanmıştır. Mevcut okunabilirlik çalışmalarının tamamı geleneksel yaklaşımla oluşturulmuş olup formüllerin neredeyse tamamında nicel değişkenler (kelime, cümle uzunluğu gibi) üzerine kurulduğu nitel değişkenlerin yani anlaşılabilirlik kısmının ise göz ardı edildiği tespit edilmiştir. Araştırma kapsamında belirlenen okunabilirlik formüllerinin tamamı 5, 6, 7 ve 8. sınıf Türkçe ders kitaplarındaki 12 metin ve 6 çocuk kitabı üzerinde uygulanmıştır. Araştırma sonucunda formüllerin düzey sonuçları arasında uyumsuzluk olduğu, Ateşman'ın (1997) formülüne göre kolay olarak belirlenen bir metin, Flesch'te (1948) çok zor; Dale ve Chall'de (1948) 10-12 yaş olarak belirlenen bir metin ARI'da (1967) 16-17 yaş olarak belirlenmiştir. Ayrıca bu formüllerdeki yönerge eksikliği ise var olan değişkenlerin de doğru olarak tespit edilmesinde belirsizlikleri arttırmıştır. Okuyucuya dair değişkenlerin ise hiçbir formüllerde işe koşulmadığı tespit edilmiştir. Türkçenin yapısal özellikleri bağlamında en çok "cümle" başlığında farklı kullanımların olduğu ortaya konmuştur. Araştırmada daha fazla eser üzerinden Türkçenin yapısal özelliklerinin tespit edilmesi, okunabilirliğin nicel değişkenlerle birlikte nitel değişkenlerin işe koşulduğu ve sürekli kendini güncelleyebilen yapay zekâya dayalı bir formülün hazırlanması önerilmiştir. Abstract With a readability formula based on AI, it is possible to detect not only the readability of a text, but also its comprehensibility at the same time. In traditional readability formulas, only level determination is made based on text-based features, while artificial intelligence-based formulas are assigned a level that includes more text-based variables as well as reader features. The most recently developed formula for Turkish was made in 2010. No new formulas have been developed during the intervening 10. In addition, formula studies based on artificial intelligence started to be developed around the world, as of 2009. In this study, it was aimed to reveal the structural features of Turkish to form the first step in the formation of the readability formula based on Turkish artificial intelligence, one of the readability formulas counted by finger, hand or eye. The research was prepared in the screening model. All the existing readability studies have been created with a traditional approach, and it has been determined that almost all the formulas are based on quantitative variables (such as word, sentence length),
Chapter
When people try to read technical information, they try to assess the relevance of information as quickly as possible. They set their own reading goals in performing this task and skip the paragraphs or sections they do not consider relevant (Janssen & Neutelings, 2001). This idea is reflected in the Web reading mantra that if a website does not grab a person immediately, they will leave. While the full truth of this mantra can be debated, the rapid reading and evaluation of information must be considered as part of designing for HII to allow people to comprehend the information.
Chapter
This chapter considers how the presentation of information on a display affects the HII. The type and order of the presentation exerts a strong influence on how people perceive and interpret information. The white area in figure 1 shows the area of the HII model relevant to this chapter. The design team’s goal must be to ensure the presentation provides only relevant information, presents it in a salient manner, and in a way that corresponds to people’s expectations and needs. It must make information relationships salient. In many failed designs, these simple concepts fail to hold true.
Article
To avoid excluding individuals with limited English proficiency from participating in research, the consent form and other documents should be presented to them in their primary language and in a format that is understandable. However, evidence suggests that, when documents are translated for prospective and actual research participants with limited English proficiency, these individuals often fail to engage with the documents and the research in the same terms as their English-speaking counterparts do. We argue that this is because methodological challenges remain after a decision to translate has been made. This study investigated how translation approaches affected reader response and intelligibility. Participants were asked to review two translated versions of a survey (which reflected a functionalist and a literal approach to translation) followed by semistructured interviews. Quantitative and qualitative analysis revealed a preference for a functionalist translation and a higher number of problems raised in regard to the literal translation. The recommendations we offer here include considering the most appropriate translation approach for a specific genre and purpose.
Article
This study examines how the readability of scientific discourses changes over time and to what extent readability can explain scientific impact in terms of citation counts. The basis are representative datasets of 135,502 abstracts from academic research papers pertaining to twelve technologies of different maturity. Using three different measures of readability, it is found that the language of the abstracts has become more complex over time. Across all technologies, less easily readable texts are more likely to receive at least one citation, while the effects are most pronounced for comparatively immature research streams. Among the more mature or larger discourses , the abstracts of the top 10% and 1% of the most often cited articles are significantly less readable. It remains open to what extent readability actually influences future citations and how much of the relationship is causal. If readability indeed drives citations, the results imply that scientists have an incentive to (artificially) reduce the readability of their abstracts in order to signal quality and competence to readers-both to get noticed at all and to attract more citations. This may mean a prisoner dilemma in academic (abstract) writing, where authors intentionally but unnecessarily complicate the way in which they communicate their work.
Article
This article aims to explore the quality of the information provided by public authorities to citizens, businesses, and other stakeholders as part of the implementation of e-services of “A State in a Smartphone” in Ukraine. The article presents the structure of the authors’ model of information quality assessment, which includes three levels of characteristics and allows calculating the integral indicator of information quality. The model involves the use of expert research methods. The results of the study indicate that the information provided to users by public authorities has a fairly high level of quality, but there are reserves for improvement.
Article
An online survey ( N = 810) examined the impact of design best practices on the perceived usability of End-User License Agreements (EULAs). Redesigning a EULA according to best-practices (without changing the EULA’s terms and conditions) led to higher perceived usability while responses to attitudinal (perceived reasonableness of conditions) and behavioral (anticipated agreement and use) items were unaffected. Readers (who reported reading EULAs frequently) (a) provided more positive evaluations of EULAs and (b) were more likely to anticipate agreeing to EULA terms than non-readers . These results suggest that best practices in document design can improve the usability of End-User License Agreements.
Article
Introduction Emergency departments (EDs) routinely provide discharge instructions due to a large number of patients with mild traumatic brain injury (mTBI) being discharged home directly from ED. This study aims to evaluate the quality of available mTBI discharge instructions provided by EDs of Malaysia government hospitals. Methods All 132 EDs were requested for a copy of written discharge instruction given to the patients. The mTBI discharge instructions were evaluated using the Patient Education Materials Assessment-Printable Tool (PEMAT-P) for understandability and actionability. Readability was measured using an online readability tool of Malay text. The content was compared against the discharge instructions recommended by established guidelines. Results 49 articles were eligible for the study. 26 of the articles met the criteria of understandability, and 3 met the criteria for actionability. The average readability level met the ability of average adult. Most of the discharge instructions focused on emergency symptoms, and none contained post-concussion features. Conclusion Majority of the discharge instructions provided were appropriate for average people to read but difficult to understand and act upon. Important information was neglected in most discharge instructions. Thus, revision and future development of mTBI discharge instruction should consider health literacy demand and cognitive ability to process such information.
Chapter
There has been widespread application of readability formulas as a tool for defining plain English in the production of texts as well as in judging existing documents. There are numerous reasons why readability formulas have been selected to fulfil this defining role. However, the findings of Duffy and Kabance along with Kniffen et al present a strong case against the readable writing approach to revision and hence against the use of a readability formula as a feedback device for the writer. Kniffen et al used a readable writing style manual. In both cases, conditions were optimal for the readability improvements to facilitate comprehension. Yet in both cases the manipulations, with one exception, resulted in no effect or, at best, marginal effect on comprehension. If the revision approach does not produce large comprehension effects under ideal testing conditions, then there must be little expectation for the approach to be effective in practical application. The findings of Duffy and Kabance, in fact, suggest that some readable writing techniques will not be effective in improving comprehension under any circumstances. The effectiveness of other simplification strategies will depend on the reading requirements and reading conditions.
Article
Some reading researchers and technical communicators assume the efficacy of readability formulas. Reading researchers use such formulas to equalize the reading difficulty of texts used in experiments. Results of an informal Internet survey indicate that some professional writers and editors use readability formulas that are integrated into word-processing software. This article proposes that readability formulas fail to predict text difficulty. The results of an experiment demonstrate that "text difficulty" is a perception of the reader and therefore cannot be objectively calculated by counting syllables, word length, sentence length, and other text characteristics.
Book
This book is for writers and graphic designers who create the many types of documents people use every day at home or school, in business or government. From high-tech instruction manuals and textbooks to health communications and information graphics, to online information and World Wide Web pages, this book offers one of the first research-based portraits of what readers need from documents and of how document designers can take those needs into account. Drawing on research about how people interpret words and pictures, this book presents a new and more complete image of the reader—a person who is not only trying to understand prose and graphics but who is responding to them aesthetically and emotionally. Dynamics in Document Design features: • Case studies of documents before and after revision, showing how people think and feel about them • Analyses of the interplay of text and pictures, revealing how words, space, visuals, and typography can work together • An informative timeline of the international evolution of document design from 1900 to the 1997
Article
In this paper major factors interacting with readability measures in validity studies are identified and described. The basis for the analysis presented are thirty-six experimental studies of the effect of readability variables upon reader comprehension and/or retention. Emphasis is placed on the interacting nature of the factors involved.
Article
Compares and contrasts standards and techniques of qualitative and quantitative usability testing methodologies. Focuses on ways technical communicators can ensure rigorous tests in terms of the validity and reliability of findings, defining criteria of validity and reliability for both quantitative and qualitative methods. Presents a list of canons, based on documented principles of rigorous research, applicable to usability testing. (SR)
Understanding the language of public documents because readability formulas don't
  • V M Holland
  • L J Campbell
An examination of the McCall- Crabbs standard test lessons in reading
  • M D Jacobson
  • C E Kirkland
  • R W Selden
Readability Standards for Army- Wide Publications
  • G R Klare
Document Design: A Review of the Relevant Research
  • J C Redish
An examination of the McCall- Crabbs standard test lessons in reading
  • Jacobson M. D.