Figure 2 - uploaded by Ching-Ni Hsieh
Content may be subject to copyright.
Source publication
Researchers suggest that claims about the meaningfulness of test score interpretations and consequences of test use should be backed by evidence that stakeholders understand the definition of the construct assessed (meaningfulness) and score reports (consequences). Evaluation of stakeholders' actual uses and interpretations of score reports in larg...
Context in source publication
Context 1
... responses to Q13 show how satisfied the participants felt about the different aspects of the TOEIC L&R score report. As seen in Figure 2, the majority of the participants felt "very satisfied" or "somewhat satisfied" with the overall design of the score report and the ease of finding the test scores. In contrast, the participants had lower degrees of satisfaction with the ease of understanding the score descriptors, list of abilities measured, percent-correct bar graphs, and footnote; more than one third rated these reporting elements "not very satisfied" or "not at all satisfied," suggesting that these aspects of the score report may be inadequate or insufficient in providing meaningful information to the test takers. ...
Citations
... Since its inception, the TOEIC L&R test has been used for the following: "to evaluate an individual's ability to communicate in English, (b) to apply for jobs, (c) to hire, train, and promote employees, and (d) to better prepare students for employment opportunities" (Hsieh, 2023;p.03). It was also specifically designed to accurately assess a learner's business English skills (Gee, 2015), and general English communication skills in the workplace (ETS, 2024). ...
The TOEIC L&R test has been a standard measurement of English proficiency in the world of business in Japan for over 30 years and has claimed to provide test takers with the business domain language skills required for success. However, independent research has shown otherwise regarding the test content that ETS distributes globally. Japanese language learners preparing for the TOEIC are exposed to vocabulary prescribed by domestic test preparation material publications in both formal and independent learning environments. Vocabulary lists provided in such textbooks published from 2015 to 2024 were analyzed to calculate the quantities of business and general academic vocabulary to understand how relevant the language content was for each of these contexts. The wide disparity in the proportions of business and academic context vocabulary found as well as high variance across the 10 year period of publishing demonstrated lack of focus on English communication skills for business. These findings prompt further investigation on TOEIC study materials including those published and distributed by ETS about how well young professionals and graduates in Japan are being prepared for linguistic challenges ahead in the business environment.
... The test items in all four parts are designed to measure five aspects of listening ability. In other words, test score results can suggest to what extent a test taker is able to (1) infer gist, purposes, and basic contexts based on information stated clearly in short spoken texts, (2) do so but in extended spoken texts, (3) understand details in short spoken texts, (4) understand details in extended spoken texts, and (5) have pragmatic understanding of a speaker's purpose or implied meaning in a phrase or sentence (Cid et al., 2017;Hsieh, 2023). As demonstrated, the TOEIC listening test is recognized as a valid assessment for gauging test takers' listening abilities in various business settings. ...
The Test of English for International Communication (TOEIC) is a globally recognized standardized test that mainly assesses listening and reading skills necessary for communication in workplace contexts. However, the listening section, which accounts for half of the total score of 990, has been regarded as problematic for some Thai English as a foreign language (EFL) learners. This study, therefore, investigates the listening difficulties of six Thai EFL learners who struggled with the TOEIC test. The participants (fourth-year English majors from a university in Thailand) were asked to complete selected TOEIC listening test items and were interviewed about their listening problems immediately afterwards. The findings provide insights into listening problems in the testing context as task-based and individual-based. We categorized the top three problems as “I answer the choice that has the same or similar words as in the listening text.”, “I cannot catch the text immediately while listening.”, and “I make wrong inferences due to partial understandings.” Based on our findings, we make several pedagogical suggestions for TOEIC teachers and test takers, including an explicit focus on phonological knowledge, paraphrasing skills, automaticity, established vocabulary lists, and extensive listening practice.
... These scores hold considerable esteem, wielding a critical role in appraising an individual's English-language competence, providing a means for showcasing linguistic abilities, and augmenting one's prospects within an increasingly competitive job market. As a result, many individuals in Japan dedicate substantial time and effort in preparing for and undertaking the TOEIC exam in order to meet these requirements and enhance their opportunities in both academic and professional domains [36]. ...
This chapter investigates the role of Test of English for International Communication (TOEIC) scores within Japanese corporations and scrutinizes the resulting linguistic hierarchies that could foster linguicism and linguistic imperialism. The prominence of English proficiency, as reflected in TOEIC’s widespread use in hiring and promotion criteria, not only privileges certain linguistic groups but also enforces inequalities that affect individuals from diverse linguistic backgrounds. This chapter acquaints readers with the cultural norm of prioritizing TOEIC scores in Japan and its implications for both multilingualism and the professional mobility of non-native English speakers. Through a comprehensive analysis, this chapter elucidates the dichotomy between the need for English proficiency in a globalized business realm and the potential discrimination stemming from over-reliance on standardized English tests. Furthermore, this chapter discusses the pivotal role of TOEIC in Japan’s educational and professional sectors and how it shapes the socio-economic landscape, influencing salaries and career progression. Finally, the chapter advances recommendations for Japanese corporations to shift toward more equitable linguistic practices. The goal is to foster a multicultural work environment that values multilingualism and respects the linguistic diversity of all employees. By reevaluating the emphasis on TOEIC scores, Japanese corporations can take meaningful steps toward combating linguistic inequality, linguicism, and the impact of linguistic imperialism.
Research in validity suggests that stakeholders' interpretation and use of test results should be an aspect of validity. Claims about the meaningfulness of test score interpretations and consequences of test use should be backed by evidence that stakeholders understand the definition of the construct assessed and the score report information. The current study explored stakeholders' uses and interpretations of the score report of a workplace English language proficiency test, the TOEIC® Listening and Reading (TOEIC L&R) test. Online surveys were administered to TOEIC L&R test takers and institutional and corporate score users in Taiwan to collect data about their uses and interpretations of the test score report. Eleven survey respondents participated in follow‐up interviews to further elaborate on their uses of the different score reporting information within the stakeholders' respective contexts. Results indicated that the participants used the TOEIC L&R test scores largely as intended by the test developer although some elements of the score report appeared to be less useful and could be confusing for stakeholders. Findings from this study highlight the importance of providing score reporting information with clarity and ease to enhance appropriate use and interpretation.
The Test of English for International Communication (TOEIC) is a language proficiency test designed to measure English language listening and reading skills and is now a widely preferred certification in gauging English language proficiency. This study investigated whether the facilitation of online intervention programs, composed of practice tests, can help in maximizing the TOEIC scores of first-year and second-year students. The study made use of an experimental research approach and utilized independent samples t-test as statistical analysis in discerning the effect of online intervention programs on TOEIC scores. Results confirm that designing an online intervention program positively impacts students' TOEIC scores and students who undergo the said program garner higher TOEIC scores compared to students who do not. Aligned with the results, it is concluded that online intervention programs can support students' TOEIC-taking capabilities and maximize their achievement in the said test.
