Guher Gorgun

• Doctor of Philosophy
• University of Alberta

Exploring the relationship between student ability and test-taking effort is an important area of study, offering insights into their approach to educational assessments. Previous research shows this relationship, yet there is a scarcity of research comparing the test-taking effort of students. In addition, researchers have frequently employed rule...

Automatic item generation may supply many items instantly and efficiently to assessment and learning environments. Yet, the evaluation of item quality persists to be a bottleneck for deploying generated items in learning and assessment settings. In this study, we investigated the utility of using large‐language models, specifically Llama 3‐8B, for...

The purpose of this dissertation is to employ three prominent atural language processing methods to assess the feasibility of automatically evaluating cloze questions generated by automatic item generation (AIG) methods. AIG methods have been developed to address the need for a large number of items for computerized assessments as well as online le...

Background Research shows that how formative assessments are operationalized plays a crucial role in shaping their engagement with formative assessments, thereby impacting their effectiveness in predicting academic achievement. Mandatory assessments can ensure consistent student participation, leading to better tracking of learning progress. Option...

Pretrained large language models (LLMs) have gained popularity in recent years due to their high performance in various educational tasks such as learner mod-eling, automated scoring, automatic item generation, and prediction. Nevertheless, LLMs are black box approaches where models are less interpretable, and they may carry human biases and prejud...

This paper summarizes our methodology and results for the BEA 2024 Shared Task. This competition focused on predicting item difficulty and response time for retired multiple-choice items from the United States Medical Licensing Examination® (USMLE®). We extracted linguistic features from the item stem and response options using multiple methods, in...

In light of the widespread adoption of technology-enhanced learning and assessment platforms, there is a growing demand for innovative, high-quality, and diverse assessment questions. Automatic Question Generation (AQG) has emerged as a valuable solution, enabling educators and assessment developers to efficiently produce a large volume of test ite...

Purpose: Previous studies examining the inter-relations between serial and discrete naming with reading have found that the ability to efficiently process multiple items presented in a sequence (indexed by serial naming) is a unique predictor of word- and text-reading fluency. However, conclusions have been tempered by the concurrent nature of the...

In educational assessment, behavioral engagement refers to the extent to which learners actively participate in the assessment process and are motivated to perform well. Behavioral engagement plays a crucial role in the design, use, and interpretation of educational assessments because learners who are engaged in the assessment process are likely t...

This study provides a comprehensive review of frequently used evaluation methods for assessing the quality of automatic question generation (AQG) systems based on computational linguistics techniques and large language models. As we present a comprehensive overview of the current state of evaluation methods, we discuss the advantages and limitation...

This chapter explores the dynamic synergy between artificial intelligence (AI) and learning analytics (LA) as catalysts for self-regulated learning (SRL). By examining the intersection of AI, LA, and SRL, this chapter sheds light on the evolving landscape of education and the opportunities it offers for tailored, data-driven learning experiences. I...

In this study, we present three types of unsupervised anomaly detection to identify anomalous test-takers based on their action sequences in problem-solving tasks. The first method relies on the use of the Isolation Forest algorithm to detect anomalous test-takers based on raw action sequences extracted from process data. The second method transfor...

The importance of non-cognitive skills for academic achievement and future success has been emphasized but the invariance among the relationships of these constructs across different groups and countries is rarely studied. In this study, we used a novel approach, psychometric network analysis, to analyze the invariance of connections between non-co...

The use of multistage adaptive testing (MST) has gradually increased in large-scale testing programs as MST achieves a balanced compromise between linear test design and item-level adaptive testing. MST works on the premise that each examinee gives their best effort when attempting the items, and their responses truly reflect what they know or can...

Using online formative assessments, students can monitor their learning and find strategies to attain their self-regulated learning (SRL) goals. However, as formative assessments are often optional and ungraded, some students may not be motivated enough to participate in such assessments. In this study, we argue that the frequency and stakes (i.e.,...

This study aims to introduce Bayesian Knowledge Tracing (BKT), a probabilistic model used in educational data mining to estimate learners' knowledge states over time. It also provides a practical guide to estimating BKT models using the pyBKT library available in Python. The first section presents an overview of BKT by explaining its theoretical fo...

In low-stakes assessment settings, students’ performance is not only influenced by students’ ability level but also their test-taking engagement. In computerized adaptive tests (CATs), disengaged responses (e.g., rapid guesses) that fail to reflect students’ true ability levels may lead to the selection of less informative items and thereby contami...

In order to facilitate student learning, it is important to identify and remediate misconceptions and incomplete knowledge pertaining to the assigned material. In the domain of mathematics, prior research with computer-based learning systems has utilized the commonality of incorrect answers to problems as a way of identifying potential misconceptio...

In low-stakes assessments with a strict time limit, some students may fail to reach the end of the test and leave some items unanswered due to various reasons, such as fatigue and lack of test-taking motivation, which are referred to as not-reached items (NRIs). NRIs were ubiquitous in the Problem Solving and Inquiry (PSI) portion of eTIMSS 2019, a...

Parameter calibration in educational assessments often requires a large sample, which involves significant test administration costs. This study uses machine learning to predict item parameters based on the textual features of questions. The data used consisted of grade-4 life science multiple-choice questions (N = 43) released from three previous...

Rapid guessing is an aberrant response behavior that commonly occurs in low-stakes assessments with little to no formal consequences for students. Recently, the availability of response time (RT) information in computer-based assessments has motivated researchers to develop various methods to detect rapidly guessed responses systematically. These m...

Examinees' unexpected response behaviors during an assessment may lead to aberrant responses that contaminate the data quality. Since aberrant responses may jeopardize the validity of inferences made based on assessment results, they should be handled for modeling students' learning and progress more accurately. Although the detection of aberrant r...

As universities around the world have begun to use learning management systems (LMSs), more learning data have become available to gain deeper insights into students' learning processes and make data‐driven decisions to improve student learning. With the availability of rich data extracted from the LMS, researchers have turned much of their attenti...

Over the last decade, Canadian students have exhibited insubstantial improvements in mathematical scores compared to other countries as indicated by large-scale educational assessments such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). In relation to students’ mat...

Research extensively highlights the importance of social-emotional skills in learning and development. In this study, we evaluated whether social and emotional variables directly impact students' perceived cognitive competence and academic performance through a structural equation model. Survey responses (N = 29,384) were collected from 114 K-12 sc...

In this study, we analyzed the influence of student disengagement on prediction accuracy in knowledge tracing models. During the data pre-processing stage, we prepared two training data: The disengaged responses were ignored in the baseline data whereas the disengaged responses were removed in the disengagement-adjusted data. Using visual analysis,...

The need to identify student cognitive engagement in online learning settings has increased with our use of online learning approaches because engagement plays an important role in ensuring student success in these environments. Engaged students are more likely to complete online courses successfully, but this setting makes it more difficult for in...

Careless and insufficient effort responding (C/IER) on self-report measures results in responses that do not reflect the trait to be measured, thereby posing a major threat to the quality of survey data. Reliable approaches for detecting C/IER aid in increasing the validity of inferences being made from survey data. First, once detected, C/IER can...

Explanatory item response modeling (EIRM) enables researchers and practitioners to incorporate item and person properties into item response theory (IRT) models. Unlike traditional IRT models, explanatory IRT models can explain common variability stemming from the shared variance among item clusters and person groups. In this tutorial, we present t...

In low-stakes assessments, some students may not reach the end of the test and leave some items unanswered due to various reasons (e.g., lack of test-taking motivation, poor time management, and test speededness). Not-reached items are often treated as incorrect or not-administered in the scoring process. However, when the proportion of not-reached...

Background/Context High-stakes testing (HST) weaves through the fabric of school life, stretching beyond the test day. Results have consequences for a school's reputation and autonomy, as well as teachers’ evaluations and students’ graduation and morale. Prior research demonstrates the constraining and inequitable effects assessments can have on st...

Researchers often use self-reported instruments to collect data from students when investigating the causes and effects of bullying. When completing the instrument, students may skip items despite anonymous data collection. To interpret data accurately, researchers must identify the causes of nonresponse. This study examined the impact of item (typ...

The Rosenberg Self-Esteem Scale was administered with a 1–4, 1–5, or 0–100 scale to 819 participants, to compare score interpretations across the different versions. A rating scale utility analysis revealed that the categories in the 101-point scale were used inconsistently; based on the analysis, adjacent categories were collapsed resulting in a 7...

The Diagnostic Assessment and Achievement of College Skills (DAACS) online system assesses newly enrolled college students’ skills in reading, writing, mathematics, and self-regulated learning, and provides individualized feedback and links to resources. The purpose of this study is to examine validity evidence regarding the internal structure of t...