Harris Shah Abd Hamid’s research while affiliated with University of Kuala Lumpur and other places

Assessment in mathematics education in the 21st century should be more directed at higher-order thinking skills (HOTS). Not only teachers but also prospective mathematics teachers should be supported to design and carry out HOTS assessments. This qualitative study applying a phenomenology approach was mainly conducted to investigate the challenges that prospective mathematics teachers face in developing HOTS questions, including their strategies for dealing with these challenges. Our informants were 20 students of master’s program in mathematics education. They were enrolled in the assessment of mathematics learning course, which was designed with a project-based learning model. Considering the learning model used, the purpose of this study then was extended to explore the benefits of implementing a project-based learning model in supporting student competence in developing HOTS questions. Data collection was carried out by (1) administering open-ended questionnaires; (2) observing the end product in the form of mathematics learning achievement tests and test blueprints; and (3) involving two experts who worked independently to judge the questions posed by students based on levels in the cognitive process dimension in the revised Bloom’s taxonomy. Data collected from the open-ended questionnaire were analyzed qualitatively, while data from observation and judgment by experts on the end product yielded by students were analyzed descriptively. Our study supports previous studies, which demonstrate that the application of a project-based learning model that involves students to develop HOTS questions deepens students’ knowledge of assessment. In developing HOTS questions, students struggled more in matching action verbs, item indicators, and test items to the level of cognitive process they defined, as well as constructing multiple-choice HOTS questions. The strategies that students took in dealing with challenges that arose in developing HOTS questions indicate that students regulate their learning.

Scoring quality has been recognized as one of the important aspects that should be of concern to both test developers and users. This study aimed to investigate the effect of scoring correction and model fit on the estimation of ability parameters and person fit in the polytomous item response theory. The result of 165 students in the Statistics course (SATS4410) test at one of the universities in Indonesia was used to answer the problems in this study. The polytomous data obtained from scoring the test results were analyzed using the Item Response Theory (IRT) approach with the Partial Credit Model (PCM), Graded Response Model (GRM), and Generalized Partial Credit Model (GPCM). The effect of scoring correction and model fit on the estimation of ability and person fit was tested using multivariate analysis. Among the three models used, GRM showed the best fit based on p-value and RSMEA. The results of the analysis also showed that there was no significant effect of scoring correction and model fit on the estimation of the test taker’s ability and person fit. From the results of this study, we recommend the importance of evaluating the levels or categories used in scoring student work on a test.