Katja Ihsberner’s research while affiliated with Leipzig University of Applied Sciences and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (4)


Item generation workflow, implemented within ItemForge.
Manual item-process classification in contrast to targeted levels of the cognitive process dimension as a confusion matrix.
Manual item-knowledge classification in contrast to targeted levels of the knowledge dimension as a confusion matrix.
Qualitative aspects of the generated items, visualizing concept appropriateness (ca), task completeness (tcm), task correctness (tcr), solution completeness (scm), and solution correctness (scr) using box plots on a 5 point Likart-Scale for which 5.0 corresponds to good, 1.0 to poor.
LLM-generated competence-based e-assessment items for higher education mathematics: methodology and evaluation
  • Article
  • Full-text available

October 2024

·

61 Reads

Roy Meissner

·

·

Katja Ihsberner

·

[...]

·

In this article, we explore the transformative impact of advanced, parameter-rich Large Language Models (LLMs) on the production of instructional materials in higher education, with a focus on the automated generation of both formative and summative assessments for learners in the field of mathematics. We introduce a novel LLM-driven process and application, called ItemForge, tailored specifically for the automatic generation of e-assessment items in mathematics. The approach is thoroughly aligned with the levels and hierarchy of cognitive learning objectives as developed by Anderson and Krathwohl, and takes specific mathematical concepts from the considered courses into consideration. The quality of the generated free-text items, along with their corresponding answers (sample solutions), as well as their appropriateness to the designated cognitive level and subject matter, were evaluated in a small-scale study. In this study, three mathematical experts reviewed a total of 240 generated items, providing a comprehensive analysis of their effectiveness and relevance. Our findings demonstrate that the tool is proficient in producing high-quality items that align with the chosen concepts and targeted cognitive levels, indicating its potential suitability for educational purposes. However, it was observed that the provided answers (sample solutions) occasionally exhibited inaccuracies or were not entirely complete, signalling a necessity for additional refinement of the tool's processes.

Download

Individualised Mathematical Task Recommendations Through Intended Learning Outcomes and Reinforcement Learning

June 2024

·

26 Reads

·

1 Citation

Guiding students towards achieving the Intended Learning Outcomes (ILOs) of an academic module as part of a mentoring process presents a significant challenge, as it is important not only to emphasize the necessary skills, but also to consider the ongoing personal progress towards achieving a learning outcome. In addition, most educational content is presented in a ’one-size-fits-all’ way, without taking into account the individual needs of students. In this paper we present a recommendation system based on Reinforcement Learning (RL) that derives its suggestions from the students’ progress towards achieving the ILOs and the current relevance of the ILOs, according to the specific didactic design of the module. The taxonomy model proposed by Anderson and Krathwohl, serves as the groundwork for abstracting ILO progress, temporal relevance, and the affiliation of recommendation items. In the process of creating a recommendation pool, experts identified the mathematical concept and the taxonomy level addressed by existing e-assessments in order to identify their possible association with ILOs. The RL agent utilizes this dynamic measurement of the student’s ILO progress - measured by the Bayesian knowledge tracing algorithm - to improve its recommendations, contributing to the ongoing personalisation of learning paths. In our evaluation, which utilized a test set of 129 mathematical tasks, the tested RL algorithms significantly outperformed a random baseline, underscoring the potential of this approach to enhance personalized learning within the realm of higher education mathematics.


Designing Digital Self-Assessment and Feedback Tools as Mentoring Interventions in Higher Education

April 2023

·

125 Reads

·

1 Citation

Mentoring in higher education fosters the development of students' competencies and skills very effectively. The individual support of self-regulated learning through formative assessment and feedback plays a vital role in mentoring processes (Butler & Winne, 1995; Schunk & Mullen, 2013). Nevertheless, providing formative assessment and feedback to all students is time-consuming for educators in traditional learning environments and nearly impossible in university lectures with large audiences. Digital tools for self-assessment and automated feedback offer the opportunity to upscale such mentoring interventions. However, developing and implementing effective digital tools for self-assessment and feedback is not a trivial task. There is a wide variety of self-assessment and feedback interventions in higher education. While, in general, they seem to impact learning positively, they do not necessarily improve learning outcomes. On the contrary, their positive effects depend on the concrete educational setting and the specific intervention implementation (Panadero et al., 2017). From a learning design perspective, the following question arises: What criteria must be considered when designing digital tools for self-assessment and feedback as mentoring interventions supporting students' self-regulated learning? This project contribution adds to answering this question. Following the paradigm of Design-Based Research (Reinmann, 2020), we aim at practically proven and theoretically reflected findings that help to improve educational practice. Firstly, we showcase four different implementations of digital self-assessment and feedback interventions in four different university courses. They vary in terms of the assessment tasks (closed vs. open questions), feedback strategies (reflective cues vs. corrective feedback), and duration (during vs. at the end of the semester) depending on the domain specific didactic settings. In a second step, we present the results from a qualitative evaluation survey we conducted in all four courses. We reflect on the mutual and contrasting findings in the light of theories of self-regulated learning and discuss the consequences we can draw from those reflections for the subsequent (re-)design process. While our considerations cannot conclude with a universal design formula, we present five preliminary guidelines for the development and implementation of digital self-assessment and feedback tools that support self-regulated learning in mentoring interventions in higher education. Keywords: educational technology, digital self-assessment, automated feedback, self-regulated learning, design-based research


Kompetenzerwerbsf\"orderung durch E-Assessment: Individuelle Kompetenzerfassung am Beispiel des Fachs Mathematik

August 2021

·

12 Reads

In this article, we present a concept of how micro- and e-assessments can be used for the mathematical domain to automatically determine acquired and missing individual skills and, based on these information, guide individuals to acquire missing or additional skills in a software-supported process. The models required for this concept are a digitally prepared and annotated e-assessment item pool, a digital modeling of the domain that includes topics, necessary competencies, as well as introductory and continuative material, as well as a digital individual model, which can reliably record competencies and integrates aspects about the loss of such.

Citations (1)


... Multi-criterion systems generate recommendations considering multiple conditions at the same time. For example, Pögelt et al. (2024) recommended learning objects which have the optimal balance between topic relevancy, difficulty suitableness, and novelty. The recommendation strategies in single-criterion systems are fixed and unable to meet various learning needs. ...

Reference:

Do personal recommendations need to be personalized? Investigating the relationships between student differences and educational recommendations
Individualised Mathematical Task Recommendations Through Intended Learning Outcomes and Reinforcement Learning
  • Citing Chapter
  • June 2024