Roy Meissner’s research while affiliated with Leipzig University and other places

In this article, we explore the transformative impact of advanced, parameter-rich Large Language Models (LLMs) on the production of instructional materials in higher education, with a focus on the automated generation of both formative and summative assessments for learners in the field of mathematics. We introduce a novel LLM-driven process and application, called ItemForge, tailored specifically for the automatic generation of e-assessment items in mathematics. The approach is thoroughly aligned with the levels and hierarchy of cognitive learning objectives as developed by Anderson and Krathwohl, and takes specific mathematical concepts from the considered courses into consideration. The quality of the generated free-text items, along with their corresponding answers (sample solutions), as well as their appropriateness to the designated cognitive level and subject matter, were evaluated in a small-scale study. In this study, three mathematical experts reviewed a total of 240 generated items, providing a comprehensive analysis of their effectiveness and relevance. Our findings demonstrate that the tool is proficient in producing high-quality items that align with the chosen concepts and targeted cognitive levels, indicating its potential suitability for educational purposes. However, it was observed that the provided answers (sample solutions) occasionally exhibited inaccuracies or were not entirely complete, signalling a necessity for additional refinement of the tool's processes.

Zusammenfassung Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies, existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work. Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs. Zusammenfassung. Wissensgraphen (englisch Knowledge Graphs , KGs), bieten uns eine strukturierte, flexible, transparente, systemübergreifende und kollaborative Möglichkeit, unser Wissen und unsere Daten über verschiedene Bereiche der Gesellschaft und der industriellen sowie wissenschaftlichen Disziplinen hinweg zu organisieren. KGs übertreffen jede andere Form der Repräsentation in Bezug auf die Effektivität. Die Entwicklung von Wissensgraphen (englisch Knowledge Graph Engineering , KGE) erfordert jedoch fundierte Erfahrungen mit Graphstrukturen, Webtechnologien, bestehenden Modellen und Vokabularen, Regelwerken, Logik sowie Best Practices. Es erfordert auch einen erheblichen Arbeitsaufwand. In Anbetracht der Fortschritte bei großen Sprachmodellen (englisch Large Language Modells , LLMs) und ihren Schnittstellen und Anwendungen in den letzten Jahren haben wir umfassende Experimente mit ChatGPT durchgeführt, um sein Potenzial zur Unterstützung von KGE zu untersuchen. In diesem Artikel stellen wir eine Auswahl dieser Experimente und ihre Ergebnisse vor, um zu zeigen, wie ChatGPT uns bei der Entwicklung und Verwaltung von KGs unterstützen kann.

Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies, existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work. Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs.

Knowledge Graphs (KG) provide us with a structured, flexible , transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies , existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work. Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs.

In addition to formal learning at universities, like in lecture halls and seminar rooms, students are regularly confronted with self-study activities. Instead of being left to their own devices, students might benefit from a proper design of such activities, including pedagogical interventions. Such designs can increase the degree of activity and the contribution of self-study activities to the achievement of learning outcomes. Especially in times of a global pandemic, self-study activities are increasingly executed at home, where students already use technology-enhanced materials, processes, and digital platforms. Thus we pick up these building blocks and introduce TecCoBot within this paper. TecCoBot is not only a chatbot, supporting students in reading texts by offering writing assignments and providing automated feedback on these, but also implements a design for self-study activities, typically only offered to a few students as face-to-face mentoring.

Like most curricula in the humanities and social sciences, the curriculum of pre-service teacher training in educational sciences often includes time-consuming reading and writing tasks, which require high quality support and feedback in a timely manner. A well-known way to provide this support to students is one-to-one mentoring. However, limited time and resources in the German university context require to effectively scale the benefits of individual feedback. The use of scalable technologies to support learning processes seems to be promising, but its development usually requires a deep technical understanding. With an interdisciplinary approach, this contribution investigates how personal mentoring can be made available to as many students as possible, taking into account the didactic, organizational and technical frameworks at universities. We describe the development and implementation process of two chatbots that both aim to support students of educational sciences in their self-study of the seminar topics and literature. The chatbots were used by over 700 students during the course of 1 year and our evaluations show promising results that bear the potential to improve the availability of digital mentoring support for all students.

EAs.LiT is an e-assessment management and analysis software for which contextual requirements and usage scenarios changed over time. Based on these factors and further development activities, the decision was made to adopt a microservice architecture for EAs.LiT version 2 in order to increase its flexibility to adapt to new and changed circumstances. This architectural style and a few adopted technologies, like RDF as a data format, enabled an eased implementation of various use cases. Thus we consider the microservice architecture productive and recommend it for usage in other educational projects. The specific architecture of EAs.LiT version 2 is presented within this article, targeting to enable other educational projects to adopt it by using our work as a foundation or template.

The classification of e-assessment items with levels of Bloom’s taxonomy is an important aspect of effective e-assessment. Such annotations enable the automatic generation of parallel tests with the same competence profile as well as a competence-oriented analysis of the students’ exam results. Unfortunately, manual annotation by item creators is rarely done, either because the used e-learning systems do not provide the functionality or because teachers shy away from the manual workload. In this paper we present an approach for the automatic classification of items according to Bloom’s taxonomy and the results of their evaluation. We use natural language processing techniques for pre-processing from four different NLP libraries, calculate 19 item features with and without stemming and stop word removal, employ six classification algorithms and evaluate the results of all these factors by using two real world data sets. Our results show that 1) the selection of the classification algorithm and item features are most impactful on the F1 scores, 2) automatic classification can achieve F1 scores of up to 90% and is thus well suited for a recommender system supporting item creators, and 3) some algorithms and features are worth using and should be considered in future studies.