Nikki Yonts’s research while affiliated with Oklahoma State University and other places

We tested the ability of generative AI (GAI) to serve as a non-expert grader in the context of school-wide curriculum assessment. OpenAI's ChatGPT-4o Large Language Model was used to create diverse student personas. Synthetic artefacts based on exam questions from two Undergraduate courses on religion and philosophy were created. A custom GPT model, Claude 3.5 and human non-expert educators were used to evaluate the artefacts based on a rubric. We performed a variety of different statistical tests on the results to identify correlations and patterns within the GAI graders and between human and GAI graders. We found that different GAI models grade reliably, and that GAI and human grades differ significantly. We identified systemic bias of the GAIs towards synthetic, GAI-generated artefacts. We find that the use of synthetic artefacts to validate GAI graders was unsuitable, and we identify different directions for further study. Despite certain limitations, we conclude that the application of GAI-based grading in curriculum assessment represents a viable and promising use of AI to offer an intermediate evaluation perspective on student work.

Presentation at the 17th annual Int Conf of Education, Research, and Innovation, 11-13 Nov 2024, Seville, Spain, given by Cristian Del Gobbo. See paper: https://doi.org/10.21125/iceri.2024.0426

To describe an inquiry-based learning activity and discuss the advantages of using such activities to demonstrate research design to undergraduate psychology students the authors presented students with the research problem of eyewitness memory recall and asked students to solve the problem, using the information provided by the course. Working in small groups, students reviewed eyewitness studies and designed their own research materials, including a videotaped theft and a CD-ROM version of the eyewitness survey and potential suspect line-up. Poststudy interviews showed that the students enjoyed the activity and felt that they had a better understanding of experimental research design than they had before the exercise. It is proposed that this activity may be a useful hands-on tool for instructors who have difficulty teaching students fundamental topics such as literature reviews, hypothesis testing, reliability and validity of research measures, and experimental design.

To describe an inquiry-based learning activity and discuss the advantages of using such activities to demonstrate research design to undergraduate psychology students the authors presented students with the research problem of eyewitness memory recall and asked students to solve the problem, using the information provided by the course. Working in small groups, students reviewed eyewitness studies and designed their own research materials, including a videotaped theft and a CD-ROM version of the eyewitness survey and potential suspect line-up. Poststudy interviews showed that the students enjoyed the activity and felt that they had a better understanding of experimental research design than they had before the exercise. It is proposed that this activity may be a useful hands-on tool for instructors who have difficulty teaching students fundamental topics such as literature reviews, hypothesis testing, reliability and validity of research measures, and experimental design.

Typescript (lithograph copy). Thesis (M.S.)--Oklahoma State University, 2000. Vita. Includes bibliographical references (leaves 48-53).