Educational Measurement - Science topic

El presente ensayo hace un análisis de las diferentes medidas educativas en cuanto a la formación de los maestros en Bolivia durante los dos periodos de gobierno de Ismael Montes, 1904 a 1909 y 1913 a 1917, siendo este uno de los primeros que contribuyó en la consolidación de este sector. Montes es parte del proyecto educativo de los gobiernos libe...

Introduction Suture skills are important for family physicians, and they are best learned during residency. To increase the residents’ readiness for clinical procedures, we developed a self-directed online suturing training program that targets knowledge and suturing techniques with video submission, evaluation, and feedback. The program was built...

This study explores the effectiveness of the Little Iggy Center, a drop-off learning facility that supports both early childhood development and teacher training. Guided by a Mixed-Methods Program Evaluation Design, it embraces a convergent parallel approach to bring together quantitative metrics and qualitative insights. The research listens to th...

Student engagement assessment plays an important role in enhancing students’ positive performance and optimizing teaching methods. In this paper, a multimodal deep learning framework is proposed for student engagement assessment. Based on this framework, we propose a method for engagement assessment that utilizes data from three modalities: video,...

Individuals from marginalized backgrounds are underrepresented in clinical psychology doctoral programs. Admission processes can limit the diversification of incoming cohorts. This article aims to examine admission barriers and changes that may disproportionately affect applicants from marginalized backgrounds, and their perceived effectiveness fro...

During the COVID-19 pandemic, Communication PhD programs reconsidered the use of the Graduate Record Examination (GRE) amid concerns about structural inequities in graduate admissions. We examined whether GRE requirements, particularly minimum score thresholds, may contribute to filtering, a mechanism by which standardized criteria disproportionate...

Background It has been proposed that the school origin of items for cross-institutional Progress Tests (PTs) may introduce a bias in favour of students from the same school, posing a potential threat to the validity and reliability of PT results and cross-institutional comparisons. The aim of this study was to examine whether origin bias is present...

Evidence suggests a relationship between literacy and numeracy skills and cognitive abilities, which could be improved by augmented reality (AR) games. This study aimed to investigate to what degree the performance in a literacy and numeracy skills AR game explains associations between total reaction time (TRT) and educational achievement measures....

Background Virtual reality (VR) training requires the incorporation of teaching resources into the medical curriculum. This study aimed to explore whether a peer tutor would have an equal effect on learning as a clinical teacher. Approach Medical students at the University of Eastern Finland were invited to take part in a voluntary exercise. The e...

Background Chat Generative Pre-Trained Transformer (ChatGPT), launched by OpenAI in November 2022, features advanced large language models optimized for dialog. However, the performance differences between GPT-3.5, GPT-4, and GPT-4o in medical contexts remain unclear. Objective This study evaluates the accuracy of GPT-3.5, GPT-4, and GPT-4o across...

The series ‘Mastering your Fellowship’ provides examples of the question formats encountered in the written and clinical examinations, Part A of the FCFP (SA) examination. The series aims to help family medicine registrars (and supervisors) prepare for this examination. Model answers are available online.

Background Amid calls to transform nursing curricula to competency-based education, assessments to capture the intricacy and complexity of practice remains inconsistent. The ubiquitous integration of the objective structured clinical examination (OSCE) in evaluation systems necessitated a consolidative review to determine practices that garner its...

This study investigates educators' perceptions of the impact of artificial intelligence (AI)-)-generated outputs on student assessment in the Philippine educational context. With the rapid integration of AI technologies in education, understanding how educators view these tools is crucial for effective implementation. A descriptive quantitative res...

Introduction High-stakes exams significantly impact introductory physics students’ final grades and have been shown to be inequitable, often to the detriment of students identifying with groups historically marginalized in physics. Certain types of exam questions may contribute more than other types to the observed equity gaps. Objective The prima...

Social and teamwork skills are essential for today’s teachers, yet their assessment in authentic contexts is challenging. This study presents the design, development and validation of an innovative computer-based test, designed to assess collaborative and teamwork skills in preprimary and primary school teachers and referred to as the ‘CoTeSt’ (Col...

A Lei Geral de Proteção de Dados Pessoais, estabelece diretrizes para o tratamento de dados pessoais no Brasil, visando garantir a privacidade, a segurança e a transparência no uso dessas informações. Assim, o objetivo geral, busca analisar a importância da proteção dos dados pessoais de crianças e adolescentes no uso das mídias sociais, à luz da l...

Background Team-Based Learning (TBL) is a student-centered teaching strategy designed to improve problem-solving skills, knowledge, and practical abilities. Despite its increasing use in nursing education globally, limited research has explored its effectiveness in Vietnam. This study evaluates the impact of TBL on learning outcomes, accountability...

Introdução: O leite materno possui um alto teor nutritivo e é recomendado pela Organização Mundial da Saúde como forma exclusiva de alimentação até os seis meses de idade e, de maneira completar, até os dois anos de idade. A intenção de amamentar é um dos aspectos mais importantes associado a duração do aleitamento materno, visto que quanto maior f...

Background Direct observation, workplace-based assessments (WBAs) are a fundamental component of competency-based postgraduate medical education. In Australian general practice vocational training, external clinical teaching visits (ECTVs) are key observation-based WBAs. Traditionally, ECTVs are conducted face-to-face, but the COVID-19 pandemic saw...

Aim This study aimed to examine the influence of puzzle‐based teaching methods on nursing students' levels of knowledge, motivation for learning, and academic self‐efficacy. Methods The study employed a randomized controlled pretest‐posttest experimental design conducted between January and June 2024 at Atatürk University's Faculty of Nursing. A t...

Background: Life satisfaction serves as a preventive agent against various emotional, cognitive, and behavioral challenges, making it a crucial cognitive indicator of subjective well-being, particularly during adolescence. Accurately assessing life satisfaction is essential for understanding and promoting adolescent mental health, especially in app...

Introduction Video‐based learning has emerged as a powerful tool to overcome the well‐documented limitations of traditional lectures. Hence, the aim of this study was to investigate the impact of the implementation of a video animation on academic performance and student's opinion. Methods A video animation on biological mechanisms behind the link...

Collaboration is a critical skill for professionals in any field to master, and group work is a prominent component of many lab courses. However, there is conflicting guidance about the best method for forming groups to maximize performance and student experiences. Based on the benefits of cognitive diversity, we hypothesized that creating maximall...

This study used the default model of ChatGPT on health care information technologist exams in Japan to evaluate the reliability of this model for medical information knowledge in non-English languages. The average correct response rates for all questions in the health care, information technology, and health information systems fields were 86%, 93%...

O abandono de animais no Brasil é uma séria questão ambiental causada pela falta de conscientização e responsabilidade social. A prática resulta em superpopulação, propagação de doenças e impactos negativos no ecossistema, devido à ausência de políticas efetivas de controle populacional e educação sobre a guarda responsável. Na Universidade Federal...

Mathematics serves as a foundational discipline for academic success and career readiness, yet persistent disparities in student performance highlight the need for deeper insights into mathematical competencies. This study rigorously examines the proficiency levels of Grade 11 students in General Mathematics using the Center for Educational Measure...

Objetivo: Conhecer o comportamento informacional de pessoas idosas durante a infodemia de Covid-19. Método: Foi realizado um estudo qualitativo, que investigou uma amostra residente em Brasília (Distrito Federal, Brasil) por meio de entrevistas semiestruturadas, realizadas por telefone. As perguntas norteadoras foram desenvolvidas com base no model...

The COVID-19 pandemic led many to rely on videoconference software to continue working and learning. Although videoconferencing carries many benefits for communication, some question whether videoconferencing may increase attention to and concerns about physical appearance. The current research aimed to bridge this gap by using an experimental para...

Background Determining meaningful faculty and student program outcomes is important for demonstrating nursing program quality. For many faculty who are not experienced in program development and evaluation, outcomes specification can be a daunting task. Method This article draws on established measurement principles and accreditation standards fou...

Les communautés autochtones de l’Arctique canadien sont confrontées à d’importants problèmes de santé publique liés aux chiens. Au Nunavik, un territoire du Québec situé au nord du 55e parallèle, les morsures de chiens sont 4 à 40 fois plus fréquentes que dans le reste de la province. Ces morsures peuvent transmettre la rage, une zoonose mortelle,...

Background As part of Saudi Vision 2030, there is increasing demand for dentistry colleges to provide training in the student assessment process. Assessment is the process of accurately determining a learner’s skills across multiple educational domains. The objectives of this study were to investigate teachers’ perspectives, assessment challenges,...

This study assessed the impact of generative artificial intelligence (GenAI) usage on the academic outcomes of engineering students within the context of Yemen's unique socioeconomic challenges. This study employed a quantitative approach through a structured 5-point Likert-scale questionnaire, which was distributed to 277 engineering students from...

This symposium brings together cutting-edge research on cognitive diagnostic models (CDMs), a psychometric framework aimed at providing detailed insights into students' cognitive processes. CDMs are particularly useful in educational settings, offering fine-grained feedback on student strengths and weaknesses, which can guide targeted instruction a...

Background This study evaluated differences in orthopedic surgery residency applicant statistics in the 2024 National Resident Matching Program (NRMP) Charting Outcomes, the first report to include applicants with a pass/fail designation on the United States Medical Licensing Examination (USMLE) Step 1. Materials and Methods Data were collected fr...

Background: Depression, anxiety, and stress are growing public health concerns among university students worldwide, particularly those pursuing undergraduate studies. These psychological conditions can significantly affect academic performance, quality of life, and satisfaction with the educational environment. As higher education becomes increasin...

Objective To examine the health, education and social inequities experienced by children with deafblindness in low- and middle-income countries. Design Secondary analysis of 36 Multiple Indicator Cluster Surveys (2017–2020), using age-adjusted modified Poisson models to compare outcomes between children with and without deafblindness. Setting 36...

Background Educational institutions in Germany are facing the challenge of providing healthcare professionals with the skills they need to collaborate interprofessionally. Appropriate skills can be acquired through suitable teaching/learning formats and cross-cutting topics. However, there is a lack of empirical results on the design of appropriate...

Introductory bioinformatics courses can be challenging to teach. Students with a biological background may have never encountered computer science, and computer science students are likely to have minimal knowledge of biology. To improve learning, we implemented a flipped spaced-repetition course. We repeated the topics through various activities a...

Aim The aim of this study is to test and evaluate a one-time communication training session. Methods The focus of the evaluation is on the self-assessed communicative self-efficacy (SE-12) of students in the undergraduate degree programme in psychology before and after completing the communication training in the skills lab, during which simulated...

Background In the academic year 2020/21, alterations were introduced in the admission procedures for medical studies, particularly within the selection quota (AdH). These changes reduced the significance of school-leaving grades (the Abitur) in favor of the Test for Medical Studies (TMS) and considerations of professional training & voluntary servi...

Aim The aim of the study was to develop a scoring rubric that provides valuable feedback to students and to gather evidence for its construct validity. Methodology The Magdeburg Reflective Writing Feedback and Scoring Rubric (MaReS) was developed in an iterative process following a symposium on reflection by a committee of the “DACH Association fo...

Objective Trained student tutors instruct medical and dentistry students in the methodology of structured intraoral diagnosis, the identification of common oral pathologies, and the fundamental treatment concepts. The objective of this study is to assess the feasibility and efficacy of a peer teaching-based interdisciplinary tutorial and a training...

Objectives A strategy-based evaluation of new teaching or examination formats is recommended in order to check their suitability before a possible integration into a medical curriculum. The aim of the project was to evaluate an established formative competence- and simulation-based examination format for medical students in a standardized way. Pro...

Advanced Placement (AP) provides college-level courses to over 1 million US secondary students annually. Black, Hispanic, and female students have historically been underrepresented in AP Computer Science (CS). A new, broadly focused course—AP CS Principles—launched nationally in 2016–17 with the goal of increasing student participation and diversi...

Science process skills (SPS) are vital for enhancing student engagement, critical thinking, and academic achievement in science education. However, Ethiopian secondary schools often rely on traditional, rote-based laboratory methods that hinder SPS development. This study examined the effect of Guided Inquiry-Based Laboratory Experiments enriched I...

Introdução: Com a crescente integração de tecnologias de inteligência artificial (IA) em diversos domínios educacionais, avaliar sua eficácia torna-se primordial. Este estudo investiga a eficácia do ChatGPT, um modelo de IA conversacional de última geração, no âmbito do ensino de medicina. Objetivo: Analisar a produção científica do ensino de medic...

A multi-dimensional educational leadership measurement tool was conceptualized and developed for the modern era, spanned across four studies. Seventy-five items were developed under 12 dimensions using in-depth interviews (n = 12), followed by an expert validation study (n = 12). The remaining 48 items were examined using different quantitative tec...

Background and Need for Innovation There is a large body of evidence that assessment systems in medical education are inequitable for many groups of learners. A common approach to improve equity has been the use of organizational strategies, where training program leaders work to develop and implement improvements in existing assessment systems fro...

Purpose To validate the internal structure of the Speech-Language Pathology Script Concordance Test (FonoTCS), which will be developed in a virtual, open-access format, to be used in the assessment of clinical reasoning among young professionals and students of speech-language pathology with a generalist background, speakers of Brazilian Portuguese...

Objectives: This study aims to assess the overall performance of ChatGPT version 4-omni (GPT-4o) on the Turkish Orthopedics and Traumatology Board Examination (TOTBE) using actual examinees as a reference point to evaluate and compare the performance of GPT-4o with that of human participants. Materials and methods: In this study, GPT-4o was test...

Data visualizations play a crucial role in communicating patterns in quantitative data, making data visualization literacy a key target of STEM education. However, it is currently unclear to what degree different assessments of data visualization literacy measure the same underlying constructs. Here, we administered two widely used graph comprehens...

This research aimed to investigate and develop an online system for educational measurement and evaluation with a focus on authentic assessment. The research process consisted of four main stages: (1) analyzing the current state and carrying out qualitative interviews on authentic assessment, (2) developing an online measurement and evaluation prog...

Background Standard setting is widely practised in healthcare education programmes and specialty examinations in many countries. However, Malaysian dental institutions still arbitrarily set a fixed 50% pass-fail assessment threshold. The purpose of this mixed-methods study was to explore faculty members’ experiences and practices in student assessm...

This study investigates the effectiveness of interactive teaching methods on students' academic performance in Social Studies and Civic Education in Nigerian secondary schools. The study adopted a quasi-experimental research design with a control group. The population consisted of 85,121 Junior Secondary School students from public secondary school...

O câncer de pele, incluindo os tipos melanoma e não melanoma, é uma preocupação crescente, especialmente entre os idosos na região sul do Brasil, devido à exposição crônica ao sol e outros fatores de risco. A prevalência dessas condições malignas está associada principalmente à exposição excessiva aos raios ultravioleta (UV), destacando a importânc...

This study investigates how curriculum design for agricultural science education in Nigeria incorporates evaluation. The study acknowledges the value of efficient assessment techniques in improving field education outcomes. The study aims to investigate several methods of evaluation, pinpoint obstacles, and provide suggestions. A conceptual framewo...

Introduction Tooth preparations have traditionally been graded using a visual system that either subjectively assigns a single mark (global grading) or objectively divides the grade into several smaller categories. (analytical grading using rubrics). The objective of this study is to determine how effectively rubrics work for evaluating dental stud...

Introducción. La validación de instrumentos de investigación es fundamental para asegurar la validez y confiabilidad de los resultados, lo que a su vez respalda la calidad de los estudios. Objetivo. Realizar un análisis psicométrico de la dimensión teórica de la «Prueba para evaluar las competencias en investigación» (PECI) y proponer una versión m...

Introduction: Referral to a probation centre is still rarely adjudicated, i.e. against about 2% of juveniles, mainly those showing signs of demoralization (about 70%). So far, the activity of the centres has confirmed the effectiveness in working with minors, as well as lower cost of their stay in the centre, compared to 24-hour facilities, which m...

Introduction: the training of future graduates from medical schools for responsible and qualified health care practice is a significant challenge. The goal is for them to be equipped to solve problems that require higher-order cognitive skills. Thus, evaluating the acquisition of such competencies becomes crucial. One assessment method that has bee...

Az orvosképzés hagyományosan elméleti és klinikai tanulmányokra válik szét, azonban egyre több orvosképző helyen jelenik meg a kettőt egymáshoz közelítő integrált kurrikulum. Az elméleti tudás stabil megszerzése kiemelten fontos, mivel ez alapozza meg a klinikai tantárgyakat, illetve a diplomaszerzés után a mindennapi klinikai gondolkodást és dönté...

Objective The aim of this study was to evaluate and compare the performance of the artificial intelligence (AI) models ChatGPT-3.5, ChatGPT-4, and Gemini on the Turkish Specialization Training and Development Examination (UEGS) to determine their utility in medical education and their potential to improve patient care. Methods This retrospective s...

O presente trabalho teve como objetivo investigar a relação da qualidade do ar dos municípios de Canoas e Esteio/RS com os casos notificados da COVID-19 no período de 1º de março de 2020 a 28 de fevereiro de 2021. A análise ocorreu por meio da busca e relação de dados de qualidade do ar para NO2, SO2, MP10 e MP2,5 obtidos do banco de dados da Funda...

Purpose This study aimed to identify the trends in scholarly research related to patient safety education programs and outcome evaluations in the field of nursing education, focusing on providing a comprehensive overview of the current state of patient safety education and suggesting directions for future program development and evaluation. Method...

This study aims to measure the validity and reliability of the Digital Education Measurement Instrument (DEMI) in assessing the level of digital education among teachers. Digital education encompasses aspects of knowledge, skills, values, and the integration of technology in teaching and learning (T&L). Although technology is increasingly being int...

This editorial introduces the second part of CEJEME's Special Issue on Artificial Intelligence and Machine Learning in Educational Measurement. Building on the foundational discussions in Part 1, this installment further explores the evolving role of AI and ML in assessment, evaluation, and learning analytics. The four articles in this issue examin...

The ‘Mastering your Fellowship’ series provides examples of the question format encountered in the written and clinical examinations for the Fellowship of the College of Family Physicians of South Africa (FCFP [SA]) examination. The series is aimed at helping family medicine registrars prepare for this examination.

Background The rapid development of information and technology and the spread of COVID-19 have made online learning and assessment more prevalent. In this context, the potential of multimedia-assisted items to provide an alternative method of learning and evaluation is significant. These items can better reflect authentic clinical situations and en...

BACKGROUND Artificial intelligence has been shown to achieve successful outcomes in various orthopedic qualification examinations worldwide. This study aims to assess the performance of ChatGPT in the written section of the Turkish Orthopedics and Traumatology Board Examination, compare its results with those of candidates who took the exam, and de...

Article history: Achievement tests are designed to provide a clear picture of students' proficiency levels. However, the perspective that focuses on judging degree of their understanding based on comparisons with their peers or classmates became limited, as it does not reflect extent to which these students possess trait measured or their mastery o...

Background The drastic evolution in digital technologies has significantly transformed educational and learning processes, necessitating faculty members to incorporate these advancements into their teaching strategies. A research gap exists in investigating the use and acceptance of these technologies in nursing education. Thus, this research aims...

Background Case studies have shown ChatGPT can run clinical simulations at the medical student level. However, no data have assessed ChatGPT’s reliability in meeting desired simulation criteria such as medical accuracy, simulation formatting, and robust feedback mechanisms. Objective This study aims to quantify ChatGPT’s ability to consistently fo...

The “First Year Experience” is a critical component of retention of STEM majors. Often, general chemistry has been labeled as a “gatekeeper” course for STEM careers due to a high attrition rate and a course that leads to increased time for graduation when students are inadequately prepared. We demonstrate that the active learning strategy Peer-Led...

BACKGROUND Due to their chemical nature as fluorinated hydrocarbon compounds, volatile anesthetics are highly potent greenhouse gases, with desflurane having by far the largest CO 2 -equivalent (CO 2 e) footprint. In everyday clinical practice, the CO 2 e footprint can easily be reduced through the increased use of propofol or sevoflurane as well a...

Introduction Objective Structured Clinical Examinations (OSCEs) are crucial in assessing clinical competencies, but their implementation varies widely across medical schools. This work examines OSCE practices across Canadian medical schools, focusing on frequency, type, and timing. Methods A survey was conducted among all 17 Canadian medical schoo...

Massive Open Online Course (MOOC) has gained widespread adoption across diverse educational domains and plays a crucial role in advancing educational equality. Nevertheless, skepticism surrounds the effectiveness of MOOC due to their notably low completion rates. To explore the outcomes of MOOC adoption in higher education and improve its applicati...

Objective The study aimed to evaluate the top large language models (LLMs) in validated medical knowledge tests in Portuguese. Methods This study compared 31 LLMs in the context of solving the national Brazilian medical examination test. The research compared the performance of 23 open-source and 8 proprietary models across 399 multiple-choice que...

As a new generation of assessment instrument, cognitive diagnosis integrates the measurement objectives into the cognitive model to diagnose the fine-grained knowledge of students. Taking the PISA 2012 dataset in mathematics from Shanghai, Hong Kong, Macau and Taiwan as the research subject, this study constructed a cognitive model with the attribu...

Objective As dental implanting becomes an increasing demand among patients with tooth loss, an efficient and effective training for students is to be necessary. In this case, we anticipate the possible application of virtual reality (VR) technology to pre-clinical implantation training (PCIT) in order to improve the students’ learning efficiency an...

Background Student development is currently taking place in an environment governed by new technologies and social media. Some platforms, such as Instagram or X (previously known as “Twitter”), have been incorporated as additional tools for teaching and learning processes in higher education, especially in the framework of image-based applied disci...

Introduction: Evaluation of students is a crucial aspect of teaching and learning in higher education, particularly in nursing education. Effective assessment methods should evaluate cognitive, psychomotor, and communication skills while promoting professionalism. This study aimed to examine the perceptions of nursing students and faculty members r...

Background Measurement invariance analysis is a critical step in evaluating the structural validity of measurement instruments, ensuring that a scale measures its intended constructs equally across different subgroups. Such analyses are essential before making subgroup comparisons. This study investigated the measurement invariance of the Self-Regu...

Introduction Medical students may arrive at medical school with some research background but not necessarily evidence-based medicine (EBM) skills. First-year preclinical medical students require foundational skills for EBM (formulating background and foreground questions, navigating information sources, and conducting database searches) before crit...

The Royal College of Physicians and Surgeons of Canada (RCPSC) plays a leading role in specialty and subspecialty post graduate medical education (PGME) in Canada. As the RCPSC accredits PGME programs, these programs are structured to meet the RCPSC Competence by Design model and their CanMEDS roles. RCPSC Certification is required by Medical Regul...

Background The creation of high-quality multiple-choice questions (MCQs) is essential for medical education assessments but is resource-intensive and time-consuming when done by human experts. Large language models (LLMs) like ChatGPT-4o offer a promising alternative, but their efficacy remains unclear, particularly in high-stakes exams. Objective...

Background Recent advancements in artificial intelligence, such as GPT-3.5 Turbo (OpenAI) and GPT-4, have demonstrated significant potential by achieving good scores on text-only United States Medical Licensing Examination (USMLE) exams and effectively answering questions from physicians. However, the ability of these models to interpret medical im...

Background Owing to the infrequent emergence of disasters and the challenges associated with their management, responders need appropriate training beyond doubt. Ensuring the highest standard of disaster management (DM) training is of paramount importance for high-quality DM. However, the literature concerning DM training monitoring and evaluation...

Background Studies have highlighted inadequate exposure to musculoskeletal education and orthopedic surgery in mandatory medical school curricula; thus, the Canadian Orthopaedic Surgery Medical Education Course (COSMEC) was designed to enhance medical education around orthopedic surgery and common musculoskeletal presentations encountered in primar...

The emergence of ChatGPT and similar new Generative AI tools has created concern about the validity of many current assessment methods in higher education, since learners might use these tools to complete those assessments. Here we review the current evidence on this issue and show that for assessments like essays and multiple-choice exams, these c...

Introduction Students have ranked head and neck anatomy as one of the most complex areas to learn as it contains highly detailed spaces. Incorporating a head and neck escape room activity into anatomy curricula provides an engaging and effective way for the students to review these spaces and their relationships. Methods We piloted a head and neck...

Background Potential benefits of enhancing nursing education using augmented reality (AR) invite further investigation of its applications. This study evaluated an AR system augmenting classes on complex cardiac problems in an undergraduate didactic nursing course. Method This study used a mixed-methods, quasiexperimental, and non-concurrent contr...