Measuring the educational environment in health professions studies: A systematic review
ABSTRACT One of the determinants of the medical student's behaviour is the medical school learning environment.
The aim of this research was to identify the instruments used to measure the educational environment in health professions education and to assess their validity and reliability.
We performed an electronic search in the medical literature analysis and retrieval system online (MEDLINE) and Timelit (Topics in medical education) databases through to October 2008. The non-electronic search (hand searching) was conducted through reviewing the references of the retrieved studies and identifying the relevant ones. Two independent authors read, rated and selected studies for the review according to the pre-specified criteria. The inter-rater agreement was measured with Kappa coefficient.
Seventy-nine studies were included with the Kappa coefficient of 0.79, which demonstrated a reliable process, and 31 instruments were extracted. The Dundee Ready Education Environment Measure, Postgraduate Hospital Educational Environment Measure, Clinical Learning Environment and Supervision and Dental Student Learning Environment Survey are likely to be the most suitable instruments for undergraduate medicine, postgraduate medicine, nursing and dental education, respectively.
As a valid and reliable instrument is available for each educational setting, a study to assess the educational environment should become a part of an institution's good educational practice. Further studies employing a wider range of databases with more elaborated search strategies will increase the comprehensiveness of the systematic review.
- SourceAvailable from: Cristian A Herrera[Show abstract] [Hide abstract]
ABSTRACT: Background: Students' perceptions of their educational environment (EE) have been studied in undergraduate and postgraduate curricula. Postgraduate EE has been measured in hospital settings. However, there are no instruments available to measure the EE in postgraduate ambulatory settings. Aim: The aim of this study was to develop the "Ambulatory Care Learning Education Environment Measure" (ACLEEM). Methods: A mixed methodology was used including three stages: (1) Grounded theory (focus groups); (2) Delphi technique to identify consensus; and (3) Pilot study. Results: Three quota samples of approximately 60 stakeholders were formed, one as Focus Groups and two as Delphi panels. Eight focus groups were carried out including 58 residents (Latin-American Spanish speakers). The results were analysed and 173 items were offered to a National Delphi panel (61 residents and teachers). They reduced in two rounds the number of important items to 54. The 54-item questionnaire was then piloted with 63 residents and refined to the final version of the ACLEEM with 50 items and three domains. Conclusions: The 50-item inventory is a valid instrument to measure the EE in postgraduate ambulatory setting in Chile. Large-scale administration of the ACLEEM questionnaire to evaluate its construct validity and reliability are the next steps to test the psychometric properties of the instrument.Medical Teacher 09/2012; 35(1). DOI:10.3109/0142159X.2012.714882 · 2.05 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: The drive to quality-manage medical education has created a need for valid measurement instruments. Validity evidence includes the theoretical and contextual origin of items, choice of response processes, internal structure, and interrelationship of a measure's variables. This research set out to explore the validity and potential utility of an 11-item measurement instrument, whose theoretical and empirical origins were in an Experience Based Learning model of how medical students learn in communities of practice (COPs), and whose contextual origins were in a community-oriented, horizontally integrated, undergraduate medical programme. The objectives were to examine the psychometric properties of the scale in both hospital and community COPs and provide validity evidence to support using it to measure the quality of placements. The instrument was administered twice to students learning in both hospital and community placements and analysed using exploratory factor analysis and a generalizability analysis. 754 of a possible 902 questionnaires were returned (84% response rate), representing 168 placements. Eight items loaded onto two factors, which accounted for 78% of variance in the hospital data and 82% of variance in the community data. One factor was the placement learning environment, whose five constituent items were how learners were received at the start of the placement, people's supportiveness, and the quality of organisation, leadership, and facilities. The other factor represented the quality of training-instruction in skills, observing students performing skills, and providing students with feedback. Alpha coefficients ranged between 0.89 and 0.93 and there were no redundant or ambiguous items. Generalisability analysis showed that between 7 and 11 raters would be needed to achieve acceptable reliability. There is validity evidence to support using the simple 8-item, mixed methods Manchester Clinical Placement Index to measure key conditions for undergraduate medical students' experience based learning: the quality of the learning environment and the training provided within it. Its conceptual orientation is towards Communities of Practice, which is a dominant contemporary theory in undergraduate medical education.Advances in Health Sciences Education 01/2012; 17(5). DOI:10.1007/s10459-011-9344-x · 2.71 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: The teachers' perspectives of the educational environment have as yet only been sparsely considered. This study aimed at validating the first German version of the Dundee Ready Education Environment Measure (DREEM) from the points of view of both students and teachers. Data from 1119 students and 258 teachers were available for analysis. Psychometric validation included the analysis of homogeneity and discrimination at item level as well as reliability (Cronbach's α), criterion and construct validity at test level. Effect sizes were calculated and the independent samples t-test was used for statistical inference testing of mean differences between two groups. Item characteristics were satisfactory in both samples. Reliability was high with α = 0.92 (students) and 0.94 (teachers), respectively. Factor analyses revealed five dimensions which slightly diverged from the five subscales postulated by the DREEM authors though. The environment was evaluated significantly (p < 0.001) more positively by teachers (M = 117.63) than by students (M = 109.75). Further significant differences were observed with regard to gender, mother language, stage of studies and previous professional training among others. With convincing psychometric properties at item and test levels, the suitability of DREEM not only for students but also for teachers to assess the educational environment has been demonstrated.Medical Teacher 11/2011; 33(11):e624-36. DOI:10.3109/0142159X.2011.610841 · 2.05 Impact Factor