No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Diseño de una rúbrica para evaluar las competencias del Prácticum del Grado en Fisioterapia. Percepción de su utilidad por los estudiantes
Abstract
Aim
To present the design of 3 rubrics to evaluate the clinical practices of the Practicum course of the Physiotherapy Degree and to know the students’ perception on the usefulness of these tools.
Material and method
The rubrics were developed according to the cost-sectional and specific competences included in the Practicum course of Degree in Physiotherapy at the Universidad Europea de Madrid, and adapted to the requirements of Royal Decree 1707/2011 and graduate profile requirements given by ANECA. The students’ perception was evaluated through an 11-item questionnaire. The students had to show their level of agreement or disagreement (Likert scale 1-4) on the usefulness of the rubric used to assess the students’ performance during the clinical practices, verifying their level of competency acquired.
Results
The evaluable criteria and achievement levels of the 3 rubrics designed (Clinical Practice, Clinical Records in Physiotherapy and Academic Reflective Diary) are presented. The questionnaire results showed that students’ perceptions about the rubrics were positive. In 8 of the 11 evaluated items, the most frequent response was “medium-high level of agreement”.
Conclusions
The 3 rubrics presented provide an assessment of the competencies included in the clinical practices of Physiotherapy Degree. The students’ perceptions were positive in aspects as the usefulness for evaluation, in allowing a more objective evaluation, comprehensively assessing and verifying the level of competences.
... In clinical practice, competence assessment is based on the direct tutors' observation of students' performance providing care to real patients in the healthcare environment. In this way, the assessment criteria should be explicit, specific, and related to the professional profile [23,24]. ...
... For this purpose, the RECOPC-FIS I (Rubric for Competence assessment in Clinical Physiotherapy Practices) was created in Spain [24] through a multicenter study conducted in the community of Madrid, with the participation of eleven universities and thirteen healthcare centers. Based on this proposal, it was possible to improve the internal validity of the tool through a committee of experts, who proposed expanding the number of items to be assessed, thus generating the RECOPC-FIS II. ...
... The feasibility, pragmatism and lack of ambiguity provided by the use of our instrument will have a positive impact on both the rigorous assessment of students and the elimination of the subjective nature of observation-based assessment. Furthermore, the 14 evaluable criteria ensure an integral competence assessment for Health Sciences students in the real clinical context [12,35,37,[43][44][45][46] and particularly in what is considered in Royal Decree 592/2014 in the Degree of Physiotherapy in Spain [24,45]. Moreover, they evaluate the most relevant transversal competences considered by other authors in the academic scope [17,47]. ...
Introduction
Competence-based learning must be integrated into the practical development of Physiotherapy. Teamwork, interpersonal relations, analytical skills or critical/clinical thinking are some examples of internationally recommended competences in this kind of university studies. Therefore, there is a need to evaluate this learning in Physiotherapy through valid tools that facilitate this task.
Objective
To analyze the psychometric properties according to competences in Clinical Practices (RECOPC-FIS II) in order to assess 14 transversal or universal competences of under-graduate students in the Physiotherapy degree.
Methods
A validation study was conducted with 197 students in the 3rd and 4th year of the Physiotherapy degree and 202 clinical tutors who assessed these students using the RECOPC-FIS II. Different psychometric properties were analyzed: factor structure, internal consistency and sensitivity to change.
Results
The RECOPC-FIS II has a high internal consistency. Its 14 items saturate in a single factor. Regarding the sensitivity to change, the rubric showed higher scores in the Practicum of the 4th year with respect to that of the 3rd year, reaching significant differences in all of them.
Conclusion
The RECOPC-FIS II is a valid and reliable instrument to assess the transversal competencies of undergraduate students of Physiotherapy during their clinical practice. Therefore, it is intended to facilitate the acquisition of essential skills for the development of their professional career. The flexibility of this tool would allow its adaptation to other health science courses.
... En los últimos años, el uso de rúbricas para fines formativos ha ido tomando protagonismo (Hack, 2015;Panadero & Jonsson, 2013), al tiempo que proporciona un método fiable (Huerta et al., 2014) y objetivo para evaluar el trabajo del estudiante (Mok & Toh, 2015) y facilitar la valoración de las competencias a desarrollar por este (Knight, 2006;Martiañez et al., 2015;O'Brien, Franks & Stowe, 2008). Asimismo, las rúbricas aportan transparencia a la evaluación (Contreras et al., 2016), permiten mejorar las calificaciones del alumnado (Laurian & Fitzgerald, 2013;McGoldrick & Peterson, 2013), así como la calidad de sus aprendizajes (Sundeen, 2014). ...
... Son numerosos los estudios que reflejan los beneficios del uso de las rúbricas en educación Entre ellos, destacamos los hallazgos que obtuvieron Timmerman, Strickland, Johnson y Payne (2011), quienes llegaron a la conclusión de que estas herramientas aportan tres ventajas importantes en la educación superior: aumentar la objetividad de las calificaciones, asegurar la validez en la evaluación del rendimiento del estudiante y, cuando se utiliza en múltiples cursos, proporcionar una medida común para evaluar en qué grado el plan de estudios está cumpliendo sus objetivos; asimismo, Espinosa (2013) demostró que las rúbricas facilitan una evaluación formativa con retroalimentación, que mejoran las calificaciones del grupo y que permiten una evaluación imparcial y objetiva; también Rekalde y Buján (2014) concluyeron en su estudio que las rúbricas permiten valorar lo que se ha conseguido y lo que falta por conseguir, dotar de sentido global a los conocimientos, habilidades y actitudes, contrastar procesos y resultados, aportar evidencias a partir de criterios conocidos, disponer de feed-back inmediato y convertir el uso de dichas herramientas en una actividad habitual con sentido ético; por último, Martiañez et al. (2015) hallaron que las rúbricas permiten una valoración más objetiva, evaluar de forma integral y conocer el nivel de competencia de los estudiantes. ...
... 370). Por último, los estudiantes también concretan como ventaja de estas herramientas que las mismas potencian la objetividad, la justicia y la igualdad en los procesos evaluativos (Martiañez et al., 2015). ...
The current educational model of training requires that all elements of the curriculum revolve around students' development of competences. Also, the assessment should be directed to check the level of achievement of the required competences. For this purpose, a variety of data collection instruments are necessary in order to reveal the diversity of competences to acquire. Rubrics are one of the main tools used for this purpose. So in this investigation we decided to analyze the grade of adequacy of these assessment tools from the perception of university students. By applying a non-experimental descriptive method, we applied a self-development questionnaire to 440 students of Pedagogy. The results indicate good validity and reliability of the instrument and stablish the rubrics have high advantages over the possible disadvantages associated.
... In addition, the analysis of access to rubrics and their use shows that they promote self-regulated learning, foster students' self-assessment of their progress and online feedback, and significantly and positively correlate with academic results. Martiañez et al. [62] analyze undergraduate students' perceptions of the usefulness of three rubrics (clinical histories, clinical cases, and reflexive diaries) that evaluate the competencies of the clinical PT internship. They conclude that students view rubrics as moderately useful, that providing rubrics to students does not guarantee that they will perceive them as valid and use them to learn, and that rubrics should be considered basic referents throughout the teaching-learning process. ...
One of the main challenges faced by physical therapy (PT) students is to learn the practical skills involved in neurological physical therapy (PT). To help them to acquire these skills, a set of rubrics were designed for formative purposes. This paper presents the process followed in the creation of these rubrics and their application in the classroom, noting that students perceived them as valid, reliable, and highly useful for learning. The perception of the validity and usefulness of the rubrics has different closely related dimensions, showing homogeneous values across the students´ sociodemographic and educational variables, with the exception of dedication to studying, which showed a significant relationship with schoolwork engagement and course satisfaction. The adequacy of the hypothesized structural model of the relationships among the variables was confirmed. Direct effects of the perception of the rubrics’ validity and engagement on course satisfaction were found, as well as direct effects of the assessment of the usefulness of the rubrics on schoolwork engagement and indirect effects on course satisfaction through this latter variable. The results are discussed taking into account the conclusions of previous research and different instructional implications.
... Competency-based education has thus been proposed as a means to optimize the preparation of health professionals [4]. The competency-based approach is focused on outcomes related to the skills, knowledge, and attitudes of graduates [5] that will allow them to work as competent professionals at the national or international level [6,7]. Competence-based learning is based on the capacity and responsibility of each student and on the development of the student's autonomy [2]; it requires specific learning methodologies, monitoring, and tutoring as well as competency-based assessment methods [8]. ...
Background:
The evaluation of competencies in the clinical field is essential for health professionals, as it allows the acquisition of these competencies to be tracked. The objective of this study was to create and evaluate the validity and reliability of a tool for measuring clinical competencies in physical therapy (PT) students to assess the quality of their performance in a professional context.
Methods:
A descriptive study was designed. The Measurement Tool for Clinical Competencies in PT (MTCCP) was developed based on the evaluation of 39 experts: 15 clinicians and 24 instructors. The content validity was evaluated using the Content Validity Index (CVI). Three professors were invited to apply the tool to 10 students. Cronbach's alpha, exploratory factor analysis, and the intraclass correlation coefficient were used to determine the reliability and validity of the scale.
Results:
The CVI was positive-higher than 0.8. Principal component analysis confirmed the construct validity of the tool for two main factors: clinical reasoning (first factor) and professional behavior (second factor). With regard to reliability, the MTCCP achieved an internal congruence of 0.982. The inter-evaluator reproducibility for clinical reasoning, professional behavior, and the total MTCCP score was almost perfect; the ICCs were 0.984, 0.930, and 0.983, respectively.
Conclusions:
The MTCCP is a valid and reliable instrument for assessing the performance of PT students in hospital settings and can be used to determine what skills students feel less confident using and what additional training/learning opportunities could be provided. Further research is needed to determine whether the MTCCP has similar validity and reproducibility in other Spanish-speaking national and international PT programs.
Background
The development of procedural skills is essential in health sciences education. Rubrics can be useful for learning and assessing these skills. To this end, a set of rubrics were developed in case of neurophysiotherapy maneuvers for undergraduates. Although students found the rubrics to be valid and useful in previous courses, the analysis of the practical exam results showed the need to change them in order to improve their validity and reliability, especially when used for summative purposes. After reviewing the rubrics, this paper analyzes their validity and reliability for promoting the learning of neurophysiotherapy maneuvers and assessing the acquisition of the procedural skills they involve.
Methods
In this cross-sectional and psychometric study, six experts and 142 undergraduate students of a neurophysiotherapy subject from a Spanish university participated. The rubrics’ validity (content and structural) and reliability (inter-rater and internal consistency) were analyzed. The students’ scores in the subject practical exam derived from the application of the rubrics, as well as the rubrics’ criteria difficulty and discrimination indices were also determined.
Results
The rubrics´ content validity was found to be adequate (Content Validity Index > 0.90). These showed a unidimensional structure, and an acceptable internal consistency (α = 0.71) and inter-rater reliability (Fleiss’ ƙ=0.44, ICC = 0.94). The scores of the subject practical exam practically covered the entire range of possible theoretical scores, showing all the criterion medium-low to medium difficulty indices - except for the one related to the physical therapist position-. All the criterion exhibited adequate discrimination indices (rpbis > 0.39), as did the rubric as a whole (Ferguson’s δ = 0.86). Students highlighted the rubrics´ usefulness for learning the maneuvers, as well as their validity and reliability for formative and summative assessment.
Conclusions
The changed rubrics constitute a valid and reliable instrument for evaluating the execution quality of neurophysiotherapy maneuvers from a summative evaluation viewpoint. This study facilitates the development of rubrics aimed at promoting different practical skills in health-science education.
Background. Physiotherapy students’ expectations of their first clinical education year can be over- or under-estimated. Expectations are related to motivational aspects of behaviour as well as satisfaction, so they may have some influence on academic performance.
Objective. The aim of this study is to describe physiotherapy students’ expectations related to their first clinical internship.
Methods. Qualitative, exploratory study. Participants included physiotherapy students prior to initiating their first clinical internship. Data were collected through semi-structured interviews. The data were analysed using thematic analysis, as proposed by Braun and Clarke. The study protocol was approved by the University of Valencia Ethics Committee of Human Research.
Results. This study included 12 students (six being women; average age 24.2±8.4 years), which was sufficient to reach data saturation. We identified 12 subcategories within five categories: i) perceived meaning of practical training, ii) reference figure during clinical internship, iii) Emotions felt in the first practical training sessions, iv) the easiest and most difficult part of practical training sessions, and v) clinical placement selection factors.
Conclusions. The study has provided detailed explanation of the students’ perceived meaning of the practical training being demonstrated that students gave a real significance to the clinical internship. Positive and negative emotions were present combined with a great enthusiasm to achieve their positives expectations. There was a substantial alignment between expectations of physiotherapy students before their first clinical internshipand the reality, which had a positive impact on their practical learning.
This study describes an experience carried out at a university that combines problem-based learning and the use of information and communication technologies (ICT). The activity pursues to deepen the learning of a subject with complex and singular content, taking as a reference the study of real cases. Specifically, the aim is to develop knowledge about the different sectoral accounting standards that are applied in the preparation of companies' financial information. The student checks for himself the usefulness of accounting knowledge to understand the financial situation of real companies. The results show a generalized improvement in student motivation and in results obtained in the subject. It demonstrates how in complex subjects that are less attractive to students active methodologies can be used to connect with them. They learn to take advantage of the resources offered by ICT to access information of great relevance in the business world.
La formación pediátrica actual apuesta por un modelo de aprendizaje autónomo
y prácti co. Lo que deriva en la necesidad de contar con instrumentos de
seguimiento, comunicación y evaluación aplicables a dicho escenario. El objeti vo
de este trabajo es comprobar si la rúbrica digital puede servir a tal efecto así como
la percepción discente que dicha innovación introduce. Se plantea un estudio de
caso sobre 12 posgradistas de Pediatría durante sus prácti cas profesionales en el
área de Urgencias de un Hospital. La herramienta uti lizada fue Co-Rubric, con la
que se elaboraron cuesti onarios de autoevaluación y una evaluación fi nal, previamente
validados por el juicio de 10 expertos. Los resultados obtenidos indican
que los discentes percibieron la innovación como una experiencia muy positi va,
mientras que desde la perspecti va docente se valoró la capacidad del instrumento
no sólo para evaluar sino también para retroalimentar él proceso.
Introduction: Joint reflection of students and teachers about the processes and factors involved in both learning and teaching and evaluation will achieve higher levels of understanding from the exchange of ideas, assessments and perceptions that are generated among the different participants in the training process. Objective: To know the Nursing students' and teachers' perception about the relevance of the clinical evaluation method used in the subject Surgical-Medical Nursing during the third academic year of the major, at a private university in Chile. Methods: Qualitative research with opinion sampling. Semi-structured surveys were applied to a group of 16 participants among teachers and nursing students, which were transcribed verbatim. Content analysis was carried out, detailing aspects of the studied phenomenon and looking for the categories of meaning that would answer the objective of the research. Results: The studied phenomenon emerged in three large units of meaning: global impression about the usefulness of the instrument; deficit dimensions of the evaluation method, and improvement proposals. Conclusions: It is important to consider the professors' and students' opinion about the evaluation method to be used, since they are an input to monitor qualitatively the behavior of the instrument, considering its flexible, open and dynamic nature, which requires a permanent revision, in order to ensure its quality, to be finally a true contribution to the teaching-learning process.
Este trabajo pretende mejorar la evaluación del prácticum elaborando un instrumento basado en TIC para su empleo por estudiantes de Ciencias de la Educación. Hemos contruido una rúbrica electrónica para la autoevaluación de la competencia " capacidad para aplicar conocimientos a la práctica " de estudiantes de Ciencias de la Educación, y ha sido validada por un grupo de expertos. Los resultados muestran la validación de contenido de la rúbrica en su versión electrónica en la plataforma eRubric y una reflexión en torno a la aplicación piloto realizada. Para la evaluación del prácticum es fundamental considerar la voz de los estudiantes y, por lo tanto, la investigación en autoevaluación a través de TIC nos puede aportar información valiosa para avanzar en su conocimiento. Concluimos que la rúbrica propuesta es válida para el objetivo de la investigación y que, aunque en periodo de pilotaje, nuestro estudio podrá aportar información valiosa para el alumnado, los gestores del prácticum y las administraciones involucradas. Entendemos que es necesario continuar investigando sobre el uso de rúbricas electrónicas adecuadas en el Espacio Europeo de Educación Superior.
En el ámbito de la educación universitaria, aunque no sólo en él, la evaluación de competencias surge
como un aspecto clave si queremos ser coherentes con el nuevo paradigma educativo. En este sentido,
una efectiva coordinación entre docentes puede ayudar a poner en común dificultades, a afrontar las
diferentes tareas y a consensuar criterios de evaluación, con el foco puesto en la búsqueda de una
mejora continua de la práctica docente. En este artículo se presenta un proyecto de coordinación de
profesores de diferentes áreas de conocimiento pero con las mismas inquietudes: cómo desarrollar y
evaluar competencias genéricas en el aula. Para ello han realizado un trabajo común en sus materias
organizando las distintas tareas en diferentes fases: se han seleccionado y definido las competencias
genéricas a desarrollar, se han consensuado indicadores y realizado diversas rúbricas para la evaluación
de las mismas y se han diseñado actividades formativas específicas en las cuales se ha implementado
esta evaluación. Con el objetivo de implicar al alumnado se ha realizado autoevaluación y co‐evaluación.
Además, para evaluar el impacto de este trabajo de coordinación se ha recogido la percepción tanto de
los alumnos como de los docentes implicados ante las nuevas herramientas de evaluación diseñadas,
mediante cuestionarios específicos y sesiones grupales de reflexión. Las reuniones periódicas
mantenidas por los profesores participantes han servido para comprobar que la coordinación del
profesorado fomenta los procesos de reflexión sobre nuestra labor permitiendo el intercambio de
experiencias entre docentes y con ello incrementado su motivación.
Palabras clave:Coordinación del profesorado, evaluación de competencias, herramientas de evaluación
Aim. To provide a global vision of the most relevant aspects to take into account of an evaluation system. Development. The article also proposes two different fundamental models as frameworks for the evaluation process. The main characteristics applicable to any evaluation system are described. Finally the paper points the challenges that evaluation in medical education is facing for the near future.
Aim: To provide a global vision of the most relevant aspects to take into account of an evaluation system. Development: The article also proposes two different fundamental models as frameworks for the evaluation process. The main characteristics applicable to any evaluation system are described. Finally the paper points the challenges that evaluation in medical education is facing for the near future.
This study suggests that students use rubrics to support their own learning and academic performance. In focus groups, fourteen undergraduate students discussed the ways in which they used rubrics to plan an approach to an assignment, check their work, and guide or reflect on feedback from others. The students said that using rubrics helped them focus their efforts, produce work o f h igher quality, earn a better grade, and feel l ess anxious about an assignment. Their comments also revealed that most of the students tend not to read a rubric in its entirety, and that some may perceive of a rubric as a tool for satisfying a particular teacher's demands rather than as a representation of the criteria and standards of a discipline.
En este artículo se presentará parte del trabajo desarrollado dentro de un proyecto interdepartamental sobre innovación docente en el desarrollo de competencias genéricas en los estudiantes. Este trabajo se hizo siguiendo la metodología de investigaciónacción, pues ésta nos permitía a los profesores indagar en la mejora de nuestra práctica docente en el desarrollo y la evaluación de las competencias genéricas en los estudiantes de grado.
Se realizó un análisis de términos que refieren a competencias genéricas, y de las reuniones conjuntas y de las reflexiones derivadas de ellas se fueron generando un listado de competencias a estudiar, y un listado de sus denominaciones alternativas conforme a la bibliografía utilizada. A partir de este listado y de la bibliografía consultada se diseñaron fichas que reúnen las principales características a tener en cuenta en el desarrollo y evaluación en el alumnado universitario de cada una de las competencias consensuadas, y que servirán de ayuda al profesor que pretenda desarrollar competencias transversales en sus asignaturas de acuerdo al EEES.
Según los descriptores que aparecen en estas fichas se desarrollaron unas fichas adicionales donde se detallarían posibles actividades formativas a realizar para el desarrollo de competencias, así como rúbricas o plantillas que pueden servir de ayuda al profesorado para la evaluación objetiva de las competencias recogidas en el listado.
Por último se presentan los resultados y opiniones obtenidos hasta ahora con nuestros alumnos. De este modo, tras la información obtenida del proceso, el equipo investigador reflexionará sobre las posibles mejoras y nuevos planes de acción a acometer en otros cursos.
Teaching the steps of evidence-based practice (EBP) has become standard curriculum for health professions at both student and professional levels. Determining the best methods for evaluating EBP learning is hampered by a dearth of valid and practical assessment tools and by the absence of guidelines for classifying the purpose of those that exist. Conceived and developed by delegates of the Fifth International Conference of Evidence-Based Health Care Teachers and Developers, the aim of this statement is to provide guidance for purposeful classification and development of tools to assess EBP learning.
This paper identifies key principles for designing EBP learning assessment tools, recommends a common taxonomy for new and existing tools, and presents the Classification Rubric for EBP Assessment Tools in Education (CREATE) framework for classifying such tools. Recommendations are provided for developers of EBP learning assessments and priorities are suggested for the types of assessments that are needed. Examples place existing EBP assessments into the CREATE framework to demonstrate how a common taxonomy might facilitate purposeful development and use of EBP learning assessment tools.
The widespread adoption of EBP into professional education requires valid and reliable measures of learning. Limited tools exist with established psychometrics. This international consensus statement strives to provide direction for developers of new EBP learning assessment tools and a framework for classifying the purposes of such tools.
Health care educators need valid and reliable tools to assess evidence based practice (EBP) knowledge and skills. Such instruments have yet to be developed for use among physical therapists. The Fresno Test (FT) has been validated only among general practitioners and occupational therapists and does not assess integration of research evidence with patient perspectives and clinical expertise. The purpose of this study was to develop and validate a modified FT to assess EBP knowledge and skills relevant to physical therapist (PT) practice.
The FT was modified to include PT-specific content and two new questions to assess integration of patient perspectives and clinical expertise with research evidence. An expert panel reviewed the test for content validity. A cross-sectional cohort representing three training levels (EBP-novice students, EBP-trained students, EBP-expert faculty) completed the test. Two blinded raters, not involved in test development, independently scored each test. Construct validity was assessed through analysis of variance for linear trends among known groups. Inter and intra-rater reliability, internal consistency, item discrimination index, item total correlation, and difficulty were analyzed.
Among 108 participants (31 EBP-novice students, 50 EBP-trained students, and 27 EBP-expert faculty), there was a statistically significant (p < 0.0001) difference in total score corresponding to training level. Total score reliability and psychometric properties of items modified for discipline-specific content were excellent [inter-rater (ICC (2,1)] = 0.91); intra-rater (ICC (2,1)] = 0.95, 0.96)]. Cronbach's alpha was 0.78. Of the two new items, only one had strong psychometric properties.
The 13-item modified FT presented here is a valid, reliable assessment of physical therapists' EBP knowledge and skills. One new item assesses integration of patient perspective as part of the EBP model. Educators and researchers may use the 13-item modified FT to evaluate PT EBP curricula and physical therapists' EBP knowledge and skills.
Resumen [Abstract] [Resum] En el entorno universitario actual, caracterizado por la implantación de un sistema educativo basado en los principios de Bolonia y centrado en el aprendizaje por competencias, el presente trabajo aborda uno de los aspectos más complejos del proceso formativo: el de la evaluación. Para ello, se realiza un análisis del concepto y los elementos que integran la competencia, así como de las diferentes etapas que conlleva su proceso de evaluación, cuyo fin último ya no es determinar el nivel de conocimientos que posee el estudiante sobre una materia concreta, sino valorar, esencialmente, en qué grado posee una determinada competencia. En este contexto, se presenta un instrumento diseñado para facilitar la valoración del logro de la competencia. Concretamente, se ha diseñado una matriz de valoración en la que se establecen, de un modo escalonado y jerárquico, diferentes grados en el dominio de la competencia por el estudiante y sus correspondientes equivalencias en el sistema de calificación utilizado en nuestro sistema universitario. Se aplica al caso concreto del título de grado en Información y Documentación, una de las titulaciones adaptadas al espacio europeo de educación superior que se empezaron a impartir en el curso académico 2008-2009.
El presente artículo centra la atención en el trabajo por competencias en el prácticum. La consideración de la insuficiencia de la formación inicial predispone a asumir el prácticum como un espacio privilegiado de socialización-iniciación profesional. Para ello se asume la competencia profesional como referente formativo de esta etapa. Una vez conceptualizada la competencia profesional y la formación basada en competencias, se abordan algunas de las implicaciones de tal asunción en el diseño, desarrollo-gestión y evaluación del prácticum.
Direct observation of medical trainees with actual patients is important for performance-based clinical skills assessment. Multiple tools for direct observation are available, but their characteristics and outcomes have not been compared systematically.
To identify observation tools used to assess medical trainees' clinical skills with actual patients and to summarize the evidence of their validity and outcomes.
Electronic literature search of PubMed, ERIC, CINAHL, and Web of Science for English-language articles published between 1965 and March 2009 and review of references from article bibliographies.
Included studies described a tool designed for direct observation of medical trainees' clinical skills with actual patients by educational supervisors. Tools used only in simulated settings or assessing surgical/procedural skills were excluded. Of 10 672 citations, 199 articles were reviewed and 85 met inclusion criteria.
Two authors independently abstracted studies using a modified Best Evidence Medical Education coding form to inform judgment of key psychometric characteristics. Differences were reconciled by consensus.
A total of 55 tools were identified. Twenty-one tools were studied with students and 32 with residents or fellows. Two were used across the educational continuum. Most (n = 32) were developed for formative assessment. Rater training was described for 26 tools. Only 11 tools had validity evidence based on internal structure and relationship to other variables. Trainee or observer attitudes about the tool were the most commonly measured outcomes. Self-assessed changes in trainee knowledge, skills, or attitudes (n = 9) or objectively measured change in knowledge or skills (n = 5) were infrequently reported. The strongest validity evidence has been established for the Mini Clinical Evaluation Exercise (Mini-CEX).
Although many tools are available for the direct observation of clinical skills, validity evidence and description of educational outcomes are scarce.
The AGREE instrument has been validated for evaluating Clinical Practice Guidelines (CPG) pertaining to medical care. This study evaluated the reliability and validity of physical therapists using the AGREE to assess quality of CPGs relevant to physical therapy practice.
A total of 69 physical therapists participated and were classified as generalists, specialist or researchers. Pairs of appraisers within each category evaluated independently, a set of 6 CPG selected at random from a pool of 55 CPGs.
Reliability between pairs of appraisers indicated low to high reliability depending on the domain and number of appraisers (0.17-0.81 for single appraiser; 0.30-0.96 when score averaged across a pair of appraisers). The highest reliability was achieved for Rigour of Development, which exceeded ICC> 0.79, if scores from pairs of appraisers were pooled. Adding more than 3 appraisers did not consistently improve reliability. Appraiser type did not determine reliability scores. End-users, including study participants and a separate sample of 102 physical therapy students, found the AGREE useful to guide critical appraisal. The construct validity of the AGREE was supported in that expected differences on Rigour of Development domains were observed between expert panels versus those with no/uncertain expertise (differences of 10-21% p = 0.09-0.001). Factor analysis with varimax rotation, produced a 4-factor solution that was similar, although not in exact agreement with the AGREE Domains. Validity was also supported by the correlation observed (Kendall-tao = 0.69) between Overall Assessment and the Rigour of Development domain.
These findings suggest that the AGREE instrument is reliable and valid when used by physiotherapists to assess the quality of CPG pertaining to physical therapy health services.
Educational institutions providing professional programs such as physiotherapy must provide high-quality student assessment procedures. To ensure that assessment is consistent, assessment tools should have an acceptable level of reliability. There is a paucity of research evaluating the reliability of clinical assessment tools used for physiotherapy students. This study evaluated the inter- and intrarater reliability of an assessment tool used for physiotherapy students during a clinical placement. Five clinical educators and one academic participated in the study. Each rater independently marked 22 student written assessments that had been completed by students after viewing a videotaped patient physiotherapy assessment. The raters repeated the marking process 7 weeks later, with the assessments provided in a randomised order. The interrater reliability (Intraclass Correlation Coefficient) for the total scores was 0.32, representing a poor level of reliability. A high level of intrarater reliability (percentage agreement) was found for the clinical educators, with a difference in section scores of one mark or less on 93.4% of occasions. Further research should be undertaken to reevaluate the reliability of this clinical assessment tool following training. The reliability of clinical assessment tools used in other areas of physiotherapy education should be formally measured rather than assumed.
Partiendo de la especial relevancia que los nuevos Planes de Estudio conceden al Practicum, se analiza el estado de la cuestión tomando en consideración 7 aspectos centrales (focus of concern) en su desarrollo: a) la forma en cómo se denomina el periodo de prácticas y cómo eso influye en su sentido curricular; b) los modelos de base del Practicum o cómo se fundamenta el proceso de aprendizaje que se pretende lograr; c) la prevalencia de cuestiones organizativas sobre curriculares y sus consecuencias; d) la importancia de la modalidad de relación interinstitucional que se establezca entre la universidad y los centros de prácticas; e) el componente emocional de la experiencia del Practicum y la necesidad de llegar más allá; f) la creación de redes y comunidades de aprendizaje; g) la incorporación progresiva de las TIC a la gestión, supervisión y tutorización del Practicum. Se concluye señalando como futuro desafío que afrontar, la integración curricular del Practicum.
http://www.verticebooks.com/detalle/guia-para-el-diseno-de-recursos-docentes-que-fomenten-el-desarrollo-y-evaluacion-de-las-competencias-transversales-en-educacion-9037.140
Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool in the hands of the students. This study focuses on the validity and reliability of the rubric as an assessment tool for student peer-group evaluation in an effort to further explore the use and effectiveness of the rubric. A total of 1577 peer-group ratings using a rubric for an oral presentation was used in this 3-year study involving 107 college biology students. A quantitative analysis of the rubric used in this study shows that it is used consistently by both students and the instructor across the study years. Moreover, the rubric appears to be 'gender neutral' and the students' academic strength has no significant bearing on the way that they employ the rubric. A significant, one-to-one relationship (slope = 1.0) between the instructor's assessment and the students' rating is seen across all years using the rubric. A generalizability study yields estimates of inter-rater reliability of moderate values across all years and allows for the estimation of variance components. Taken together, these data indicate that the general form and evaluative criteria of the rubric are clear and that the rubric is a useful assessment tool for peer-group (and self-) assessment by students. To our knowledge, these data provide the first statistical documentation of the validity and reliability of the rubric for student peer-group assessment.
Objective
The primary goal of this study is to evaluate physiotherapy students’ assessment about the importance of their competences training level during their Academic Years. These competences have been taken from the “Libro blanco del Título de Grado en Fisioterapia”.
Objective
To learn the opinion of Spanish Physical Therapists on the present physical therapy curriculum and future degree program.
Material and method
Qualitative study by telephone survey.
Results
50 physical therapists from different geographic and university origins were polled. A total of 53.1% consider that they did not receive adequate academic training, 60% would include more subjects related to the manual therapy and 46% would increase the credits for subjects such as anatomy, biomechanics and physiology. A total of 94% of the physical therapists consider that the future degree program should include 4 and 5 years and should include the specialization.
Conclusions
There is a low degree of satisfaction regarding present curriculum with training deficiencies in techniques of manual therapy and insufficient content regarding the subject load for subjects on structure and function of the human body. The feature receiving the highest value by the physical therapists surveyed is practical training. The future degree studies should be carried out in a four year program and include the speciality. Thus, a future review of the curricular development is necessary.
Rubrics are rating scales-as opposed to checklists-that are used with performance assessments. They are formally defined as scoring guides, consisting of specific pre-established performance criteria, used in evaluating student work on performance assessments. Rubrics are typically the specific form of scoring instrument used when evaluating student performances or products resulting from a performance task. There are two types of rubrics: holistic and analytic (see Figure 1). A holistic rubric requires the teacher to score the overall process or product as a whole, without judging the component parts separately (Nitko, 2001). In contrast, with an analytic rubric, the teacher scores separate, individual parts of the product or performance first, then sums the individual scores to obtain a total score (Moskal, 2000; Nitko, 2001).
Several benefits of using scoring rubrics in performance assessments have been proposed, such as increased consistency of scoring, the possibility to facilitate valid judgment of complex competencies, and promotion of learning. This paper investigates whether evidence for these claims can be found in the research literature. Several databases were searched for empirical research on rubrics, resulting in a total of 75 studies relevant for this review. Conclusions are that: (1) the reliable scoring of performance assessments can be enhanced by the use of rubrics, especially if they are analytic, topic-specific, and complemented with exemplars and/or rater training; (2) rubrics do not facilitate valid judgment of performance assessments per se. However, valid assessment could be facilitated by using a more comprehensive framework of validity when validating the rubric; (3) rubrics seem to have the potential of promoting learning and/or improve instruction. The main reason for this potential lies in the fact that rubrics make expectations and criteria explicit, which also facilitates feedback and self-assessment.
Aprendizaje basado en competencias: una propuesta para la evaluación de las competencias genéricas
- M Poblete
- A Villa
Poblete M, Villa A. Aprendizaje basado en competencias: una
propuesta para la evaluación de las competencias genéricas.
Bilbao: Ediciones Mensajero, S. A. U.; 2007.
Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation
- B M Moskal
- J A Leydens
Moskal BM, Leydens JA. Scoring rubric development: Validity
and reliability. Practical Assessment, Research & Evaluation.
[Internet]. 2002;7 [consultado 12 Mar 2014]. Disponible en:
http://pareonline.net/getvn.asp?v=7&n=10
Barcelona: Agencia per a la Qualitat del Sistema Universitari de Catalunya
- J Carreras
- L A Branda
- A Castro
- M R Fenoll
- A Gual
- Jnm Géhenne
Carreras J, Branda LA, Castro A, Fenoll MR, Gual A, Géhenne
JNM. Guía para la evaluación de competencias en Medicina.
[Internet]. Barcelona: Agencia per a la Qualitat del Sistema
Universitari de Catalunya; 2009 [consultado 12 Mar 2014]. Disponible en: http://www.aqu.cat/doc/doc 71595240 1.pdf
Evaluación de competencias en Ciencias de la Salud
- I Durante
- J R Lozano
- A Martínez
- S Morales
- M Sánchez
Durante I, Lozano JR, Martínez A, Morales S, Sánchez M. Evaluación de competencias en Ciencias de la Salud. México:
Panamericana; 2012.
Evaluación de programas de prácticas clínicas
- T Gallego
Gallego T. Evaluación de programas de prácticas clínicas. Barcelona: Médica Jims, S. L.; 2012.
El diario reflexivo académico como recurso de aprendizaje en las prácticas clínicas: una experiencia en grado de fisioterapia de la Universidad Europea de Madrid
- N L Martiáñez
- E Delgado
- A Martínez
Martiáñez NL, Delgado E, Martínez A. El diario reflexivo
académico como recurso de aprendizaje en las prácticas
clínicas: una experiencia en grado de fisioterapia de la
Universidad Europea de Madrid. Actas de las IX Jornadas
Internacionales de Innovación Universitaria: Retos y oportunidades del desarrollo de los nuevos títulos en educación
superior. Villaviciosa de Odón: Universidad Europea de Madrid;
2012.