An Internet-based evaluation system for a surgical residency program.
ABSTRACT Efficient and meaningful evaluation of performance is critical to the professional development of trainees in surgical residency programs. Current paper-based evaluation instruments have numerous limitations. We developed an Internet-based evaluation system to more rapidly and efficiently assess the experience of residents, faculty, and rotations.
An on-line evaluation system was designed and implemented in October 1999. Custom evaluations were created for residents, faculty, and rotations. Evaluations were completed via the Internet site from remote locations with standard computers and standard Web browser software. Completed evaluations were automatically available on-line for review and data analysis. Data were analyzed by chi-square analysis with a probability value of less than 0.05 considered statistically significant.
Compliance in completion of evaluations improved from 50% to 80% in the initial 6 months of implementation (p < 0.01) with the Web-based system. There was no significant difference between faculty and resident compliance. In evaluation of "ease of use," a total of 612 responses were received over this period with a total average score of 3.5 (5 point scale, 5 = strongly agree). Residents' opinions (average score, 3.69) were slightly more positive than those of faculty (average score, 3.31). Confidentiality was improved over paper-based systems by a detailed security network.
This Internet-based evaluation system is a potentially powerful instrument for evaluating our surgical residency program and making changes to improve the educational experience in a timely and efficient manner.
- SourceAvailable from: Adina Kalet[show abstract] [hide abstract]
ABSTRACT: The Accreditation Council of Graduate Medical Education (ACGME) and the Residency Review Committee require a competency-based, accessible evaluation system. The paper system at our institution did not meet these demands and suffered from low compliance. A diverse committee of internal medicine faculty, program directors, and house staff designed a new clinical evaluation strategy based on ACGME competencies and utilizing a modular web-based system called ResEval. ResEval more effectively met requirements and provided useful data for program and curriculum development. The system is paperless, allows for evaluations at any time, and produces customized evaluation reports, dramatically improving our ability to analyze evaluation data. The use of this novel system and the inclusion of a robust technology infrastructure, repeated training and e-mail reminders, and program leadership commitment resulted in an increase in clinical skills evaluations performed and a rapid change in the workflow and culture of evaluation at our residency program.Journal of General Internal Medicine 06/2004; 19(5 Pt 2):492-5. · 3.28 Impact Factor
- [show abstract] [hide abstract]
ABSTRACT: A web-based evaluation system for residents to provide feedback on faculty was piloted in four training programs at the Aga Khan University prior to institution-wide implementation. Of the four programs, less than 50% of forms were submitted by residents of three programs while more than 70% of forms were submitted by the residents of one program. This study was conducted to identify reasons for the varying participation rates of the four programs with a view to improving the system. A qualitative approach was employed using focus group discussions (FGDs). Volunteers were invited and three groups of eight to ten residents each were formed. Participants for FGDs were selected from all residency years. FGDs were used to identify residents' perceptions regarding the web-based faculty evaluation system and to identify residents' problems and concerns with completing the web-based faculty evaluating forms. Technical issues in completing and submitting the forms online were identified to be the main deterrents to completing the evaluation forms. Non-accessibility of a resource person for resolving technical problems with the software and the burden of taking time out to complete the forms were considered as limiting factors by many residents. Residents recommended a focused orientation session to the new system within the departments. Residents' confidence and support are key to promoting adequate participation in web-based evaluations. Focused orientation sessions, reinforcement, reminders, assurances of confidentiality, and removal of technical glitches should help to improve resident participation.Education for Health Change in Learning & Practice 08/2010; 23(2):334.
- [show abstract] [hide abstract]
ABSTRACT: Web-based course evaluation systems offer the potential advantage of timely evaluations. The authors examined whether elapsed time between teaching and student evaluation of teaching impacts preclinical courses' quality ratings. The overall relationship of elapsed time with evaluation rating was explored with regression and ANOVA. Time between teaching event and evaluation was categorized by weeks. Within-teaching-events means and variances in evaluations related to elapsed weeks were compared using repeated-measures ANOVA. With more elapsed weeks, quality mean ratings increased (P < .001) and variability decreased (P < .001); effect sizes were small (average effect size = 0.06). Trends were similar in regression analysis and for data aggregated by event. Summaries of event quality are negligibly impacted by evaluation timing. Future studies should examine the impact of other Web-based evaluation features on evaluation.Academic medicine: journal of the Association of American Medical Colleges 10/2008; 83(10 Suppl):S29-32. · 2.34 Impact Factor