No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Instruments for obtaining student feedback: A review of the literature
Assessment & Evaluation In Higher Education
Abstract
This paper reviews the research evidence concerning the use of formal instruments to measure students’ evaluations of their teachers, students’ satisfaction with their programmes and students’ perceptions of the quality of their programmes. These questionnaires can provide important evidence for assessing the quality of teaching, for supporting attempts to improve the quality of teaching and for informing prospective students about the quality of course units and programmes. The paper concludes by discussing several issues affecting the practical utility of the instruments that can be used to obtain student feedback. Many students and teachers believe that student feedback is useful and informative, but for a number of reasons many teachers and institutions do not take student feedback sufficiently seriously.
... Regarding the number of citations per article (data updated on September 2024), the study with the most citations (7025) was "The Power of Feedback" (Hattie and Timperley 2007). Above 400 citations are the studies "Instruments for Obtaining Student Feedback: A Review of the Literature" (Richardson 2005) − 499 citations and "Students as Co-creators of Teaching Approaches, Course design, and Curricula: Implications for Academic developers" (Bovill, Cook-Sather, and Felten 2011) − 428 citations. ...
... In the literature analysed, student surveys play an important role in QA (Cardoso, Santiago, and Sarrico 2012;Borch, Sandvoll, and Risør 2020;Pastore, Manuti, and Scardigno 2022), Collecting feedback is a key strategy for monitoring, evaluating and continuously improving the quality of teaching (Richardson 2005;Jara and Mellar 2010;O'driscoll 2012;Kolodii et al. 2021). Hattie and Timperley (2007) explore feedback in the classroom; while Koutselini (2020), Moroz and Moroz (2022) and Zaitseva and Law (2023) address it in distance education; Harvey (2003) emphasises the importance of timely reporting; and Watson (2003) highlights that greater awareness of implemented actions increases student participation in feedback. ...
... Lowe and Shaw (2019) highlight the use of email as one of the best feedback practices; while Zaitseva, Milsom, and Stewart (2013) emphasise the value of open text comments. despite the importance of student participation in QA, many authors claim that institutions and lecturers do not take feedback 'seriously enough' (Richardson 2005). In addition, there are students who respond apathetically or thoughtlessly to surveys (Elassy 2013), and this participation is considered 'passive' (Maxwell-Stuart and Huisman 2018). ...
This article presents the results of a systematic literature review on student
contributions to quality assurance processes in higher education.
The research had three main outcomes: the first identified relevant studies
on the subject under investigation. The second, more innovative, analysed
the main issues related to student participation, involvement/
engagement and commitment as a stakeholder in QA processes. The
analysis revealed that students participate in QA processes, both internally
and externally, in three distinct dimensions: – Participation: the
importance of student participation associated with the teaching and
learning process is recognised, emphasising aspects such as feedback,
assessment, satisfaction and partnerships. – Involvement: centred on governance
and partnerships, with students acting as partners in governance
and decision-making processes, in an active and involved participation in
QA processes. – Commitment: refers to the deepest level of involvement,
where students demonstrate a commitment to QA. The third outcome
focused on the results of the three research questions posed, which
made it possible to analyse and characterise the selected studies, define
a comprehensive framework of student contributions in the field of QA
and point out areas requiring further investigation that could guide
future research.
... Another common form of evaluating teaching performance is through the customer base which is the students in this particular case. This type of method has been favored for its appeal to the learners since they are the target consumers of learning services (Richardson, 2005). Personal impressions shared by students can reveal to teachers multiple aspects of knowledge transmission process, such as communication methods, explanations, and classroom atmosphere. ...
... This work coaxes that SETs are instrumental in giving information regarding instructors' execution to inform and enhance trainee instruction, as well as, for formative conclusions with respect to functions of faculty improvement and accreditations. Richardson (2005) rightly points out, the student is a key factor in determining the practical utility of the conventional teaching practices. The gathering of students' feedback can identify areas that require improvements in the teaching methods, the clarity of teachers' communication, and whether students feel comfortable learning in the class or not. ...
... The first benefit we get from using the student feedback as a measure in evaluating instructors' performance is the key engagement by learners. Richardson (2005) notes that the students occupy a desirable vantage point into the internal efficiency of the disseminating teaching strategies because often they actually undergo the teaching process. This perspective is important in discerning the strengths of the instructors and the weakness of instructor education programs. ...
The study aims at identifying the role of students’ feedback as an evaluation tool for teaching performance. In order to make informed decisions and enhance instructional practices, educators and educational institutions must prioritize the collection of feedback on teaching effectiveness (Hattie, 2009; Marzano, 2017). The study utilized a mixed-approaches strategy, integrating qualitative and quantitative methods, to investigate the research methodology for getting feedback on teaching from Pakistani students. Constant data from structured survey questionnaires and variegated focus group discussions are used in this research to capture students’ sentiments on teaching effectiveness. Some of the most important aspects are the clarity of the instructions, motivational level, openness and the degrees of informational and organizational learning environment. Inasmuch as the study is learner centred, their perceptions are used to determine the similarities and differences in feedback to allow for an understanding of how students evaluate the effectiveness of a teacher. It included is intended to educate educators and institutions on how to properly implement aspects outlined by the students with regard to instructional methods and educational outcomes. This research can benefit the wider community particularly in the context of educational quality assurance and improvement of teaching practice.
... It is important to note that none of these assumptions have been confirmed by empirical research (25). Furthermore, there is a lack of information on the validity and reliability of questionnaires available, which directly affects outcomes and accountability of the results (7,26). SET has been linked to higher grade point averages, as faculty often reduce workloads and grade more leniently to boost ratings. ...
... SET results are influenced more by individual professors than specific courses. Factors such as teacher popularity, communication skills, evaluator gender, workload, exam difficulty, class timing, and student grades can all impact evaluations (23,26,29,30). Some researchers have explored the correlation between students' overall averages and SET, distinct from the potential influence of grade satisfaction when students know their final grades (31). ...
Purpose/objectives: Student evaluation of teaching (SET) is a crucial method for assessing teaching performance in higher education. This study aimed to evaluate the impact of grouping students by their academic averages on SET results in dental education. Methods: In this descriptive-analytical study, 109 dental students were divided into five groups based on their academic averages: 48 in the top 30%, 15 in the top 10%, 17 in the middle 10%, 16 in the lower 10%, and 48 randomly selected from the bottom 30%. These groups completed questionnaires evaluating five university professors across four aspects: teaching methods, academic proficiency, communication and behavior, and organization and discipline in theoretical lessons. Two weeks later, internal reliability was assessed by redistributing the questionnaires to 10% of participants. Statistical analyses were performed using Chi-Square and ANOVA with SPSS 18, considering results significant at P < 0.05. Results: The internal reliability of the statements was moderate. Significant differences in mean scores among rater groups were observed for teaching methods (P = 0.001), academic proficiency (P = 0.043), and communication and behavior (P = 0.012). However, no significant difference was found for organization and discipline (P = 0.855). Conclusion: Grouping students according to their averages may influence SET in certain teaching aspects. However, this approach does not appear to affect professors' organization and discipline evaluation.
... Elliott and Shin (2002) declared that something is shaped by various outcomes and experiences in the institute, that is, students' satisfaction. Among the topics covered by Richardson (2005) were students' satisfaction with their educational experiences, the quality of the teaching environment, course modules, student evaluations of instruction and perceptions of academic quality. According to Duque and Weeks (2010), a few dimensions such as educational quality, support resources and learning outcomes benefit students' satisfaction with their education. ...
... According to Elliott and Shin (2002), student satisfaction is influenced by various factors, including academic outcomes and the campus environment. Furthermore, Richardson (2005) employs various characteristics of educational quality to assess the learning environment and student satisfaction to assess the quality of education. The indicator includes student opinions of academic excellence and educational experience satisfaction. ...
Purpose
This study aims to determine the impact of the United Nations Educational, Scientific and Cultural Organisation (UNESCO) education quality indicators nexus using teaching and learning materials, physical infrastructure and facilities, teaching methods and assessment on students’ satisfaction (SS) in Pakistani higher education institutions (HEIs).
Design/methodology/approach
An investigation was conducted using empirical research involving 440 postgraduate students from 11 private universities in Lahore, Punjab Province, Pakistan. Data were collected through an online questionnaire. Participants were selected using a stratified sampling technique, and the data were analysed using partial least squares structural equation modelling.
Findings
The findings proved the existence of a significant and positive relationship between teaching and learning materials, physical infrastructure and facilities, teaching methods and assessment of SS in Pakistani HEIs.
Originality/value
The novelty of this article lies in the fact that it addresses the gap in the existing literature by exploring the impact of teaching and learning materials, physical infrastructure and facilities, teaching methods, assessment and student satisfaction in the context of private higher education in Pakistan.
... It is important to note that none of these assumptions have been confirmed by empirical research (25). Furthermore, there is a lack of information on the validity and reliability of questionnaires available, which directly affects outcomes and accountability of the results (7,26). SET has been linked to higher grade point averages, as faculty often reduce workloads and grade more leniently to boost ratings. ...
... SET results are influenced more by individual professors than specific courses. Factors such as teacher popularity, communication skills, evaluator gender, workload, exam difficulty, class timing, and student grades can all impact evaluations (23,26,29,30). Some researchers have explored the correlation between students' overall averages and SET, distinct from the potential influence of grade satisfaction when students know their final grades (31). ...
Purpose/objectives: Student evaluation of teaching (SET) is a crucial method for assessing teaching performance in higher education. This study aimed to evaluate the impact of grouping students by their academic averages on SET results in dental education. Methods: In this descriptive-analytical study, 109 dental students were divided into five groups based on their academic averages: 48 in the top 30%, 15 in the top 10%, 17 in the middle 10%, 16 in the lower 10%, and 48 randomly selected from the bottom 30%. These groups completed questionnaires evaluating five university professors across four aspects: teaching methods, academic proficiency, communication and behavior, and organization and discipline in theoretical lessons. Two weeks later, internal reliability was assessed by redistributing the questionnaires to 10% of participants. Statistical analyses were performed using Chi-Square and ANOVA with SPSS 18, considering results significant at P < 0.05. Results: The internal reliability of the statements was moderate. Significant differences in mean scores among rater groups were observed for teaching methods (P = 0.001), academic proficiency (P = 0.043), and communication and behavior (P = 0.012). However, no significant difference was found for organization and discipline (P = 0.855). Conclusion: Grouping students according to their averages may influence SET in certain teaching aspects. However, this approach does not appear to affect professors' organization and discipline evaluation.
... Qualitative indicators include learner satisfaction, employers' perception of the relevance of skills acquired, and the impact on individuals' personal and professional development. According to Richardson [36], learner satisfaction is closely linked to engagement and academic success, making it a critical measure of training effectiveness. Moreover, employer feedback provides valuable insights into the alignment of acquired skills with market needs, further underscoring the importance of qualitative indicators in evaluating vocational training programs. ...
... Learner satisfaction is another important qualitative indicator; regular surveys help to identify program strengths and weaknesses, as satisfaction is often linked to perceptions of teaching quality, content relevance and available resources [36]. Richardson [37] points out that learner satisfaction is correlated with engagement and academic success. ...
Vocational training plays a central role in economic and social development, helping individuals to adapt to rapid changes in the job market. This scientific article provides an in- depth review of the literature on the characteristics and indicators of quality in vocational training programs. It identifies several key dimensions, including the relevance of pedagogical content, which must be in line with market requirements, and the competence of trainers, essential for transmitting up-to-date knowledge. The integration of educational technologies that facilitate access to learning and optimize pedagogical processes is vital. In addition, program accessibility and equity are emphasized, ensuring that these training courses are open to all, regardless of socio-economic, geographical or other constraints. Quality assessment is based on precise indicators, such as the professional integration rate of graduates, which measures the effectiveness of programs in generating job opportunities. Learner satisfaction is also an important indicator, reflecting the perceived quality of the teaching and learning environment. In addition, the quality of jobs obtained, certification success rates and the level of commitment of participants are crucial criteria for judging a program’s performance. The article also underscores the importance of international standards, such as ISO 9001 (quality management), ISO 21001 (management systems for educational organizations) and the EFQM (European Foundation for Quality Management) excellence model, in the continuous improvement of training systems. These standards offer structured frameworks for quality assurance, although their application sometimes presents limitations, particularly in terms of adaptation to local specificities and the changing needs of different sectors. The challenges faced by these programs include the ability to respond rapidly to changes in the labor market, the management of limited resources, the involvement of stakeholders, and the integration of new technologies. The article proposes solutions such as long-term strategic planning, the involvement of public and private players, and the development of partnerships to share resources and best practices in vocational education and training. In conclusion, the study emphasizes importance of a comprehensive and collaborative approach to ensuring quality vocational training, capable of adapting to constant market changes and making a significant contribution to sustainable economic and social growth. This review demonstrates that continuous improvement in vocational training quality relies on an integrated, collaborative approach that engages all stakeholders to meet the dynamic needs of the labor market.
... Student feedback in academic education is an important factor in investigating the quality of services provided by HEIs. Students' opinions are mainly concerned with evaluations of faculty effectiveness and program quality [9][10][11] as well as student satisfaction [12]. These evaluations are routinely carried out using either internationally recognized and valid questionnaires or questionnaires constructed and adapted to the requirements of the institution concerned. ...
... Although the use of open-ended questions can provide valuable qualitative data on student feedback, they are often considered time-consuming and demanding to process. As a result, they may be unsuitable for many researchers conducting large-scale surveys [11]. ...
This study introduces a hybrid text summarization technique designed to enhance the analysis of qualitative feedback from online educational surveys. The technique was implemented at the Hellenic Open University (HOU) to tackle the challenges of processing large volumes of student feedback. The TextRank and Walktrap algorithms along with GPT-4o mini were used to analyze student comments regarding positive experiences, study challenges, and suggestions for improvement. The results indicate that students are satisfied with tutor–student interactions but concerns were raised about educational content and scheduling issues. To evaluate the proposed summarization approach, the G-Eval and DeepEval summarization metrics were employed, assessing the relevance, coherence, consistency, fluency, alignment, and coverage of the summaries. This research addresses the increasing demand for effective qualitative data analysis in higher education and contributes to ongoing discussions on student feedback in distance learning environments. By effectively summarizing open-ended responses, universities can better understand student experiences and make informed decisions to improve the educational process.
... Students' opinions in academic education is an important factor in investigating the quality of services provided by HEIs. Students' opinions are mainly concerned with evaluations of faculty effectiveness and programme quality [9][10][11], as well as student satisfaction [12]. These evaluations are routinely carried out using either internationally recognized and valid questionnaires or questionnaires constructed and adapted to the requirements of the institution concerned. ...
... Although the use of open-ended questions can provide valuable qualitative data on student feedback, they are often considered time-consuming and demanding to process. As a result, they may be unsuitable for many researchers conducting large-scale surveys [11]. ...
This study investigates the effectiveness of a text summarization method applied to open-ended student evaluations at the Hellenic Open University, aiming to improve the analysis of qualitative feedback from online educational surveys. To address the challenges of processing large volumes of student feedback, an automated summarization technique utilizing advancements in Natural Language Processing (NLP) and text mining is proposed. Using the TextRank and the Walktrap algorithms, student comments on positive aspects, study challenges, and improvement suggestions were analyzed. The analysis revealed that while some students expressed satisfaction with tutor-student interactions and the organization of educational material, there were also negative comments about outdated content in some modules and scheduling issues. The findings highlight the importance of qualitative feedback for education quality, providing actionable insights for improving curricula and teaching effectiveness. This research responds to the growing need for effective qualitative data analysis in higher education and contributes to ongoing discussions about student satisfaction in distance learning environments. By effectively summarizing open-ended responses, university staff can better understand student experiences and make informed decisions to enhance the educational process.
... In higher education, fostering a quality culture and providing full support for quality assurance measures demands accepting students as complete and equal partners. In other words, the inclusion of students in the quality culture of a higher education institution proves to be very useful (Richardson, 2005; Bruce et al., 2007;Tot, 2013). Students can be the source of many creative ideas, but they should be allowed to express them (Jevre-mov et al., 2016, p. 293). ...
... The examined effects of the gender structure, the method of study, the years of implementation of field training, and academic achievement stood out as significant correlates of the evaluation of teaching, which was confirmed by many earlier studies (Franceško et al., 2002;Handal, 2003;Richardson, 2005;Milojević et al., 2011;Biggs Tang, 2011;Anđelković-Pešić et al., 2012;Milojević et al., 2013;Jevremov et al., 2016;De Rijcke et al., 2016). Statistical significance (p<0.001) ...
Purpose: To develop, in addition to institutional evaluation, the need for summative evaluation
as one of the elements of improving the didactic-methodical standard of teaching in higher police
education, a student evaluation model has been designed to systematically and in an organized
manner monitor the implementation and project measures for improving field training as a
form of special forms of teaching at the University of Criminal Investigation and Police Studies.
Methods and Findings: From 2016 to 2019, 416 students participated. Men (229) were statistically
significantly higher (χ2=4,240; df=1; p=0.039) than women (187). The level of acquired
knowledge was rated at 3.73±0.68. The overall impression of field training was 3.68±0.67. The
formation of an assessment of the level of acquired knowledge and acquired skills and the overall
impression of field training statistically significant (p<0,001) has a common influence on the
way of studying and years of training. The distribution of the relative frequencies of the most
common open response groups is: 124 students (29.8%) declared for the longer duration of the
training; objections to poor personal equipment were made by 70 students (16.1%); 58 students
suggested more practical exercise (14%); 30 students proposed the introduction into training of
new technical means of the police (7.2%); 21 students proposed preparatory training in Belgrade
(5%); etc., while only one student replied that he was not interested in field training (0.2%).
Value: Based on the analyzed results, a redesign of the field training for the academic year
2022/2023 was carried out, which included almost all the suggestions of students. The results
of this redesign have yet to be recorded and analyzed, but it is already clear that the model of
a redesign of program contents at the UCIPS based on the evaluation of both students and
training contractors, as well as the future employer (the Ministry of Interior of the Republic of
Serbia), is the right way to model the optimal study program.
... Google Forms also prevents participants from being identified in any circumstances. The response rate to the survey was 57%; scholars consider a response rate of 50% and above as satisfactory, particularly for online surveys [83][84][85]. ...
Sustainable production and consumption in the food supply chain are critical for the United Nations’ sustainable development goals (SDGs). Therefore, it is significant to identify the factors that shape individuals’ food consumption behaviour. Türkiye prioritises sustainable food consumption and the prevention of food loss and waste as a national focus. Accordingly, this study aims to identify the drivers of individuals’ food consumption habits in Türkiye through a survey with the participants being households in Izmir, the third most populous city in Türkiye. More specifically, the study has two main objectives: (i) to identify the factors influencing households’ food consumption preferences and (ii) to demonstrate how the factors concerning sustainable food consumption interact through a survey conducted in Izmir. Based on the state-of-art literature, an online survey was completed by 515 respondents in Izmir. Through an analysis of the survey responses, this study provides a descriptive analysis of socio-demographic variables and a correlation analysis between socio-demographics and sustainable food consumption behaviours, including food shopping behaviour, food purchasing and consumption behaviour, dietary habits, and food waste behaviour. The results demonstrate that socio-demographic factors such as age, gender, education level, income level, and the number of residents in the household are significant for sustainable food consumption behaviour. The survey results also demonstrate that the respondents do not consider carbon footprint generation from food production and transportation in their decisions or behaviours. The study’s main limitation is that the survey is implemented in a single city, Izmir. Future research may extend the scope to other cities in Türkiye, allowing a comparative analysis.
... Accordingly, 37 responded in the process (overall response rate is 64%). As stated by Richardson (2005), a response rate of 50% is regarded as an acceptable response rate in social research surveys. ...
Purpose
Although several models have been developed for handling construction materials, no standard mechanism has been formed to assess the circularity in construction material selection in terms of Circular Economy (CE). Thus, the purpose of this research is to explore the importance of CE for construction material selection and to develop a framework to assess the circularity of construction materials of building construction projects in terms of CE principles.
Design/methodology/approach
This research was conducted under quantitative phenomenon. A Systematic Reviews and Meta-Analyses (PRISMA) based systematic literature review (SLR) was initially conducted to identify the CE principles applied in the construction industry. Then, a questionnaire survey was conducted to evaluate the level of importance of each CE principle on construction material selection to develop a circularity assessment framework. Expert interviews were conducted to validate the proposed framework.
Findings
A total of 41 articles, systematically screened and selected through the SLR process, were used to explore the application of CE principles in the construction industry. A total of 10 CE principles were identified through SLR and the level of importance of each principle was first derived through Mean Weighted Average (MWA). Reuse was identified as the most important CE principle while reduce and recycle obtained second and third ranks. Then the MWA of each CE principle was converted to a percentage to develop the circularity assessment framework.
Research limitations/implications
The research was limited to an objective evaluation of the importance of each CE principle for construction material selection of building construction projects.
Originality/value
The developed circularity assessment framework provides a new impetus for circular material selection in building construction projects for reducing the embodied impact of building materials on overall lifecycle impact of buildings.
... The study collected data from 76 firms, accounting for an 80% response rate far beyond the recommended 50% response rate (Richardson, 2005). From the findings, the respondents' demographic characteristics show that most were male (72.1%). ...
This paper provides evidence on the role of different firm resources in strategy execution usingevidence from Uganda's road transport firms. Data were collected through a surveyquestionnaire from a sample of 76 transport firms. Partial Least Squares - Structural EquationModelling (PLS-SEM) was used to analyze the data. Results indicate that financial, human andphysical resources have a positive and statistically significant contribution to successfulexecution of strategies. Centrally to previous thinking, physical resources lead to morevariations in strategy execution followed by financial and then human resources. Hence,transport firms must be quick, flexible, and innovative in acquiring and utilizing relevantresources to survive in the volatile environment. This study is unique since it explored strategyexecution from a landlocked country Uganda, which relies more on road transport to connect tothe neighbouring borders in the East African Community for most of its imports and exports.Additionally, the study examined the individual contribution of different firm-specific resourceson strategy execution in a developing country with less formalized transport system.Keywords: firm resource, strategy execution, transport firms, Uganda
... The study used census technique to have all the 105 academic staff at the university participate in the study but the response rate turned out at 67 percent. Richardson (2005) states that 50 percent is regarded as an acceptable response rate in social research postal surveys. Thus, 67 percent response rate was deemed acceptable for the study. ...
The common approach method of evaluating instruction in higher education classes is to have students provide feedback on “effectiveness” of learning in a given period. Evaluations generally request specific feedback on measures of teaching effectiveness and on particular aspects of a course, as well as global rating questions. Countless myths and misperceptions regarding course evaluations exist and inevitably influence faculty, university administrators and students’ perceptions. In spite of solid research to counter these assumptions, such beliefs persist and continue to spread. The study was envisaged to focus on assessing lecturers’ perceptions on student-lecturer evaluations in universities. Its specific objectives were: to examine lecturers’ perception on students’ competence in evaluating their teaching effectiveness; to determine the influence of gender, experience, professionalism, departmental discussions and designation on lecturers’ perception of students’ competency in evaluating lecturers’ teaching effectiveness; and to examine lecturers’ perceptions on the formative and summative purpose of students in the evaluations. The study was conducted in Kibabii University and targeted all the teaching staff of the university. The study adopted a descriptive research design. The data was collected using a questionnaire that was validated and reliability of 0.917 attained. Data was analysed using both descriptive and inferential statistics. The results indicate that there was no significant difference between male and female lecturers about their perception on students’ evaluation. The study revealed that feedback on students’ evaluation helps lecturers to improve their teaching and interaction. The respondents disagreed that reports from the student evaluation be used for promotion and increment of their salary. The study concluded that inadequate instructional materials are among the key factors that affect the quality of teaching and learning. The study recommended that the university should provide adequate instructional materials to enhance a conducive environment for learning.Keywords: Assessment, evaluation, quality, perception, course.
... 32,[35][36][37] Candida albicans is one of the main disease-causing fungi in humans, which can cause congenital diseases in immunosuppressed individuals and a significant number of external infections. [38][39][40] Despite significant advancement regarding the understanding of Cryptococcus neoformans from a biological point of view, there is still a substantial gap in the literature regarding the role and function of F-box proteins in this organism. F-box proteins have been well-characterized in model organisms such as Schizosaccharomyces pombe and Saccharomyces cerevisiae, revealing their roles in stress responses, nutrient sensing, and cell cycle progression 41 . ...
Cryptococcus neoformans is a pathogenic yeast that poses significant health risks, particularly to immunocompromised individuals, leading to severe infections and high mortality rates. This review delves into the critical roles of the F-box protein in C. neoformans, emphasizing its involvement in sexual reproduction and virulence. The yeast's ability to form melanin and polysaccharide capsules and its growth at mammalian body temperature are critical factors in its pathogenicity. The F-box protein (Fbp1) is specifically linked to pathogenicity and immune evasion, playing a significant role in the synthesis of sexual spores and the regulation of meiosis and nuclear division. Mutants lacking Fbp1 exhibit defects in these processes, underscoring its essential role. Various experimental approaches, including in vivo pathogenicity assays, bioinformatics tools, transcriptomics and proteomics analysis, protein-protein interactions, and genetic knockouts, are suggested for evaluating the role of F-box proteins. Comparative genomics highlights the presence of 1796 F-box genes in the wheat genome, emphasizing their roles in cellular processes and amino acid functions. Studies on other fungi, such as Schizosaccharomyces pombe and Candida albicans, provide insights into the conserved and unique functions of F-box proteins. Understanding the molecular mechanisms of F-box proteins can provide valuable insights into fungal pathogenesis and potential therapeutic targets. Further research is essential to develop more effective treatments for infections caused by C. neoformans, particularly in light of emerging drug resistance. This review underscores the importance of F-box proteins in the biology and pathogenicity of C. neoformans, highlighting their potential as targets for therapeutic intervention.
... Student evaluations can be conducted using quantitative instruments like questionnaires and surveys (Malgorzata et al., 2016;Richardson, 2005), or qualitative methods such as dialogue-based evaluations (Rebecca & Dobbins, 2013;Steyn et al., 2019). While there is less research on qualitative tools (Borch et al., 2020), they are believed to offer more detailed, context-specific feedback useful for course improvements. ...
Student Evaluation of Teaching (SET) is a widely spread and widely acknowledged means of evaluating teaching effectiveness in higher education institutions (HEIs). Student evaluation of teaching (SET) has been extensively researched, but there are still contradicting attitudes toward its use and validity, caused mainly by student grade bias. A performance metric is needed that can be used to identify student evaluation ratings independent of grading bias. This paper explores various purposes for which SET can be utilized, and the perceptions of the key stakeholders on its relative importance. To remove grade bias from evaluation, a figure of merit (F.O.M) is derived that can be used to identify a student evaluation rating independent of grading bias. The F.O.M is derived using statistical analysis that considers preprocessing of data, and removal of outliers to make it robust. It is found that F.O.M values between 0.7 and 1.3 reflect non-grade bias. Based on this F.O.M, recommendations are built to support the validity of SET in the whole process of teaching effectiveness alongside increased stakeholder engagement.
... Moreover, student perceptions show a much higher predictive validity on many learning outcomes compared to teacher ratings of their own teaching quality (e.g., Seidel & Shavelson, 2007). In addition, research shows that students' ratings vary primarily as a function of the teachers' teaching skills (Benton & Cahsin, 2012;Richardson, 2005). Additionally, student perceptions showed significant correlations with ratings by trained classroom observers (e.g., Kuhfeld, 2017;van der Scheer et al., 2019). ...
This meta-analysis summarizes the available evidence on the effectiveness of student feedback interventions in primary and secondary schools on different aspects of teaching quality. It aims to examine indications of success conditions for an effective use of student feedback in practice. The analysis included 23 studies with 314 effect sizes. We estimated an overall weighted mean effect size of d = 0.27 based on a three-level random-effects model. For effectiveness with respect to various dimensions of teaching quality, only minor and mostly nonsignificant differences were found. Further moderator analyses showed larger effect sizes for studies in which teachers received support for interpreting student feedback and implementing changes in their teaching. Moreover, studies in which teachers were encouraged to discuss the feedback with their students have larger effect sizes than studies in which teachers were not encouraged to do so. Item and answering scale characteristics also showed significant effects on the effectiveness of student feedback. Implications for further research and the use of student feedback in schools are discussed.
... Traditionally, quantitative evaluations such as questionnaires with Likert scale responses have been used to measure student satisfaction. These tools allow structured data collection and statistical analysis of results (Richardson, 2005). However, recent research points out that quantitative methods can be limiting when they are not accompanied by an understanding of the qualitative context (Takona, 2023). ...
... In large-scale surveys, open-ended questions can provide valuable qualitative feedback from students, but analyzing this data requires significant time investment. 17 Manual coding into categories by identifying recurring trends and concepts has traditionally been the mainstream way to analyze qualitative data. 18 To maintain a coherent structure, codes must be shaped accordingly. ...
Understanding the satisfaction of students' academic experience is essential for enhancing the educational process and lowering attrition rates. In this context, universities conduct surveys via questionnaires, including both closed-ended and open-ended questions, to track and assess students' feedback. While closed-ended questions seem useful for rapid quantitative analysis, open-ended ones usually obtain deeper insights and capture a wide spectrum of student experiences. This paper proposes an automated hybrid text summarization approach to handle the problem of assessing vast amounts of qualitative data from responses to open-ended questions. Data preparation, anonymization, normalization, sentence ranking, sentence clustering, extractive, and abstractive summarization, are the main tasks of the proposed approach, which is based on a Microservice-Oriented Architecture. Such an architecture ensures flexibility and scalability, enabling the easy integration of new components that may enhance the approach's overall functionality. Moreover, the applicability of the proposed approach is demonstrated through a dataset containing written responses in the modern Greek language from an online questionnaire completed by about 20.000 students attending study programs at the Hellenic Open University (HOU). The results show that automated text summarization proved to be efficient regarding the time required to produce reports, while also maintaining satisfactory quality.
... The literature on student evaluations of lecturers' teaching suggests that whether a lecturers' teaching should be evaluated is not important; rather, what is important is who should do the evaluation, for what purpose, and by what means (Adeyemo, 2015;Iyam, & Aduwa-Oglebaen, 2005). Richardson (2005) noted that universities/colleges can evaluate lectures' teaching by: classroom observation, student ratings, student achievement, peer rating, self-rating, teacher interview, parents rating, competency tests, and indirect measures. However, of these approaches, student evaluation, though engrossed in controversy, has gained popularity globally as the primary source of assessing teaching in higher education (Atek, Salim, Halim, Jusoh, & Yusuf, 2015;Inko-Tariah, 2013). ...
This descriptive study explored perceptions of lecturers of student evaluations of their teaching at the University of Malawi, Kamuzu College of Nursing. Data were collected from the entire population of lecturers (N=71). Descriptive statistics, t-test, and one-way analysis of variance were computed using SPSS to analyse the data. It was established that generally lecturers had a positive perception towards student evaluation of their teaching, and that their perception did not differ by age, sex, academic qualification, professional rank, or teaching experience. The study further revealed that lecturers' perceptions of student evaluations of their teaching was more positive when the purpose was formative. The study concluded that student evaluations of the lecturers' teaching was not a problem but the question to be answered was the purpose for which such evaluation should serve. It was, therefore, recommended that University of Malawi should harness the implementation of student evaluations of lecturers teaching while triangulating with other evaluation methods.
... Feedback collected at the end of the semester, mid-semester [1], or intermittently [2], can provide instructors with useful insights into students' experiences in a course. While these feedback responses are sometimes used for course or instructor evaluation [3], they can also contextualize and convey students' experiences throughout the semester to supplement traditional performance indicators such as grades. Mid-semester feedback in particular enables instructors to make timely adjustments to the course during the semester, improving the learning environment and end-of-course evaluations [4]. ...
This innovative practice paper presents an LLM-supported technique to help instructors respond effectively to periodic students' reflections. Efficient communication between instructors and students is integral to supporting a productive learning environment. Recognizing the significance of understanding students' perceptions and challenges, we present the initial implementation of a system to help instructors analyze and respond to students' feedback promptly and effectively. This research is inspired by and extends prior works where instructors sent progress check emails to students, with some works finding that such communication increased students' motivation. To collect feedback, we administer regular student reflections throughout the semester that capture how students feel about the course and uncover the challenges they face. This regular feedback-gathering approach allows instructors to better track their students' progress and respond to comments throughout the semester to provide guidance. However, reading and responding to each reflection manually in the context of their overall learning experience can be time consuming. To address this challenge, we introduce an LLM-based automated approach that generates tailored, performance-contextualized responses to student reflections that can be used to guide first-contact interventions. The generated reflection responses (GRRs) address issues discussed in student reflections and provide advice, support, course information, and follow-up questions to the students. Additionally, they provide feedback to students based on their accomplishments and behavioral data within the learning management system (LMS), such as submission patterns. In this work, we discuss our method of generating responses based on students' reflections and their LMS behavior. We also present example scenarios of the proposed approach. Preliminary results indicate that this approach can help instructors facilitate positive educational interactions with students and that the participating students view the interventions favorably, fostering a constructive learning environment. This work provides an initial presentation of our large language model-based response generation method to motivate further investigation into AI-assisted student support mechanisms Index Terms-student experience, reflection
... Completed and submitted usable questionnaires came from 125 companies representing a 55.6% response rate. This response rate is far higher than the 50% rate recommended for survey-based studies (Richardson, 2005;Ahimbisibwe et al., 2020). Table 1 indicates the sample characteristics of the firms that participated in the study. ...
Purpose
This paper aims to contribute to the understanding of the relationship between the different innovation capabilities and the international competitiveness of agro-processing firms in Uganda.
Design/methodology/approach
This study adopted a cross-sectional research design. Data were collected through a questionnaire survey of 125 firms exporting agro-processed products that are members of the Uganda Export Promotions Board (UEPB). Data was analyzed using the Statistical Package for Social Scientists (SPSS).
Findings
The study established that product innovation capabilities and organizational innovation capabilities are positively related to the international competitiveness of agro-processing firms in Uganda while process innovation capabilities and marketing innovation capabilities are not.
Research limitations/implications
A structured questionnaire was used to solicit responses and no follow-up interviews were conducted to inform the reasons behind the respondents’ responses. Nevertheless, the study reveals that agro-processing firms in Uganda should concentrate on building product and organizational innovation capabilities if they are to compete effectively in international markets.
Originality/value
To the authors’ knowledge, this is the first study to investigate the relationship between the four forms of innovation capabilities (product, process, marketing and organizational) and international competitiveness particularly among agro-processing firms from an African country.
... As a whole, the model emphasizes the organic connection of the various elements of instructional design, showing how efficient instructional design can be achieved by integrating the categories of teacher-led activities and length of interventions, the sample size of students and assessment tools, as well as the type of subject matter of the curriculum and the content generated, in order to achieve the goal of enhancing students' academic achievement. Specifically, the activity categories and intervention duration planned by the teacher influences the learning experience and effectiveness of the students, the sample size of students and assessment tools influence the feedback and adjustments made by the teacher to the effectiveness of the instruction, and the type of discipline of the course and the type of content generated by the generative artefacts determines the content and methodology of the instruction (Richardson, 2005). ...
The use of generative artificial intelligence (Gen-AI) to assist college students in their studies has become a trend. However, there is no academic consensus on whether Gen-AI can enhance the academic achievement of college students. Using a meta-analytic approach, this study aims to investigate the effectiveness of Gen-AI in improving the academic achievement of college students and to explore the effects of different moderating variables. A total of 28 articles (65 independent studies, 1909 participants) met the inclusion criteria for this study. The results showed that Gen-AI significantly improved college students’ academic achievement with a medium effect size (Hedges’s g = 0.533, 95% CI [0.408,0.659], p < .05). There were within-group differences in the three moderator variables, activity categories, sample size, and generated content, when the generated content was text ( g = 0.554, p < .05), and sample size of 21–40 ( g = 0.776, p < .05), the use of independent learning styles ( g = 0.600, p < .05) had the most significant improvement in college student’s academic achievement. The intervention duration, the discipline types, and the assessment tools also had a moderate positive impact on college students’ academic achievement, but there were no significant within-group differences in any of the moderating variables. This study provides a theoretical basis and empirical evidence for the scientific application of Gen-AI and the development of educational technology policy.
... In higher education, there is reasonably strong evidence that students are generally good judges of teaching quality (McKeachie, 1990). The two most well-known of student perceptions of teaching quality inventories are the Students' Evaluations of Educational Quality (SEEQ), and Course Experience Questionnaire (CEQ) (Richardson, 2005). Across the two instruments common characteristics of teaching practice are foregrounded (i.e., CEQ: good teaching, clear goals and standards, appropriate workload and assessment, emphasis on independence; SEEQ: learning/value, enthusiasm, organization, group interaction, individual rapport, breadth of coverage, examinations/grading, assignments and workload/difficulty). ...
Students' perceptions of teaching quality and teachers' interaction with their families , along with student self-efficacy and interest in a subject matter to performance. However, little is known about how student perceptions of teaching quality and teacher-family interactions in the subject of English relate to performance in reading comprehension. The current study explores how the teacher-related perceptions of New Zealand senior secondary students relate to their reading attitudes and tested reading comprehension. Partially mediated structural equation modelling revealed that students' perceptions of teaching quality directly and positively influenced student self-efficacy, interest, and performance and indirectly through self-efficacy. However, perceptions of teacher-family interactions negatively influenced student self-efficacy and performance and were not related to interest. Although the model was statistically invariant across demographic variables, there were large latent mean differences across school decile bands for reading performance. These findings reinforce the importance of student perceptions of good teaching and raise challenges about how teachers interact with families.
... Completed and submitted usable questionnaires came from 125 companies representing a 55.6% response rate. This response rate is far higher than the 50% rate recommended for survey-based studies (Richardson, 2005;Ahimbisibwe et al., 2020). Table 1 indicates the sample characteristics of the firms that participated in the study. ...
Purpose
Preferential market access is a pillar of the enabling clause in international trade, and as such has received scholarly attention in the recent past. This study aims to empirically examine intensity and utilisation of Uganda’s preferential market access.
Design/methodology/approach
Secondary data at the six-digit level of the harmonised system was sourced from the International Trade Centre’s and UN COMTRADE’s Trade Map database on trade flows to compute; trade intensity indices, Balsa’s revealed comparative advantage (RCA) indices and the existing potential trade. The time period was 2013–2022.
Findings
It is evident that Uganda is not taking full advantage of its preferential market access. The findings of the trade intensity analysis revealed that the strength of trade relations between Uganda and its preferential markets has been consistently weak in the period 2013–2022, while the intensity of trade relations with its FTA partners has been strong. The RCA results revealed that all Uganda’s agricultural exports to its preferential markets have a comparative advantage, with exports of roses reporting an exceptionally high RCA relative to other agricultural exports. The RCA results also show that the comparative advantage of Uganda’s coffee exports recovered after a significant fall in the period 2014–2022. The analysis of the existing potential for trade reveals a disproportionally low market share held by Uganda across all product lines, notably, only 2.3% of the US$29bn coffee imports in its preferential markets.
Research limitations/implications
One limitation of this study is that it primarily relied on quantitative data from the International Trade Centre and thus lacks an exhaustive understanding of the circumstances at the export firm level. Qualitative data, such as interviews with exporters and policymakers, could provide deeper insights into the specific challenges and opportunities faced by Ugandan businesses in these markets.
Practical implications
This paper highlights the need for additional trade facilitation measures to address preferential market access bottlenecks such as stringent rules of origin and call for an aggressive government intervention in enhancing the export readiness of SMEs in Uganda. In addition, the paper is informative to Uganda’s exporters regarding the existing and potential increase in their exports to international markets.
Originality/value
The strength of Uganda’s trade relations with its preferential markets is empirically examined in this paper and provides useful insights for enhancing utilisation of preferential market access by beneficiary countries.
... High-quality instruction and well-designed course materials facilitate enhanced teaching quality and evaluation, enabling students to comprehend and implement educational ideas and practices effectively. Utilizing evaluation strategies that yield valuable feedback and precisely gauge student advancement significantly enhance students' favorable opinions of the course (Richardson, 2005). ...
ARTICLE INFO ABSTRACT This study investigated the contribution of teacher (theory and practice) TEFER courses to improving the clinical training of student teachers through close integration of theory and practice has been examined across various dimensions: linking theory to practice, promoting research-based practice learning, fostering pedagogical reflective discourse, enhancing teaching quality, learning and assessment, increasing partnership between academia and schools, and fostering both individual and collective engagement. The participants of the study consisted of (36) female students in the bachelor's degree. A multi-item questionnaire was developed for research purposes examining the perceptions and attitudes of teacher students regarding the contribution of TEFER courses to integrating theory and practice in clinical training. A multi-item questionnaire was developed for research purposes examining the perceptions and attitudes of teacher students regarding the contribution of TEFER courses to improving teaching. The students were asked to provide personal feedback to open-ended questions. The findings revealed that teaching students have high perceptions and attitudes towards professional development through participation in the Theory and Practice course. Results indicate a significant bridging of the gap between theory and practice, with a high level of practice-based learning and reflective pedagogical dialogue. Participants believe the course encourages research and problem-solving skills, enhances students' feelings and behaviors, and improves academic achievement and modern educational skills. Furthermore, the study shows increased partnership between the academy and the school, with students understanding and dealing with the educational system more effectively, strengthening the partnership between colleges and schools, and promoting communication among lecturers, principals, teachers, and students. The course fosters personal and collective responsibility in students, contributing to a high overall partnership between the academy and school, as well as integrating students into various school topics and building a bond of friendship and acquaintance.
... The concept of satisfaction plays a crucial role in HE marketing theory and practice [29]. According to previous research, student satisfaction is a complicated matter [30]. Elliot and Healy [31] noted that student satisfaction reflects a subjective, short-term attitude arising from the evaluation of a student's educational experience. ...
This study examines the role of student co-creation behavior in contributing to student satisfaction, perceived university image, and student positive word of mouth (WOM). Using a sample of 513 students from a Taiwanese university and conducting partial least squares structural equation modeling, the findings indicate that co-creating value is critical to student satisfaction, university image, and positive WOM. The results also show the effect of student satisfaction and university image on student positive WOM. This study confirms the pivotal role of student participation in co-creating value in enhancing satisfaction with the university experience, creating and sustaining a positive image, and building the credibility of the university. This research is particularly important to higher education institutions because it has practical implications for decision-makers, brand managers, and HE marketers who wish to improve understanding of the relationship between the university and students in the process of co-creating value and its outcomes.
... Collecting and responding to feedback is essential for enhancing student experience. [3] Armann and Stockham also stated that except for improving rapport between teachers and students, feedback could provide valuable insights about assessment and teaching process. [4] Hence, for universities to be successful should find what students consider high important and incorporate such feedback into their priorities. ...
This research delves into the disparity between the provision of feedback opportunities and the clarity of feedback implementation for business and management students at the University of Sussex and the University of Brighton. Employing a qualitative constructivist approach, the study aims to uncover the barriers to student feedback integration and propose solutions for enhancing university responsiveness. The investigation reveals significant differences in feedback implementation rates and the perceived value of student opinions, with implications for teaching quality and student satisfaction. The study employs questionnaires, interviews, and focus groups to gather insights from students and staff, emphasizing the need for transparent communication and actionable feedback loops. The findings suggest that universities must prioritize student voices to foster a positive learning culture and improve institutional practices.
... Gathering student perspectives is paramount for curriculum review as they are key stakeholders and have direct experience within the educational system. Research suggests that student feedback can contribute to improving curriculum design, as it provides insights into the skills gaps, outdated content, and areas that require greater emphasis (Richardson, 2005). Consequently, understanding the perceptions of MPM students regarding their employability and productivity can guide curriculum enhancements targeted at meeting the ever-changing demands of the public management sector. ...
The study focused on the evaluation of curriculum changes in the Master in Public Management program. These changes aim to align the curriculum with industry trends, emphasize practical application and real-world experiences, and
increase specialization and flexibility. The study concludes that these enhancements
have positively impacted students' employability and market competitiveness in the public management field. Additionally, the curriculum review has improved the productivity and effectiveness of public managers by providing them with updated
knowledge and skills, enhancing their problem-solving abilities, and emphasizing the importance of continuous learning. The curriculum review has also significantly enhanced students' employability through the emphasis on practical skills,
communication and collaboration skills, and specialized knowledge and adaptability. Overall, the findings suggest that the curriculum review has successfully prepared students for success in their future careers.
... Table 1 illustrates that the response rate was 93%. A response rate of more than 50% is deemed appropriate according to [21], therefore the acquired response rate of 93% shows that the returned rate was both acceptable and adequately representative. According to the jobs or professions of the respondents, 37% of the sample consisted mostly of technicians. ...
A major contributor to economic growth in Nigeria, the construction sector is frequently beset by large cost and schedule overruns that jeopardize project profitability and impede the development of infrastructure. This study looks into how project management techniques affect these overruns in building construction projects in Nigeria. The goal of the study is to pinpoint the major project management variables that lead to overspending and delays while also offering recommendations for best practices to address these problems. A mixed-methods strategy was used, integrating qualitative case studies with quantitative data analysis. Project engineers, contractors, Architects, and Technicians were given standardized questionnaires to complete about a variety of Nigerian building projects. In-depth interviews and focus groups were also held to acquire a deeper understanding of the real-world difficulties and experiences encountered in the field. The study pinpoints several crucial project management techniques that affect time and cost performance. The results show that Inadequate construction site management and supervision ranked 1 st with an RII of 0.7877, Improper construction techniques and lack of experience of the consultant came 2 nd and 3 rd with RII of 0.7215 and 0.7213 respectively. Late payments during work progress and Use of improper equipment are contributors to project time overrun; while, factors affecting cost overrun are fluctuation in building material costs ranked 1st with RII of 0.7354, frequent design changes, (0.7138), omissions and errors in bills of quantities (0.6754), import of construction materials (0.6754). The report suggests enhancing project management through effective resource management techniques, ongoing training and certification programs, and implementing advanced technology to improve communication, resource management, and planning precision, contributing to sustainable growth in Nigeria's construction industry. INTRODUCTION The building sector significantly impacts economic productivity, investing in services, industry, and agriculture. The public and private sectors aim to improve project performance through cost reduction, timely completion, and quality improvement [1]. Project control aims to ensure projects meet goals, finish on time, and within budget, but modern methods in the building industry often lead to delays and overruns. Construction projects often face overspending due to ignorance of time and expense variables [2]. These overruns impact project goals, schedule, quality, and productivity, potentially leading to bankruptcy. Understanding these factors is crucial for efficient project management and financial losses [3]. Developing nations face increased vulnerability to ineffective performance, leading to delayed services, higher expenses, and decreased income [4]. Nigeria's construction industry frequently experiences budget and schedule blowouts, exacerbated by misconceptions and arguments. Nigeria's building construction industry is vital for economic growth and employment but often faces cost and time overruns [5]. Understanding these issues is crucial for improving project performance and sustainable development. This study investigates the impact of project management techniques on time and cost overruns in Nigerian building construction projects. Also, to identify best practices and analyze stakeholder management to provide practical advice for industry professionals. The causes of time and cost overruns in Nigeria project construction were identified which are common in developing nations, but research on the causes is lacking to sum up, to resolve cost
... Usually, backward feedback is collected using a formal survey at the end of a semester as a student satisfaction element (Mulliner & Tucker, 2017;O'Donovan, 2017). However, this approach is criticised by researchers because the respondents of these student satisfaction surveys could not benefit from the outcome of them (Richardson, 2005). Therefore, a mechanism to collect frequent backward feedback from students would be more beneficial for both educators and students. ...
Background
Feedback is central to formative assessments but aligns with a one‐way information transmission perspective obstructing students' effective engagement with feedback. Previous research has shown that a responsive, dialogic feedback process that requires educators and students to engage in ongoing conversations can encourage student active engagement in feedback. However, it is challenging with larger student cohorts. Learning Analytics (LA) provides promising ways to facilitate timely feedback at scale by leveraging large datasets generated during students' learning. However, current LA design and implementation tend to treat feedback as a one‐way transmission rather than a two‐way process.
Objectives
This case study aims to improve LA design and practice to align with dialogic feedback principles by exploring an authentic dialogic feedback practice at scale.
Methods
We explored a dialogic feedback practice of a course having 700 undergraduate students. The case study used quantitative and qualitative analysis methods to investigate what students expect from feedback, how educators respond to students' feedback requests, and how students experience feedback.
Results and Conclusions
The results emphasise the need to focus on cognitive, relational and emotional aspects of the feedback process. In aligning LA with dialogic feedback principles, we propose that LA should promote the following objectives: reflection, adaption, personalisation, emotional management, and scaffolding feedback provision.
... The CEQ is used globally to determine undergraduate student satisfaction and to identify areas for improvement [11,12]. There is substantial evidence supporting its reliability and validity with undergraduate and medical students [13][14][15][16][17][18][19], and it has been utilised in ophthalmology interventions evaluating the FC [20][21][22]. However, the efficacy of the FC for ophthalmology education in a completely virtual setting is still insufficiently measured [9]. ...
Purpose
This study aimed to measure student satisfaction with a revised ophthalmology delivery format, which due to the pandemic had previously relied on a remote online flipped classroom (OFC) format compared to a blended learning format. This educational strategy combined online learning with in-person seminars and practical patient centred sessions. Our previous investigations demonstrated a significant lack of student satisfaction with a curriculum solely reliant on a remote OFC, as such we hypothesised that a blended learning approach would result in improved levels of student satisfaction and knowledge gain.
Methods
Non-randomised intervention study of two groups; group 1 = OFC group and group 2 = BL group, compared perspectives of 4th year ophthalmology students using a validated course evaluation questionnaire (CEQ).
Results
A total of 59 students from the BL group (n = 257; response rate = 23.0%) and 28 from the OFC group agreed to participate in the study (n = 114; response rate = 24.6%). Participants in the BL group felt it was easier to determine the standard of work that was expected (77.42% v 60.71%) and demonstrated significantly increased satisfaction with staff motivation of students (95.16% v 64.29%, p <0.001) and provision of feedback (74.19% v 46.43%, p = 0.004), compared to the OFC group. Furthermore, students in the BL group also felt the course significantly improved their analytical skills (64.52% v 42.85%, p = 0.023) and ability to work as part of team (69.36% v 25%, p <0.001) as well as reporting reduced dissatisfaction with the level of choice afforded in terms of how they would learn (33.88% v 60.71%, p = 0.31) and the how they were assessed (59.68% v 89.28%, p = 0.004). No evidence of a statistical difference in exam score was observed.
Conclusions
The COVID-19 pandemic necessitated an unavoidable pivot to online and distance learning, to meet the challenges presented by government mandates and social distancing requirements. Since many of these directives have been reversed, it is important to evaluate the effectiveness and learner perceptions’ of the online and distance learning interventions. In this study we demonstrated a significant student preference for BL compared to the OFC approach, with comparable student performances determined by MCQ examinations. Our findings suggest a preference for reintroducing in-person and patient engagement activities in post-pandemic health professions education.
... Despite the utility of qualitative student feedback, significant challenges remain in putting its usage into practice (Richardson, 2005). Shah and Pabel (2019) note the success in use of quantitative data but point to limited prior progress in the analysis and practical use of qualitative feedback. ...
This paper assesses the potential for the large language models (LLMs) GPT-4 and GPT-3.5 to aid in deriving insight from education feedback surveys. Exploration of LLM use cases in education has focused on teaching and learning, with less exploration of capabilities in education feedback analysis. Survey analysis in education involves goals such as finding gaps in curricula or evaluating teachers, often requiring time-consuming manual processing of textual responses. LLMs have the potential to provide a flexible means of achieving these goals without specialized machine learning models or fine-tuning. We demonstrate a versatile approach to such goals by treating them as sequences of natural language processing (NLP) tasks including classification (multi-label, multi-class, and binary), extraction, thematic analysis, and sentiment analysis, each performed by LLM. We apply these workflows to a real-world dataset of 2500 end-of-course survey comments from biomedical science courses, and evaluate a zero-shot approach (i.e., requiring no examples or labeled training data) across all tasks, reflecting education settings, where labeled data is often scarce. By applying effective prompting practices, we achieve human-level performance on multiple tasks with GPT-4, enabling workflows necessary to achieve typical goals. We also show the potential of inspecting LLMs’ chain-of-thought (CoT) reasoning for providing insight that may foster confidence in practice. Moreover, this study features development of a versatile set of classification categories, suitable for various course types (online, hybrid, or in-person) and amenable to customization. Our results suggest that LLMs can be used to derive a range of insights from survey text.
... This investigation is anticipated to address previous limitations in assessing educational quality in higher education institutions, which often focus solely on the relationship between teaching activities and students' satisfaction with learning outcomes. The assumed direct correlation between teaching quality and student satisfaction with learning is currently under scrutiny due to recent findings suggesting that satisfaction may be influenced by factors beyond teaching quality alone Prosser & Barrie, 2003;Richardson, 2005). ...
This study aims to explore the impact of teaching quality on learning engagement, with the objective of crafting a framework for enhancing educational quality management in higher education institutions. To achieve this goal, the study employed a theoretical model of education assessment originating from Western contexts and applied it to a research sample from Vietnam. By adapting two measurement tools and testing methodologies, the study assesses the relationship between teaching quality and academic engagement. Our study involves 244 graduate students and alumni from a Vietnamese university, with an average age of 41.8 (SD = 6.41), comprising 166 female students (68%). Results from factor analysis, reliability analysis, and correlation analysis underscore the reliability and validity of our scale for measuring course experience as a proxy for teaching quality. Furthermore, hierarchical multiple regression analysis reveals a significant correlation between course experience, reflecting the quality of teaching activities, and students’ learning engagement. Discussions of the research limitations and future research directions are also indicated in this paper.
... Because the course was brand new, it was important to look at what was occurring on a weekly basis in order to be responsive to the needs of the students. Feedback has also been identified as being the most effective when it is immediate [46], and this methodology allowed for frequent course feedback. ...
Photovoice is a type of participatory action research in which individuals document their experiences through photography. Through the taking, captioning, and reflecting on photographs that they have taken, participants are able to affect change within their communities. Participants also take part in an interview or focus group about their photos at the end of the photovoice process in which they determine themes that appear in their photos, allowing them to participate in the research being done. We used the photovoice methodology in a small, project-based, upper-division, physics capstone course at the University of Colorado Boulder, in which students worked on an authentic industry project in partnership with a company in the quantum industry. As an example of the types of research results and benefits one could obtain using photovoice, we present a discussion of how we implemented the photovoice process within this course and present some of our results, including students’ experiences with the photovoice process. Photovoice may be particularly useful in understanding new, unique courses, as it allows students to co-create research that highlights ideas about the course that researchers would not know to ask about in more traditional research methodologies such as reflection questions.
Published by the American Physical Society 2024
... As many authors have done before in different contexts, the main goals of the research are to understand in depth the causes that lead Andalusian university students to drop out of their studies, considering the effect of the factors involved (Behr et al., 2020) such as financial policy, the educational quality of the institutions and others related to the students themselves (educational background or study ability), in order to be able to predict and prevent this situation. The research also aimed to develop concrete proposals for reorienting educational policies and training models to mitigate the predictors of academic dropout (Richardson, 2005) and to strengthen persistence in studies (Nieuwoudt and Pedler, 2023). ...
The creation of a student feedback system has allowed for the evaluation and analysis of the college faculty's performance. In the educational system, student evaluation of instruction is essential. The main objectives are to analyse the student’s responses and gain their opinions. The academic staff can be evaluated and assessed by students in higher education using web-based platforms through feedback. This has been shown to be advantageous because, based on online reviews and comments from students, it frequently enhances the performance of academic staff. Students can submit their views by logging in with their login information to the online portal for student feedback. And through the admin login, administrators may view/analyse the performance of the respective faculty in the form of charts
Despite the affordances of professional development (PD) in teaching and learning in facilitating quality teaching and contributing to students’ learning outcomes, there is merit in exploring educators’ needs by enabling conditions to further develop their existing knowledge and practices. In fact, it is vital that PD in teaching and learning in higher education (HE) actively engages and meets instructor individual needs. This paper explores PD within the HE context and identifies corresponding categories common and in relation to quality teaching. Based on the review, most PD in teaching and learning could be identified within one of four categories that contribute to quality teaching: (1) assessment approaches; (2) instructional approaches; (3) communication and delivery approaches; and (4) affective responses. Equally important, instructor agency should be supported by encouraging instructors to identify and manage their PD needs via validated measures, which this review found lacking within the literature. With the expectation that such a measure would provide insights into the refinements of current PD in teaching and learning provisions, and enable HE institutions to facilitate PD in teaching and learning to better meet and fulfil instructor individual needs, it is recommended that such a measure be developed and deployed to identify instructors’ needs.
ABSTRAK Kajian ini bertujuan mengenal pasti kriteria atau faktor yang mempengaruhi pelajar dalam menilai prestasi pengajaran pensyarah. Data kajian telah diambil menerusi kaedah temuduga kumpulan fokus dan soal selidik. Temuduga kumpulan fokus telah dilaksanakan dengan melibatkan 22 orang pelajar di Fakulti Ekonomi dan Pengurusan, Universiti Kebangsaan Malaysia (UKM). Soal selidik juga telah dilaksanakan di fakulti tersebut dengan sampel sebanyak 800 orang pelajar. Hasil kajian ini telah dianalisis menggunakan statistik deskriptif. Kajian ini mendapati pelajar bersetuju mengenai kepentingan penilaian terhadap pensyarah supaya kualiti pengajaran pensyarah dapat ditingkatkan. Kajian ini juga membuktikan bahawa pelajar mengambilkira kelima-lima kriteria yang ditetapkan dalam sistem penilaian pengajaran pensyarah di UKM, iaitu (i) penguasaan bahan dan keyakinan dalam penyampaian, (ii) kaedah penyampaian yang jelas dan menarik, (iii) dedikasi dan komitmen semasa penyampaian kursus, (iv) menepati masa dan (v) memberi layanan sama rata kepada semua dan mesra pelajar. Selain daripada lima kriteria ini, pelajar juga mengambilkira faktor lain iaitu penyediaan bahan pengajaran yang lengkap oleh pensyarah serta sikap humor pensyarah di dalam kelas. Dapatan kajian ini membuktikan bahawa kriteria penilaian yang digunakan di UKM selari dengan faktor yang diambilkira oleh pelajar dan ianya sah dan boleh dipercayai. Kata kunci: penilaian pengajaran, prestasi pensyarah, pelajar, universiti ABSTRACT This study aims to identify the criteria or factors that influence students in evaluating lecturer's performance. Data of this research was obtained through focus group interview and questionnaire. Focus group interview was carried out involving 22 students at the Faculty of Economics and Management, Universiti Kebangsaan Malaysia (UKM). A survey was conducted at the faculty by giving out questionnaires to a sample of 800 students. The results of this study were then analyzed using descriptive statistics. This study found that students agree on the importance of evaluating lecturers so that the quality of lecturers' teaching can be improved. This study also demonstrates that students take into account the five criteria set out in the evaluation system of lecturers in UKM: (i) material possessiveness and confidence in the delivery, (ii) clear and interesting presentation methods, (iii) dedication and commitment during delivery courses, (iv) punctuality and (v) equal treatment of all students and student-friendliness. In addition to these five criteria, students also take into consideration other factors such as the provision of complete teaching materials by lecturers and lecturers' use of humor in the classroom. The findings show that the evaluation criteria used in UKM corresponded to the factors considered by the students, and are thus considered valid and reliable.
Background
Student feedback is crucial for evaluating the effectiveness of institutions. However, implementing feedback can be challenging due to practical difficulties. While student feedback on courses can improve teaching, there is a debate about its effectiveness if not well-written to provide helpful information to the receiver.
Objective
This study aimed to evaluate the impact of coaching on proper feedback given by dental students in Saudi Arabia.
Methods
A total of 47 first-year dental students from a public dental school in Riyadh, Saudi Arabia, completed 3 surveys throughout the academic year. The surveys assessed their feedback on a Dental Anatomy and Operative Dentistry course, including their feedback on the lectures, practical sessions, examinations, and overall experience. The surveys focused on assessing student feedback on the knowledge, understanding, and practical skills achieved during the course, as aligned with the defined course learning outcomes. The surveys were distributed without coaching, after handout coaching and after workshop coaching on how to provide feedback, designated as survey #1, survey #2, and survey #3, respectively. The same group of students received all 3 surveys consecutively (repeated measures design). The responses were then rated as neutral, positive, negative, or constructive by 2 raters. The feedback was analyzed using McNemar test to compare the effectiveness of the different coaching approaches.
Results
While no significant changes were found between the first 2 surveys, a significant increase in constructive feedback was observed in survey #3 after workshop coaching compared with both other surveys ( P <.001). The results also showed a higher proportion of desired changes in feedback, defined as any change from positive, negative, or neutral to constructive, after survey #3 ( P <.001). Overall, 20.2% reported desired changes at survey #2% and 41.5% at survey #3 compared with survey #1.
Conclusions
This study suggests that workshops on feedback coaching can effectively improve the quality of feedback provided by dental students. Incorporating feedback coaching into dental school curricula could help students communicate their concerns more effectively, ultimately enhancing the learning experience.
L’évaluation de l’enseignement par les étudiant.e.s (ÉEÉ) demeure le principal outil utilisé pour estimer les compétences du corps professoral ; leur embauche, leur agrégation et leur promotion, tant aux États-Unis qu’au Canada, malgré les critiques formulées à l’égard de sa légitimité et sa validité en tant qu’instrument de mesure. Cependant, une littérature abondante et majoritairement en anglais dénonce la présence de biais qui compromettent les résultats des ÉEÉ avec pour corollaire des pratiques discriminatoires envers les femmes, les personnes racisées et les personnes issues d’autres groupes marginalisés. Cet article présente un résumé de cette littérature et fait quelques suggestions pour de futures recherches.
This chapter looks at ensuring quality within block design. It seeks to answer the question “how do you know you’re doing what you said you would?” by establishing a framework(s) for action research with engagement from educators(s) and learners.
BACKGROUND
Racism arrived in Australia with colonisation and its intentionally oppressive policies and actions towards Aboriginal and Torres Strait Islander Peoples. To a large extent, the colonial and biomedical agenda are maintained by Australia’s health system that underlies much of the racialised health inequities in the country. Dentistry significantly lags behind medicine and other health care areas in the uptake of antiracism, with the dental accreditation body only acknowledging racism as a determinant of oral health in 2022.
OBJECTIVE
This project will comprehensively develop the evidence required for an anti-racist dental health system in Australia through co-design of the following objectives: (1) Development of an anti-racist curriculum for dental students; (2) Workforce strategies that support the attraction, retention, and wellbeing of the Aboriginal and Torres Strait Islander dental workforce; and, (3) Oral health promotion training for Aboriginal Health Workers/Practitioners (AHW/P).
METHODS
This project is grounded in decolonising methodologies and Indigenous methodologies, which inform our ways of working at the knowledge interface. Co-design Yarning sessions will inform the development and implementation associated with each of the objectives through tabulation and narrative synthesis of sessions. Objectives will be evaluated with both quantitative and qualitative measures and analysed accordingly with Inverse Probability of Treatment Weighting, content analysis, or reflexive thematic analysis.
RESULTS
The study received ethical review approval in February 2024 and received funding in June 2024. The co-design phase for each objective will run from July 2024 to February 2025. The dental curriculum will be developed in 2025 and delivered to the 2026 student cohort. Evaluation data will be collected from the comparator student cohort in 2025 and the implementation cohort in 2026. Data collection for the development of workforce strategies will be collected from October 2024 to July 2025, the framework will be developed from August to December 2025 and disseminated in 2026. Oral health promotion training will be developed from August to February 2024, implemented from March to June 2025, and qualitative evaluation data will be collected between July to September 2025.
CONCLUSIONS
The proposed research will enhance anti-racism training of non-Indigenous health practitioners, support Aboriginal and/or Torres Strait Islander dental workforce, and improve oral health training for AHW/P. Together, these strategies will build oral health knowledge at the Community level, in turn supporting Aboriginal and Torres Strait Islander self-determination of oral health.
ارزشیابی از درس بهعنوان یکی از ابزارهای بسیار ارزشمند ارتقاء کیفیت تدریس و یادگیری محسوب می¬شود، زیرا می¬تواند بازخورد مفیدی برای استادان و دانشگاه در مورد کیفیت عناصر برنامه درسی فراهم ¬سازد. ازاین رو هدف پژوهش حاضر بررسی دلایل، ضرورت و چگونگی ارزشیابی از درس و موانع آن در دانشگاه¬های ایران است. بدین منظور با 20 نفر از متخصصان و اعضای هیئتعلمی دانشگاه¬های کشور که در حوزه ارزشیابی و برنامه درسی اهل نظر هستند، مصاحبه نیمه ساختاریافته به عمل آمد. مصاحبه¬ها، پس از پیاده شدن، بهصورت مقوله¬بندی باز تجزیهوتحلیل شدند. یافته¬ها حاکی از آن است که ارزشیابی از درس به چهار دلیل انجام و به سه دلیل جانیفتادن فرهنگ صحیح نقد و مطالبه، نگرانی و دلهره برخی از دانشجویان از بازخورد استادان و نگرانی مدیران دانشگاه، دانشکده و گروه و استادان از بازخورد دانشجویان در دانشگاه¬ها انجام نمی¬شود. یافته¬ها همچنین نشان داد ارزشیابی از درس می¬تواند بهوسیله دانشجویان فعلی، دانش¬آموختگان، همکاران و ذی¬نفعان خارج از دانشگاه و از طریق مشاهده، مصاحبه، رایانه¬ای، پرسشنامه و روش¬های خلاقانه انجام شود.
Teaching is a complex process. The shaping of the process varies in individuals. One example of such is explained here which would demonstrate that beliefs and practice revolve around each other within the local and individual contexts. This reflextion narrates an early childhood teacher educator’s journey of understanding and applying practices of adult teaching in Aotearoa New Zealand. I analyse and describe the factors that contribute to the philosophy and practices that characterise my understanding of teaching adults.
51 Papua New Guinea undergraduate mathematics students evaluated the performance of their mathematics lecturers by completing a survey previously used with US and Australian students (H. W. Marsh, 1981). Facets in the survey included enthusiasm, organization, group interaction, individual rapport, examinations, assignments, workload, breadth of coverage, and learning value. Results show that Ss in the present study differed from Western students in their perception of what is required in a lecturer's performance. These Ss from a developing country did not distinguish between the quality of organization of the lecture and the rapport created between lecturer and Ss, as did students from developed countries. It is suggested that lecturers in developing countries should be person-oriented to be effective. (6 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
There has been little research on the experiences of students with a hearing loss in mainstream higher education. This investigation compared perceptions of academic quality in 265 students with a hearing loss who were taking courses by distance learning and 178 students taking the same courses who had no declared form of disability. Students who were classified as hard of hearing (rather than deaf) produced significantly lower ratings of the appropriateness of their academic workload than did the students with no declared disability, but the ratings produced by students who were classified as deaf were not significantly different from those produced by the comparison group. In other respects, the students with a hearing loss were remarkably similar to the students with no declared disability in their perceptions of academic quality and their overall satisfaction with their courses.
Many colleges and universities have adopted the use of student ratings of instruction as one (often the most influential) measure of instructional effectiveness. In this article, the authors present evidence that although effective instruction may be multidimensional, student ratings of instruction measure general instructional skill, which is a composite of 3 subskills: delivering instruction, facilitating interactions, and evaluating student learning. The authors subsequently report the results of a meta-analysis of the multisection validity studies that indicate that student ratings are moderately valid; however, administrative, instructor, and course characteristics influence student ratings of instruction. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
This article reviews research indicating that, under appropriate conditions, students’ evaluations of teaching (SETs) are (a) multidimensional; (b) reliable and stable; (c) primarily a function of the instructor who teaches a course rather than the course that is taught; (d) relatively valid against a variety of indicators of effective teaching; (e) relatively unaffected by a variety of variables hypothesized as potential biases (e.g., grading leniency, class size, workload, prior subject interest); and (f) useful in improving teaching effectiveness when SETS are coupled with appropriate consultation. The authors recommend rejecting a narrow criterion-related approach to validity and adopting a broad construct-validation approach, recognizing that effective teaching and SETs that reflect teaching effectiveness are multidimensional; no single criterion of effective teaching is sufficient; and tentative interpretations of relations with validity criteria and potential biases should be evaluated critically in different contexts, in relation to multiple criteria of effective teaching, theory, and existing knowledge.
In the past decade student evaluation of teaching has expanded within Australian universities, with the results of evaluations increasingly being used to make judgements about teaching quality, career advancement and the funding of teaching. An important component of any student evaluation of teaching system is, therefore, the communication of rating results in a manner that enables fair and meaningful interpretations and comparisons of results by the wide range of evaluation users. This paper describes the development in one Australian university of Rating Interpretation Guides (RIGs) which take into account the influence of different teaching contexts on ratings and encourage evaluation users to explore ratings in terms of a range of scores rather than focusing on a single mean score. RIGs represent an important innovation in the communication of evaluation results for both formative and summative evaluation purposes. They are designed to enable meaningful interpretations of ratings which in turn will encourage appropriate use of results.
The Course Experience Questionnaire (CEQ) is a development of work originally carried out at Lancaster University in the 1980s. It is used as a measure of perceived teaching quality in degree programmes in national annual surveys of all graduates in the Australian higher education system and is increasingly being employed as a measure of the quality of teaching in universities in the UK. This article discusses the development and use of the CEQ and the construction of a new generic skills scale. Exploratory and confirmatory factor analyses of large multidisciplinary samples of students and graduates from several universities established the reliability and validity of both the full and short forms of the instrument, and identified a two-factor higher order structure. The criterion validity of the instrument was also established, with scores on the CEQ demonstrating positive correlations with students' approaches to learning, perceived course satisfaction, academic achievement and reported generic (‘enterprise˚s) skills development. The instrument also demonstrated discriminant validity via its capacity to differentiate between pedagogically distinct programmes. These results confirm the validity and usefulness of the CEQ as a performance indicator of university teaching quality.
The research on student ratings of instruction, while voluminous, has had minimal focus on the perceptions of the students who do the ratings. The current study explored student perspectives on course and teacher ratings as well as some issues related to teaching effectiveness and faculty roles. It was found that students are generally willing to do evaluations and to provide feedback, and have no particular fear of repercussions. However, they have little confidence that faculty or administrators pay attention to the results, and do not even consult the ratings themselves. The students view teaching and advising as the most important roles that should be played by faculty, yet project that faculty, while also viewing teaching as the most important, would rank research above the more student-interactive advising. Canonical correlations among various scales reveal a strong emphasis on such issues of the importance of faculty respect for student views.
Distance learners were found to differ from face‐to‐face learners in that they were more ‘isolated’ and experienced lower levels of self‐confidence. They also displayed a higher desire for structure in their learning materials, and this was interpreted as a tactic for dealing with their more difficult learning situation. If distance educators react simply by providing the desired structure, they necessarily neglect certain desirable educational goals such as fostering willingness to organize one's own learning or to evaluate one's own work. For these reasons, it is preferable to try to develop teaching and learning approaches which help students master their difficulties, rather than accepting the difficulties as a limiting factor in distance learning settings.
The Course Experience Questionnaire (CEQ) has been developed in Australia as a performance indicator in order to measure the quality of teaching on particular degree programmes. Students’ responses to the CEQ have been shown to vary systematically (a) across different institutions, (b) with students’ rated satisfaction with their degree programmes as a whole and (c) with their approaches to studying in higher education. It is argued that the CEQ constitutes a global index of perceived teaching quality that can be used in a British setting.
How lecturers teach is supposed to be a reflection of what they believe in but sometimes it seems this is not always the case. This paper reports on two studies designed to examine how much consistency there is between psychology lecturers’ self-reports about the way they teach and their actual teaching practices. The first study used an adapted version of Ramsden’s (1992) Course Experience Questionnaire (CEQ) and the second, an adapted version of Norton, Morgan and Thomas’ (1995) Ideal Self Inventory (ISI). Lecturers’ responses to both measures were compared with those provided by their students. For the majority of both the CEQ and ISI items there were no statistical differences, suggesting an overall broad match between lecturers and students. The fact that there were some discrepancies raises interesting questions about how far it is possible to use students’ perceptions as a measure of lecturers’ consistency between their beliefs and practices.
This empirical study compares the effects of problem-based learning (PBL) and subject-based learning on 2nd-year occupational therapy students' experience of their courses. Two hundred and twenty-five 2nd-year students from six schools, classified as subject-based (traditional), problem-based or 'hybrid', completed the course experience questionnaire (CEQ). This reliable and valid self-reporting instrument evaluates students' perception of course features such as workload, independence and type of assessments. The results of the multivariate analysis of variance indicated that enhancement of learning was related directly to the extent to which the philosophy of problem-based learning was implemented. Those studying in the most 'pure' PBL courses had very strong perceptions of independence and choice, and felt that their assessment methods had fewer adverse consequences on learning. The findings are important in that such contexts are known to stimulate deep learning. Although PBL students are expected to play a large part in formulating their own goals, in this study they reported some lack of clarity of objectives. This raises issues in the debate about who really generates the objectives in a PBL course. The CEQ proved to be sensitive to subtle variations in curricular features, which recommends it for wider use in comparative studies.
Many colleges and universities have adopted the use of student ratings of instruction as one (often the most influential) measure of instructional effectiveness. In this article, the authors present evidence that although effective instruction may be multidimensional, student ratings of instruction measure general instructional skill, which is a composite of three subskills: delivering instruction, facilitating interactions, and evaluating student learning. The authors subsequently report the results of a metaanalysis of the multisection validity studies that indicate that student ratings are moderately valid; however, administrative, instructor, and course characteristics influence student ratings of instruction.
Positive self-concepts are important and desirable in a broad range of contexts, particularly in educational settings. However, surprisingly little attention has been paid to the nature, measurement and practical implications of university teachers’ self-concepts — their self-perceptions of their own teaching effectiveness. This chapter integrates extensive research literatures in self-concept and in students’ evaluations of teaching effectiveness (SETs) in order to develop a multidimensional university teaching self-concept instrument (Teacher’s Evaluation of Educational Quality; TEEQ) and evaluate its psychometric properties (factor structure, reliability, validity). A multitrait-multimethod analysis of relations between multiple dimensions of teaching self-concept and corresponding student rating dimensions provides good support for the construct validity of the teaching self-concept responses. Consistent with self-concept research, we hypothesized that feedback from significant others (students) influences self-concept. In support of this hypothesis, teacher self-concepts were moderately correlated with SETs (median r = .20) for teachers who had not previously received student feedback, whereas the agreement was substantially higher (median r = .40) for teachers who had previously received SET feedback. Implications for further research on teacher reflection and for improving teaching effectiveness in higher education are discussed.
This second in a two-part series of papers on the methodology of research on college impact is concerned with problems in collecting data and in measuring environmental attributes of colleges. While any method of data collection is subject to certain unique types of systematic errors, the investigator has available several techniques for detecting such errors or for minimizing their efforts on his conclusions. Although the most popular methods for environmental assessment have utilized student perceptions, inferences concerning the observed "effects" of such measures are ambiguous in view of the possibility that the student's perception of his institution can itself be influenced by institutional impact. One possible solution to this problem is to base environmental measures on directly observable events rather than on perceptions.
The Course Experience Questionnaire (CEQ) and a short form of the Approaches to Studying Inventory (ASI) were adapted for use in distance education and were administered to students taking six different courses with the Open University. The constituent structure of the CEQ was preserved in this distinctive context, and a second-order factor analysis confirmed its status as a measure of perceived academic quality. The students' scores on the individual scales of the CEQ and the ASI shared almost half of their respective variance. It is concluded that approaches to studying in distance education are strongly associated with students' perceptions of the academic quality of their courses.
The applicability paradigm has been used in five studies to evaluate the applicability of items from two North American instruments designed to measure students' evaluations of teaching effectiveness. In the present investigation, the paradigm is used at the new University of Western Sydney, Macarthur (UWSM). Items from both instruments were seen as appropriate and important, and differentiated among lecturers chosen as good, average and poor teachers. A multitrait-multimethod analysis of responses from the two instruments supported their convergent and divergent validity. The pattern of items judged to be most important at UWSM was more similar to patterns found at two research universities than patterns in the technical and further education sector or at two other institutions. These results support the applicability of the instruments at UWSM and across a diversity of educational settings.
Two student evaluation surveys developed in the United States were administered to a sample of Australian university students. Students were asked to evaluate one of the best and one of the worst lecturers in their university experience, to indicate inappropriate items, and to select the most important items. Each of the 63 items was seen as appropriate by most students, each item was chosen by at least a few students as being most important, and all items — except those related to Workload/Difficulty — differentiated between best and worst lecturers. Separate factor analyses of the two surveys revealed the same factors that had been identified in American settings. Furthermore there was good agreement between the factors from the two surveys that were hypothesized to measure the same components of effective teaching. The findings demonstrate the feasibility of evaluating effective teaching in Australian universities and the appropriateness of two American surveys to an Australian setting.
Research on students' approaches to learning in higher education has consistently demonstrated strong relationships between approaches to studying and perceptions of the learning environment. The vast majority of these studies have been carried out with Australian and British students. Using the Approaches and Study Skills Inventory for Students (ASSIST) and the Course Experience Questionnaire (CEQ), this study investigated these relationships with a large sample of Canadian undergraduates. The factor structure of the ASSIST was confirmed at the main scale level. The factor structure of the CEQ was largely confirmed, though some small changes were noted. Previous findings of significant correlations between approaches and CEQ scales were supported. The strongest relationships were found between heavy workload/inappropriate assessment and surface approach, and between generic skills and deep approach. Consistent with other studies, age was found to be a significant variable with regard to approaches. Implications for the practice of higher education staff development are discussed.
Ph.D. and research students' evaluations of supervision were collected across Australian universities to provide an extensive assessment of the quality of research supervision and appropriateness of research facilities. Here we evaluate issues, complexities, challenges, and appropriateness for using such ratings to make benchmarking comparisons between different universities and programs.
This paper discusses an electronic course survey system designed to support evaluation of a modular programme in management taught through distance education. The scale and communication issues raised by distance education, when compounded by a complex rolling modular programme, make conventional evaluation complicated, slow and expensive. To overcome these problems, an automated evaluation process was introduced, partly based on web-based surveying, but significantly emphasising the reporting process and allowing performance indicators and complex analyses to be embedded in high quality word-processed reports, that can be quickly and easily generated after each presentation of a module. The paper discusses the system and the impact that it has had on the evaluation process, before looking to future opportunities for developments in this field, and discussing some of the implications - both expected and unanticipated - that have come out of using automation in the evaluation process.
The literature abounds with psychometric studies of course evaluation measures and articles debating the merits of student ratings of instruction, but little research has focused on faculty perceptions of this procedure. In the present study faculty perceptions are explored at a teachers' college where evaluation is carried out annually on a sample of courses. The sample includes 101 instructors who completed the research questionnaire. Faculty attitudes reflected a broad range of responses towards validity of student ratings, and their usefulness for improving instruction. Although overall attitudes were mildly positive, few instructors reported changing instruction as a result of student ratings. Moreover, few supported sending evaluation results directly to college administrators or publishing them for student consumption.
Increasingly, evaluation is seen primarily as means of achieving quality improvement in higher education. The student-centred, open evaluation system in the medical programme at the University of Sydney has been developed with the fundamental aim of maintaining and improving programme quality. Students' potential concerns about their learning experiences are actively sought through a variety of qualitative and quantitative collection methods. Feedback is divided into three types: individual, group and year. Methods of data collection are described for each type of feedback and issues involved in managing feedback are discussed. Frequencies are reported for categories of individual student feedback in 1998 and 1999, classified according to whether comments were positive or negative. To effectively support improvement in course quality, an evaluation system must be based on a clear educational rationale, use a variety of methods and be managed with a sensitivity to the needs of students and teachers.
Assessment of teachers through student opinions has been the most used method to define the characteristics of teaching quality. There is no previous literature applying the Semantic Differential (SD) with this objective. SD can detect the attributes of the 'ideal teacher', and find the evaluative, emotional or affective components linked to this concept. In this work, we analyse the factorial structure of a SD scale, to test the hypothesis of the existence of EPA factors, usually found in research directed to different objects of attitude. The most relevant dimensions of the 'ideal teacher' show similar profiles to those dimensions obtained with other assessment procedures. This result confirms the adequacy of SD as an instrument to assess of quality teaching.
The idea of assessing how much students learn, improve, or grow in college, as well as how they stand at graduation, has been gaining momentum over the 'reputational' and 'resources' approaches. This study is to demonstrate how this new approach can work even if an institution has already committed to using an externally-developed survey instrument to assess student collegiate experience. It attempts to articulate student collegiate experience using self-reports and to construct the gain scales that can be used as the outcome measures in an institution's assessment efforts.
An investigation was conducted into 3- or 4-year departmental sets of student feedback questionnaire data from one university. Only four out of 25 departments had significant changes to any of the six dimensions in the 3- or 4-year period, and three of these significant changes were falls. There is, therefore, no evidence that the use of the questionnaire was making any contribution to improving the overall quality of teaching and learning of the departments, at least as perceived by the students. If it was, there should have been evidence of rising values. The following reasons why the use of the questionnaire might not have been conducive to improving teaching quality are discussed. The possibility that teaching quality is inherently stable is rejected. It is possible that feedback from the questionnaire was not used effectively. Related to this is whether instructors perceived that the university rewarded good teaching, so felt there was an incentive to make use of the feedback. The emphasis of the system was on appraisal, which might negate any developmental effect. The standard questionnaire and the associated procedures may have lacked flexibility and been inappropriate for innovative forms of teaching. The study questions whether student feedback questionnaires are utilising resources effectively if they are administered in an environment similar to the university in question, which appears reasonably typical.
The Course Experience Questionnaire (CEQ) has been developed in Australia as a performance indicator for measuring the quality of teaching on particular degree programmes. Students' responses to the CEQ have been shown to vary systematically (a) across different institutions, (b) with students' rated satisfaction with their degree programme as a whole, and (c) with their approaches to studying in higher education. In the present study, responses obtained from students in one British academic department replicated the broad constituent structure of the CEQ, although they also confirmed problems with the composition of one of its subscales concerned with appropriate assessment. It is concluded that the CEQ yields a global index of perceived teaching quality that can be used in a British setting.
This study investigates the association between students' perceptions of learning context and approaches to studying at an individual level. The emphasis of this study thus focuses on individual student differences and, in order to facilitate the analyses presented, a statistical procedure (multidimensional unfolding analysis) has been introduced that is new to educational research of this nature. This procedure is described in non-technical terms as well as in a more technical Appendix. The results of this study confirm the existence of the important association between qualitatively different perceptions of learning context and approaches to studying at the individual level. They illuminate, furthermore, the subtly different ways in which individual students orchestrate their approaches to studying in relation to their perception of contextual variables. The diagnostic value of an unfolding analysis is discussed and it is concluded that the methodology described in this study constitutes a potentially powerful paradigm for improving the quality of student learning by facilitating intervention in terms of the perception of the context in which learning occurs.
Performance indicators (PIs) in higher education have focused chiefly on research outputs. They have largely ignored the teaching function of universities and colleges. This article outlines the development of a student evaluation instrument designed to measure the teaching performance of academic organisational units. The theory of teaching and learning that underlies the Course Experience Questionnaire (CEQ) is described. The instrument's statistical qualities and its ability to discriminate intelligibly between different courses are discussed in the context of results from national trials in Australian higher education. The principal conclusion reached is that the CEQ offers a reliable, verifiable and useful means of determining the perceived teaching quality of academic units in systems of higher education that are based on British models. Several technical and political issues remain unresolved in its application as a PI.
The purposes of this monograph are to provide an overview of findings and of research methodology used to study students' evaluations of teaching effectiveness, and to examine implications and directions for future research. The focus of the investigation is on the author's own research that has led to the development of the Students' Evaluations of Educational Quality (SEEQ) instrument, but it also incorporates a wide range of other research. Based upon this overview, class-average student ratings are: (1) multidimensional; (2) reliable and stable; (3) primarily a function of the instructor who teaches a course rather than the course that is taught; (4) relatively valid against a variety of indicators of effective teaching; (5) relatively unaffected by a variety of variables hypothesized as potential biases; and (6) seen to be useful by faculty as feedback about their teaching, by students for use in course selection, and by administrators for use in personnel decisions. In future research a construct validation approach should be employed in which it is recognized that: effective teaching and students' evaluations designed to reflect it are multidimensional/multifaceted; there is no single criterion of effective teaching; and tentative interpretations of relationships with validity criteria and with potential biases must be scrutinized in different contexts and must examine multiple criteria of effective teaching.
The present study used meta-analytic methodology to synthesize research on the relationship between student ratings of instruction and student achievement. The data for the meta-analysis came from 41 independent validity studies reporting on 68 separate multisection courses relating student ratings to student achievement. The average correlation between an overall instructor rating and student achievement was .43; the average correlation between an overall course rating and student achievement was .47. While large effect sizes were also found for more specific rating dimensions such as Skill and Structure, other dimensions showed more modest relationships with student achievement. A hierarchical multiple regression analysis showed that rating/achievement correlations were larger for full-time faculty when students knew their final grades before rating instructors and when an external evaluator graded students’ achievement tests. The results of the meta-analysis provide strong support for the validity of student ratings as measures of teaching effectiveness.
This paper investigates the relationship between staff research and teaching in doctoral education with a special reference to disciplinary variations. The relationship between staff research (the 2001 RAE scores) and the effectiveness of doctoral education as perceived by students is analysed. On the whole, little relationship between departmental research performance (the Research Assessment Exercise scores) and effectiveness of doctoral education is found in Education and Chemistry, especially in the aspects of supervision. The results of research environment are more complex. Although the general findings — the absence of a significant relationship between research and teaching — apply to both Education and Chemistry departments, doctoral education is more favourably perceived by doctoral students in Chemistry than in Education. Finally, a theoretical diagram of Teamwork and Individualist research training structures in Ideal and Degenerative types is created to discuss these findings.
In recent years, measuring the efficiency and effectiveness of higher edu cation has become a major issue. National governments are now demanding greater public accountability for funds invested in the sector, resulting in the emergence of various performance indicators relating to both teaching and research. The Course Experience Questionnaire (CEQ) was developed to measure the perceived quality of teaching in degree programmes. It evolved from research that identified curriculum, teaching and assessment as key determinants of students' approaches to learning and, in turn, the quality of their learning outcomes. The CEQ data are intended for use in making comparisons within fields of study over time and/or across institutions. However, no European study has reported on its suitability to evaluate teaching within an accounting programme. This paper outlines the development of the CEQ and confirms its reliability and construct validity for use in the accounting discipline in an Irish context.
This paper reports on the use of the Course Experience Questionnaire (CEQ) as an instrument to monitor the medical programme at the University of Sydney and in particular to measure improvements in teaching quality with the introduction of the new graduate-entry problem-based programme. In addition, it raises the more general issue of interpretation of CEQ results in courses designed around problembased learning (PBL). Students' perceptions of teaching quality were sought using a whole class questionnaire survey and small group interviews. Students in the new programme rated their course more highly than did students in the old programme with respect to good teaching, appropriate assessment, generic skills and overall satisfaction. These improvements did not hold with respect to the clarity of goals and standards, nor for perceptions of an appropriate workload. The results are interpreted in context and it is argued that particular items in the CEQ do not reflect the educational philosophy or the instructional processes of PBL programmes.
Student evaluation of teaching is a commonly accepted means of obtaining feedback on the quality of university teaching. However, its usefulness in contributing to improved teaching performance is dependent on the extent to which staff respond to and apply the information obtained in this way. This paper reports the results of a project designed to extend the application of student evaluations by using them as the basis for staff development. Survey findings from 87 staff and 127 students indicate the priorities for academic and course development perceived by each group. Booklets based on these perceived needs were developed by collaborative staff-student groups and made available to enable staff to improve their teaching in response to student evaluations.
Mechanisms for the quality assessment of teaching in the higher education systems of the UK, The Netherlands, France and Germany give varying statuses to students’ assessment of teaching, specifically that done by means of questionnaires. Despite numerous assertions of the general validity of many aspects of such assessments, previous research — very little of which has been based upon the European experience — has nevertheless shown various biases in these evaluations (biases being defined as aspects of evaluation unrelated to the intrinsic characteristics of the teaching). It is also possible to hypothesise other sources of bias that are not analysed in depth in the existing literature; some of these may be specific to the higher education systems of individual countries. The possible existence of biases must necessarily entail some problems in the interpretation of questionnaire results and thus dilemmas in their application to decision‐making by institutions.[1] This is revised version of a paper presented to the Fourth International Conference on Assessing Quality in Higher Education at the University of Twente, Enschede, The Netherlands, 28–30 July 1992.
The "Students Evaluation of Educational Quality" is an instrument measuring dimensions of college teaching effectiveness. A study showed that, for ratings of 123 instructors in 3,079 classes over 13 years, each instructor has a distinct profile of ratings that generalizes over time, different courses, and course level. (Author/MSE)
A typical evaluation instrument used in the USA is subjected to critical analysis with reference to two paradigms of teaching, the transmission model and the engaged-critical model. This analysis reveals that the evaluation form is not representative of all conceptions of teaching, but is more consistent with the transmission paradigm. As a prescriptive cultural and educational artifact, the survey instrument implicitly militates against alternative models of teaching. The article questions the neutralitys of a document that purports to transcend different disciplines, epistemologies, teaching methods, conceptions of learning, and institutional use.
A number of recent articles in this journal, and two responses in the form of 'Points for Debate', have considered whether academics need training to support their teaching roles. This debate has been undertaken largely in the absence of evidence of the impact of training. This paper reports findings from the first stage of a study of the impact of initial training of university teachers in universities in the UK. Data from 72 teachers showed significant improvements in scores on three of the six scales of the Student Evaluation of Educational Quality (SEEQ) questionnaire after one semester of two- and three-semester long training programmes. Interpretation of this positive finding is complicated by problems with the self-selection of teachers, poor response rates and the difficulty of distinguishing the effects of training from the effects of experience of teaching. Improvements in the research design, implemented in a subsequent study, are outlined.
The pressure for accountability has led higher education institutions, even the 'elite' universities, in the United Kingdom to consider instituting teaching assessment by students. Both formal and informal methods of obtaining students' opinions are used in practice. A formal questionnaire administered at the end of term or semester is not useful for a teacher to change his or her approach to benefit the current batch of students taught by him or her. An approach that involves obtaining students' views and comparing them with self-ratings by teachers is proposed. Case study results are presented in support of the efficacy of this approach. The implications and limitations are discussed.