No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Cheat Sheet or Open-Book? A Comparison of the Effects of Exam Types on Performance, Retention, and Anxiety
Abstract
The differences between open-book, cheat sheet, and closed-book exams were examined in two different types of psychology courses. A total of 297 students enrolled in eight sections of Introductory Psychology and 99 students enrolled in four sections of Statistics participated in this study. Exam types were counterbalanced across sections of the same course. Students were given either open-book, cheat sheet, or closed-book exams, took a surprise quiz two weeks after the exams to measure retention of course material, completed a preference questionnaire, and took a pre-test measure of test anxiety on open-book and cheat sheet tests. While students did slightly better on open-book exam than on closed-book exams, they also much preferred open-book and cheat sheet exams over closed-book exams, and had lower levels of anxiety when taking open-book exams compared to cheat sheet exams. Based on these results, open-book exams may be a superior style of examination for a variety of psychology courses.
... For the quantitative study, a questionnaire was deployed by the researcher to obtain the necessary data concerning the participants' attitudes toward OBEs (Appendix 1). The questionnaire was developed based on the studies of Feller (1994), Eilertsen and Valdermo (2000), Roediger III and Karpicke (2006), Heijne-Penninga et al. (2010), Gharib et al. (2012), Teodorczuk et al. (2018), Francis (1982), Chan and Mui (2004), and Heijne-Penninga et al. (2011). The questionnaire consisted of 14 statements based on the Five-Point Likert scale. ...
... Take the first research question into consideration which states, "What is the attitude of the English Language and Literature Department students at Al-Balqa Applied University toward the OBEs?" The study found positive attitudes of the learners, and this finding is in line with the research conducted by Chan and Mui (2004), Gharib et al. (2012), Gujral and Gupta (2015), and Ramamurthy et al. (2016) where learners have exhibited highly positive attitude toward OBEs. The results of the current study also support the findings of a study by Doghonadze and Demir (2013), where the overall assessment found the students to perceive the OBE more positively than teachers, though none of them sees it as a perfect kind of exam. ...
... This makes the OBE preparation less stressful and more thorough. The results here support previous research efforts carried out by Chan and Mui (2004), Gharib et al. (2012), Doghonadze and Demir (2013), Gharib and Phillips (2013), and Ramamurthy et al. (2016). ...
Introduction
The paper aims to investigate the attitudes of English Language and Literature students toward implementing the open-book exam in literature courses.
Methods
The research applied a mixed-method approach. The researchers interviewed five undergraduate students enrolled in the American Literature course for the qualitative part. In contrast, the research includes the results of 62 questionnaires for the quantitative part. Moreover, a survey questionnaire was developed by the researchers to collect data concerning the participants’ attitudes toward OBEs.
Results
The study found a positive attitude toward OBEs. Moreover, the study participants demonstrated that OBEs reduce stress levels that occur during exams in general. A statistical representation of the findings showed that the sample students have a positive attitude in relation to this unconventional assessment protocol.
Discussion
OBEs should be allowed more space in academic institutions since today’s world does not depend on the amount of information a job applicant deposits in his memory. What matters is the person possesses enough high-order thinking abilities and skills like problem-solving.
... Students may not reflect what they know exactly in a test due to test anxiety. In addition to improving performance, the cheat sheet may help reduce test anxiety (Erbe, 2007;Gharib, Phillips & Mathew, 2012). ...
... Paquin, Miller and Baron (2020) also explored the effects of different exam types and reached the conclusion that the use of cheat sheets reduced exam failure rate from 11.8% to 0%. Likewise, Gharib, Phillips and Mathew (2012) compared different exam types, examined students' preferences and anxiety levels in introductory psychology course. They found out that students got higher scores on open-book exam than on closedbook exams, they preferred open-book and cheat-sheet exams over closed-book exams, and they were less anxious when they took open-book exams compared to cheat-sheet exams. ...
... The data about the opinions of prospective teachers were collected via "Standardized Interview Form" developed by the researcher after the literature review (Song, Guo & Thuente, 2016;Gharib, Phillips & Mathew, 2012;Cone, 2003;Wachsman, 2002). The interview form included both closed-and open-ended questions. ...
The aim of the study is to investigate the use of cheat sheets in teacher education. The research analyzed state test anxiety levels of prospective teachers, the correlation between their cheat sheet evaluation scores and exam scores, and opinions of prospective teachers about the cheat sheet. Thus, the study was conducted as a convergent parallel mixed-method research in which both quantitative and qualitative methods were used simultaneously. The study group consisted of 24 prospective teachers. Data regarding state test anxiety levels of the prospective teachers were gathered just before the exam via State Test Anxiety Scale. The cheat sheet evaluation rubric designed by the researcher was used to evaluate the cheat sheets prepared by the prospective teachers. The prospective teachers' exam scores were obtained from their responses. The data about opinions of the prospective teachers were collected via Standardized Interview Form developed by the researcher. The findings of the study revealed that prospective teachers' overall state text anxiety was at "very little" level. There was a statistically significant and positive correlation between the prospective teachers' cheat sheet evaluation scores and their exam scores. Qualitative findings revealed that the prospective teachers started taking notes at the beginning of the semester to write on their cheat sheets; however, they started to write their cheat sheets the day before the exam. They wrote their cheat sheets in 3-5 hours, and spent 3-5 hours studying for the exam. The prospective teachers rarely used their cheat sheets during the exam; however, the cheat sheet helped them learn the material, and they wanted to use cheat sheets in other courses. The prospective teachers used different strategies while writing their cheat sheets. Based on the conclusions of the study, teacher educators are also recommended to let prospective teachers use cheat sheets during exams to reduce their anxiety, to increase their exam scores, and most importantly, to help them learn the material thoroughly.
... [8,35,36] A notable benefit of open-book exams is their impact on student emotions, specifically reduced anxiety and increased confidence. [5,36,37] Allowing the use of resources during an exam, such as the use of notes, textbooks and websites or permission to collaborate with peers, is a form of cognitive offloading that can offer emotional and performance benefits as compared to exams with closed-book restrictions. [38] While an instructor still needs to make it clear what cognitive tools are allowed, students report feeling less anxious and more optimistic and relaxed when taking an open-book exam. ...
... [6,8,15] Students may better prepare for an open-book exam, because they know that cramming and memorization won't provide the required level of conceptual understanding. [8,36,37] Alternatively, students may be over-confident and dedicate less effort into studying for an open-book exam, because they feel they can "wing it" and hunt for answers during the test. [5,6,15] Authors correlate long-term retention of information to the effort and depth of pre-exam preparation in either format. ...
... [5,13,40] Alternatively, students may perform worse on open-book exams, because they underestimated the difficulty of the questions, or they spend too much time hunting for information rather than formulating their responses. [27,37] From an instructor perspective, implementing an openbook exam is not as simple as repurposing a closed-book exam yet permitting students access to resources and peers during the exam. Questions best suited to an open-book context go beyond basic recall, targeting higher-order thinking and real-world problem solving. ...
Assessment is an essential step in the teaching and learning process. Traditional examination methods (closed-book, time-constrained, invigilated, multiplechoice) prevail in higher education despite support foralternative approaches wherein students construct knowledge through active, authentic activities. A review ofthe scholarly literature focused on merits and limitations of traditional closed-book exams in-person and in anonline, remote course delivery context, as well as benefits, concerns, and considerations of transitioning to open-book exams at a time of upsurge in online learning. Within the dichotomy of traditional versus alterative exam strategies, the literature is inconclusive on shared matters, including student academic integrity, study habits, anxiety, performance, and long-term retention of information.
... In addition, some studies have shown that the testing effect is similar between open-book and closed-book tests, even though contrary results have also been reported. In a review of the literature comparing open-book and closed-book test formats, Durning and colleagues (2016) reported that among the five studies that examined the testing effect between these formats, four showed that the testing effect was similar between these test formats (Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008;Agarwal & Roediger, 2011;Gharib, Phillips, & Mathew, 2012;Pauker, 1974), whereas, one showed that the testing effect was lower in the open-book format than in the closed book format (Moore & Jensen, 2007). However, the results are not as straight forward as it appears. ...
... Because this issue has an important implication for educational practice, it warrants further investigation. As noted above, the results of research are conflicted as to whether open-book and closedbook tests differ in promoting long-term retention of studied material, with some studies showing that these two formats are similar (e.g., Agarwal et al., 2008;Agarwal & Roediger, 2011;Pauker, 1974;Gharib et al., 2012) and the other studies showing that open-book tests are less effective than closed-book tests, particularly, in promoting deep learning (e.g., Agarwal & Roediger, 2011;Pauker, 1974). However, not all learning situations require deep learning, such as during the early stage of learning. ...
... In contrast, if the focus is shifted toward a long-term goal of education (i.e., building knowledge), it may not matter whether a test is open-book or closedbook because what matters most is whether students will develop knowledge of whatever they are learning. In fact, the past studies investigating the effect of test format using the testing effect paradigm showed that both open-book and closedbook formats produced similar performance on the final closed-book test, indicating that both formats would promote long-term memory (e.g., Agarwal et al., 2008;Agarwal & Roediger, 2011;Pauker, 1974;Gharib et al., 2012). However, a concern with these studies is that they used text-based materials, and therefore, it is possible that the lack of difference between open-book and closed-book formats was simply reflecting the fact that the materials were well-organized and meaningful, thereby conducive to elaborative processing regardless of the format of the initial quiz. ...
If assessment is the purpose of testing, open-book tests may defeat the purpose. However, a goal of education is to build knowledge, and based on the literature, open-book tests may not be inferior to closed-book tests in promoting long-term retention of information. Participants studied Swahili-English pairs and either re-studied or took an initial quiz, which was cued recall or recognition in an open-book or closed-book format. One week later, the final closed-book recognition test showed higher performance in the quizzed conditions than in the study-twice condition, replicating the testing effect. However, performance was similar across the quizzed conditions, indicating that testing promoted long-term retention regardless of test format (open-book versus closed-book) and test type (cued recall versus recognition). Open-book tests are not inferior to closed-book tests in building knowledge and can be particularly useful in online classes because preventing cheating is difficult when closed-book tests are administered online.
... One active control condition that is particularly informative when it comes to retrieval practice as the source of the testing effect is open-book testing (Agarwal et al., 2008;Agarwal and Roediger, 2011;Gharib et al., 2012;Roelle and Berthold, 2017). Here, students are allowed to consult their notes or textbooks while taking the (practice) test. ...
... There is also one field experiment investigating the benefits of open-book versus closed-book tests, which was conducted with students in an introductory psychology course (Gharib et al., 2012). Here, the manipulation was slightly different in that actual exam types (closed-book, open-book, and cheat sheet) were manipulated rather than no-stakes or low-stakes practice tests. ...
... Thus, final test delays of up to 1 week might have been too short to detect a retrieval practice effect compared to openbook testing. The study by Gharib et al. (2012) used a delay of 2 weeks but their findings are not easily comparable to the other studies, as the manipulation did not concern a learning condition in terms of a practice test but an exam condition: students had either access to their notes, to a self-prepared cheat sheet, or to none of the former in the exam. In this case, potential differences due to the actual exam might have been obscured because students prepare differently for closed-book and for open-book exams (e.g., Theophilides and Koutselini, 2000). ...
The present field study compared open-book testing and closed-book testing in two (parallel) introductory university courses in cognitive psychology. The critical manipulation concerned seven lessons. In these lessons, all students received two to three questions concerning the content of the respective lesson. Half the participants (open-book group) were allowed to use their notes and the course materials, which had been distributed at the beginning of each class; the other half was not allowed to use these materials (closed-book group). A surprise test conducted in the eighth week demonstrated better results for the closed-book group. Further 6 weeks later, the final module exam took place. A number of questions in this exam concerned the learning matters instructed during the critical seven lessons. Even with respect to these questions, the closed-book group performed better than the open-book group. We discuss these results with respect to two possible explanations, retrieval practice and motivational differences.
... In addition, some studies have shown that the testing effect is similar between open-book and closed-book tests, even though contrary results have also been reported. In a review of the literature comparing open-book and closed-book test formats, Durning and colleagues (2016) reported that among the five studies that examined the testing effect between these formats, four showed that the testing effect was similar between these test formats (Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008;Agarwal & Roediger, 2011;Gharib, Phillips, & Mathew, 2012;Pauker, 1974), whereas, one showed that the testing effect was lower in the open-book format than in the closed book format (Moore & Jensen, 2007). However, the results are not as straight forward as it appears. ...
... Because this issue has an important implication for educational practice, it warrants further investigation. As noted above, the results of research are conflicted as to whether open-book and closedbook tests differ in promoting long-term retention of studied material, with some studies showing that these two formats are similar (e.g., Agarwal et al., 2008;Agarwal & Roediger, 2011;Pauker, 1974;Gharib et al., 2012) and the other studies showing that open-book tests are less effective than closed-book tests, particularly, in promoting deep learning (e.g., Agarwal & Roediger, 2011;Pauker, 1974). However, not all learning situations require deep learning, such as during the early stage of learning. ...
... In contrast, if the focus is shifted toward a long-term goal of education (i.e., building knowledge), it may not matter whether a test is open-book or closedbook because what matters most is whether students will develop knowledge of whatever they are learning. In fact, the past studies investigating the effect of test format using the testing effect paradigm showed that both open-book and closedbook formats produced similar performance on the final closed-book test, indicating that both formats would promote long-term memory (e.g., Agarwal et al., 2008;Agarwal & Roediger, 2011;Pauker, 1974;Gharib et al., 2012). However, a concern with these studies is that they used text-based materials, and therefore, it is possible that the lack of difference between open-book and closed-book formats was simply reflecting the fact that the materials were well-organized and meaningful, thereby conducive to elaborative processing regardless of the format of the initial quiz. ...
If assessment is the purpose of testing, open-book tests may defeat the purpose. However, a goal of education is to build knowledge, and based on the literature, open-book tests may not be inferior to closed-book tests in promoting long-term retention of information. Participants studied Swahili-English pairs and either re-studied or took an initial quiz, which was cued recall or recognition in an open-book or closed-book format. One week later, the final closed-book recognition test showed higher performance in the quizzed conditions than in the study-twice condition, replicating the testing effect. However, performance was similar across the quizzed conditions, indicating that testing promoted long-term retention regardless of test format (open-book versus closed-book) and test type (cued recall versus recognition). Open-book tests are not inferior to closed-book tests in building knowledge and can be particularly useful in online classes because preventing cheating is difficult when closed-book tests are administered online.
... Most of the research cited above examined the relationship between test performance and whether or not a student used a help sheet. Only a few studies (Dickson & Miller, 2006;Gharib, Phillips, & Mathew, 2012;Visco, et al., 2007) have explored how the specific content of a help sheet is related to performance. Dickson and Miller (2006) found significantly higher test performance when students used an instructor provided help sheet compared to a student provided help sheet. ...
... Visco et al. (2007), examined student generated help sheets and concluded that students likely need additional direction on what content to include on a help sheet in order to enhance performance. Finally, Gharib et al. (2012) examined the quality of students' help sheets and found a reliable and positive relationship between the quality of the help sheet content and test performance; where a quality measure was obtained by rating a help sheet for organization and amount of detail. ...
... Results of the preliminary analyses suggest that it is not enough just to consider whether a student has access to a help sheet or not, but rather a careful examination of the help sheet content is required. Similar to Gharib et al. (2012), overall quality of the help sheet was found to be a very important characteristic of the help sheet. As overall quality increased, test scores also increased. ...
In this essay, we discuss the value of open educational resources to increase student learning opportunities, as well as their challenges and promises.
... On the other hand, it has been reported that OBEs do not reduce test anxiety relative to CBEs among students who find OBEs unfamiliar and unpredictable and felt pressured to write answers using their knowledge [16,17]. Some students' responses indicated that the preparation for the OBEs was a burden, mainly due to a lack of understanding of how to prepare. ...
... Assessment workshops for faculty training could be adopted to share pilot results and practical tips in adopting OBEs for their courses. Test anxiety has reportedly been associated with poor performance in exams due to worrying about the outcome and experiencing negative emotions during the test [17]. Previous studies found a significant negative correlation between test anxiety and academic achievement, including students' GPA [18]. ...
Background
While closed-book examinations (CBEs) have traditionally been implemented in dental education, open-book examinations (OBEs) are being introduced for the purpose of acquiring higher levels of knowledge and promoting long-term memory. This study examines whether it is effective to use a blended assessment of CBEs and OBEs for dental students to reduce test anxiety and enhance academic performance.
Methods
Using a quasi-experimental research method, a blended assessment that combined CBEs in class and OBEs online was designed for a dental course. In 2020, when the pandemic was at its peak, student assessment was ineffective, and the 2020 cohort was omitted for our study; instead, two cohorts of predoctoral dental students (N = 178) enrolled in Restorative Dentistry in the spring semesters of 2019 and 2021 were included in the study. These students were informed about the experimental design, and they provided written consent for data collection, thereby voluntarily participating in the survey. Their self-perceived responses to open-ended survey questions on assessment methods were qualitatively analyzed.
Results
There was no significant difference in test anxiety between the CBEs and OBEs at the p-values of 0.001 in all items. Traditional and blended assessment showed a similar trend of lower scores in midterm exams compared to higher scores in final exams, thus discriminating against students’ performances. In particular, a low-achieving group was better predicted by a blended assessment. An analysis of the students’ self-perceived responses produced highly topical themes, including exam burden, learning effects, and fairness issues.
Conclusions
This study confirmed the feasibility of blended assessment that can be implemented in online and in-person educational environments. Moreover, it can be used as the groundwork to develop new models of assessment in dental education.
... Likewise, a study compared the differences between students' scores in OB and CB exams in different types of psychology courses. The authors found that students did better on OB exams [12]. The previous review showed inconsistent findings in relation to the impact of resource access on students' scores. ...
... Some researchers found no difference in marks [7,17,18]. Other researchers found higher marks among students who took the online test [11,12,19]. Anaya, et al. [20] had mixed results, with students in face-to-face tests attaining higher scores for some classes and lower scores for the other classes, and overall results were not significant. ...
Background
Access to electronic (E) resources has become an indispensable requirement in medical education and practice.
Objective
Our objective was to assess the effect of E-resources access during examination on end-course-exam scores of medical and dental students.
Methods
A quasi-experimental study which included two cohorts of medical ( n = 106 & 85) and three cohorts of dental students ( n = 66, 64 and 69) who took end-course- exams. Each exam was composed of two- parts (Part I and II), that encompassed equal number of questions and duration. Access to E-resources was allowed in part-II only. Items Difficulty Index (DI), Discrimination Index, (DisI), Point Biserial, (PBS) and cognitive level were determined.
Results
The study included 390 students. The proportion of items at various levels of DI, DisI, and PBS and the average values for item DI, DisI in both parts of each exam were comparable. The average scores in part-II were significantly higher than part-I ( P < 0.001, < 0.001 and 0.04) and lower-order cognitive-level items scores were higher in three exams ( P < 0.0001, 0.0001, 0.0001). Higher- order cognitive level items scores were comparable between part I and II in all courses. The significant factor for change in marks were questions cognitive level and type of the course.
Conclusion
Access to E-resources during examination does not make a significant difference in scores of higher-order cognitive level items. Question cognitive level and course type were the significant factors for the change in exam scores when accessing E-resources. Time-restricted E-resources accessed tests that examine higher cognitive level item had no significant academic integrity drawback.
... Assessments implemented during COVID-19 included: online open or closed-book examinations, examination cancellation with increased weighting of other assessments, or coursework-based assessment. There are currently contradictory findings in the literature over how the discriminative ability of OBEs compares to closedbook examinations (CBE) traditionally used during in-person invigilated assessment [13][14][15][16][17][18][19][20][21]. While some studies have shown comparable student performance [13][14][15][16], pass rates [20] and ability to discriminate in OBE and CBE, other research demonstrates CBEs are superior to OBEs in these respects [17] and vice versa [21]. ...
... There are currently contradictory findings in the literature over how the discriminative ability of OBEs compares to closedbook examinations (CBE) traditionally used during in-person invigilated assessment [13][14][15][16][17][18][19][20][21]. While some studies have shown comparable student performance [13][14][15][16], pass rates [20] and ability to discriminate in OBE and CBE, other research demonstrates CBEs are superior to OBEs in these respects [17] and vice versa [21]. Even when the cohort as a whole show similar performances in both assessment scenarios, it is impossible to demonstrate whether the performances of each individual have stayed at the same levels [22,23]. ...
Background
COVID-19 posed many challenges to medical education in the United Kingdom (UK). This includes implementing assessments during 4 months of national lockdowns within a 2-year period, where in-person education was prohibited. This study aimed to identify medical school assessment formats emerging during COVID-19 restrictions, investigate medical students’ perspectives on these and identify influencing factors.
Methods
The study consisted of two phases: a questionnaire asking medical students about assessment changes they experienced, satisfaction with these changes and preference regarding different assessments that emerged. The second phase involved semi-structured interviews with medical students across the UK to provide a deeper contextualized understanding of the complex factors influencing their perspectives.
Results
In the questionnaire responses, open-book assessments had the highest satisfaction, and were the preferred option indicated. Furthermore, in the case of assessment cancellation, an increase in weighting of future assessments was preferred over increase in weighting of past assessments. Students were also satisfied with formative or pass-fail assessments.
Interview analyses indicate that although cancellation or replacement of summative assessments with formative assessments reduced heightened anxiety from additional COVID-19 stressors, students worried about possible future knowledge gaps resulting from reduced motivation for assessment-related study. Students’ satisfaction level was also affected by timeliness of communication from universities regarding changes, and student involvement in the decision-making processes. Perceived fairness and standardisation of test-taking conditions were ranked as the most important factors influencing student satisfaction, followed closely by familiarity with the format. In contrast, technical issues, lack of transparency about changes, perceived unfairness around invigilation, and uncertainty around changes in assessment format and weighting contributed to dissatisfaction.
Conclusions
Online open-book assessments were seen as the most ideal amongst all participants, and students who experienced these were the most satisfied with their assessment change. They were perceived as most fair and authentic compared to real-life medical training. We seek to inform educators about student perceptions of successful assessment strategies under COVID-19 restrictions and provide evidence to allow debate on ongoing assessment reform and innovation. While this work looks specifically at assessment changes during COVID-19, understanding factors affecting student perception of assessment is applicable to examinations beyond COVID-19.
... First, proponents of open-note exams claim students learn how to gather and critically analyze material from multiple sources, as opposed to closed-note exams which reward short-term storage and quick retrieval [9][10][11]. Additionally, open-note exams decrease students' exam anxiety [12][13][14] and minimize a desire to cheat [15,16]. Green, Ferrante, and Heppard (2016) suggested that (1) higher education must evolve along with culture and technology, and (2) the ability to answer fact-based multiple-choice questions by rote memory is not adequately preparing students for future careers. ...
... However, this warrants further investigation, especially because students were able to see their actual score on the exam prior to taking the survey asking about their perceived performance. Additionally, students reported decreased test anxiety levels, which also aligns with previous literature [12][13][14]52]. ...
Although closed-note exams have traditionally been used to evaluate students in undergraduate biology classes, open-note exams are becoming increasingly common, though little is known about how students prepare for these types of exams. We investigated student perceptions of and their preparation habits for online open-note exams in an undergraduate biology class, as compared to their previous experiences with closed-note exams in other classes. Specifically, we explored the following research questions: (1a) How do students perceive open-note exams impact their exam scores, their anxiety, the amount they studied, and the amount their peers studied? (1b) How do these perceptions impact performance outcomes? (2a) How do students prepare for open-note exams? (2b) How do these preparation methods impact performance outcomes? Results demonstrate students perceived increased exam scores, decreased exam-anxiety, decreased study time spent personally, and decreased study time spent by their peers for open-note exams, as compared to past experiences with closed-note exams. Open-ended survey responses analyzed through first- and second-cycle analyses showed students adapted their study habits by focusing on note preparation and broad conceptual understanding rather than rote memorization. Using linear mixed effects models to assess student performance, we found students who focused on understanding, note preparation and using external resources outperformed students who did not report those study habits. As institutions shift towards flexible and scalable assessments that can be used in face-to-face or online environments, the use of open-note exams can promote effective study habits and reward higher-order thinking with intentional guidance from the instructor.
... The value of game-based quiz for learning has been wellinvestigated recently in higher education settings (2,(31)(32)(33). Traditional in-class quizzes, closed or open-book, have been routinely used as a summative assessment. ...
... Instructors may or may not discuss the quiz questions. On the other hand, an openbook quiz seems to reduce anxiety, puts less weight on memorisation, encourages deeper engagement with the course materials and is more realistic as it mimics the real-life working environment (33). Taking a step further, when learning, formative assessment and game elements are combined, it could potentially enhance student engagement in the course, encouraging learning without threatening esteem (30,34). ...
Introduction: In the context of pharmacy education worldwide and in Malaysia, the use of digital technologies to promote higher level thinking and discussions is seen as preparing the millennials as pharmacists in the 21st century. Together with leveraging on millennials' penchant for mobile technology, gamified online quizzes as an assessment tool that help promote active and collaborative learning in a Medicinal Chemistry course have been used.
Objectives: This study investigates students’ perception of the impact of gamified online quizzes on their learning in a Medicinal Chemistry course.
Method: This study employs mix method research comprising descriptive analysis, content analysis from informal chats and researchers' observation to gather the findings for the study. Three gamified online quizzes using Quizizz, were implemented outside classroom time, in place of traditional quizzes. Multiple attempts were allowed within a stipulated time. As interventions, post-quiz discussions were conducted during class time. Students completed an end- of-the-course survey.
Results: Out of 63 respondents, more than 96% felt that the gamified online quizzes enhanced their learning as they learned from the instant feedback, their mistakes and post- quiz discussions. Overall student performance based on the percentage and accuracy of answering the quiz improved with time. Student qualitative comments on the survey, the course social media (closed group) and informal chats supported the findings from the descriptive data analysis of the study.
Conclusions: From students’ perception, the gamified online quizzes were found to be enjoyable and effective in enhancing active, peer learning in an undergraduate medicinal chemistry course outside class time. For instructors, the online quiz served as an efficient tool for formative assessment in a large classroom setting, and could replace traditional classroom quizzes.
... Agarwal & Roediger, 2011;Agarwal et al., 2008). In the classroom, some studies show no difference between open-and closed-book exams (Gharib et al., 2012;Pauker, 1974), whereas others suggest that closed-book exams may lead to more learning (Moore & Jensen, 2007;Rummer et al., 2019; of course, these classroom findings could result from students studying more for closed-book tests, an indirect benefit of test format). ...
... Although not a direct comparison, the present results are consistent with the limited prior work on open-book versus closed-book exams. Just as the present experiments found no learning differences between open-book and closed-book essays, multiple prior studies found no differences on a delayed final exam after students had taken either an initial open-book or closed-book test (Agarwal et al., 2008;Gharib et al., 2012). A common pattern found in these experiments is that students do better on the initial test when it is open-book, similar to how participants in the present studies included more content and wrote better quality essays in the open-book condition. ...
Knowing when and how to most effectively use writing as a learning tool requires understanding the cognitive processes driving learning. Writing is a generative activity that often requires students to elaborate upon and organise information. Here we examine what happens when a standard short writing task is (or is not) combined with a known mnemonic, retrieval practice. In two studies, we compared learning from writing short open-book versus closed-book essays. Despite closed-book essays being shorter and taking less time, students learned just as much as from writing longer and more time intensive open-book essays. These results differ from students’ own perceptions that they learned more from writing open-book essays. Analyses of the essays themselves suggested a trade-off in cognitive processes; closed-book essays required the retrieval of information but resulted in lower quality essays as judged by naïve readers. Implications for educational practice and possible roles for individual differences are discussed.
... The value of game-based quiz for learning has been wellinvestigated recently in higher education settings (2,(31)(32)(33). Traditional in-class quizzes, closed or open-book, have been routinely used as a summative assessment. ...
... Instructors may or may not discuss the quiz questions. On the other hand, an openbook quiz seems to reduce anxiety, puts less weight on memorisation, encourages deeper engagement with the course materials and is more realistic as it mimics the real-life working environment (33). Taking a step further, when learning, formative assessment and game elements are combined, it could potentially enhance student engagement in the course, encouraging learning without threatening esteem (30,34). ...
Introduction: In the context of pharmacy education worldwide and in Malaysia, the use of digital technologies to promote higher level thinking and discussions is seen as preparing the millennials as pharmacists in the 21st century. Together with leveraging on millennials' penchant for mobile technology, gamified online quizzes as an assessment tool that help promote active and collaborative learning in a Medicinal Chemistry course have been used. Objectives: This study investigates students’ perception of the impact of gamified online quizzes on their learning in a Medicinal Chemistry course. Method: This study employs mix method research comprising descriptive analysis, content analysis from informal chats and researchers' observation to gather the findings for the study. Three gamified online quizzes using Quizizz, were implemented outside classroom time, in place of traditional quizzes. Multiple attempts were allowed within a stipulated time. As interventions, post-quiz discussions were conducted during class time. Students completed an end- of-the-course survey. Results: Out of 63 respondents, more than 96% felt that the gamified online quizzes enhanced their learning as they learned from the instant feedback, their mistakes and post- quiz discussions. Overall student performance based on the percentage and accuracy of answering the quiz improved with time. Student qualitative comments on the survey, the course social media (closed group) and informal chats supported the findings from the descriptive data analysis of the study. Conclusions: From students’ perception, the gamified online quizzes were found to be enjoyable and effective in enhancing active, peer learning in an undergraduate medicinal chemistry course outside class time. For instructors, the online quiz served as an efficient tool for formative assessment in a large classroom setting, and could replace traditional classroom quizzes.
... Principle courses of a quantitative nature, such as engineering economy, often permit the use of a reference during examination. Several studies have cited benefits from the approach such as reduced test anxiety [8][9][10], encouraged preparation [11], and discouraged cheating [12]. As a result, instructors frequently permit students to use resources, such as: (1) the text book, to encourage use of long-term reference; (2) the class notes, to encourage good note taking and class attendance; (3) a notecard, to encourage focused study and prioritization of important topics; (4) an instructor-prepared formula reference sheet; or (5) a student-prepared "cheatsheet", to encourage thorough review and organization of the material. ...
... Responses include: (a) always, (b) sometimes, and (c) rarely. 10. I have difficulty locating and retrieving information from my cheat sheets during exams. ...
... In comparison, students who were given more control over their testing situation often reported lower test anxiety (Gharib et al., 2012;Zeidner, 1998). Online exams allowed students to control many anxiety-provoking factors; they can create a comforting environment rather than a "cold exam hall", the relaxed timeframe gave students more control over their pace through the exam. ...
In New Zealand, the COVID-19 pandemic arrived soon after the 2020 academic year began, posing additional challenges for first-year students navigating the transition to university while adapting to the pandemic. This thesis explores the experience of first-year students as they navigate this dual challenge. Using Interpretative Phenomenological Analysis (IPA), three participants from Massey University Albany Campus were interviewed to describe the pandemic's impact on their wellbeing using the Te Whare Tapa Whā model, and its impact on their university experience and day-today learning. Analysis revealed four themes. In Disruption, the participants struggled with the very concept of a pandemic, causing intangible losses and grief. Wellbeing focussed on the pandemic's impact on participants' physical and mental health. The participants demonstrated divergent pathways to mental ill-health over time. Lockdown with Others highlighted how lockdown brought the household members' different lifestyle needs into conflict and created distractions to the participants' studies. Lastly, Studying during the Pandemic covers the participants' initial adjustment to the university and their experience during Auckland's two lockdowns. The participants reported less collaboration with others and reduced motivation when studying online. This thesis is the first to examine the collision of the pandemic and first-year university experience in New Zealand. The results highlight the pandemic's lasting impact on the participants' worldview. As with other research, the participants had unrealistic expectations of university. They attributed the failure to experience their idealised university life to the pandemic, perceiving their university experience as not a genuine one. Particular findings of note include the unintended impact of the university's grade-adjustment policy reducing student motivation for learning; the impact of sleeping patterns during the lockdown on 2 household dynamics; and the participants' framing of physical exercise during the lockdown as a space-claiming action. Practice recommendations based on these findings include clear communication of academic expectations at university to first-year students; incorporating independent learning into the core first-year curriculum; and avoiding technology solutionism in online learning by positioning authentic relationships between faculty and students, and between the students, as the basis of the student experience.
... Although Covid English teams are endeavouring to minimise the anxiety that arises from this, the effect is not remarkable. Perhaps by adding an open-book exam to the assessment, learners would be more free to find material and consolidate their knowledge (Gharib et al., 2012), which would greatly alleviate their test anxiety. . Unlike other courses, Covid English was scheduled for August, when undergraduate students had not yet started their official journey of the September semester. ...
Loneliness and anxiety are one of the well-researched concepts in education. Some educationalists recognise the loneliness and anxiety of students as important issues to consider (Pekrun, 2006; Hawkley & Cacioppo, 2010; Odacı & Kalkan, 2010). Admittedly, with the Covid-19 pandemic and the development of technology, language classrooms present unprecedented loneliness and anxiety, and potential changes in teaching formats, contents and pedagogy. In response to the negative emotional factor that students could encounter in Covid-19, the article, firstly, takes the context of an online course Development Project (DP) in the UK as an entry point. This is followed by introducing and explaining the teachers' measures to cope with loneliness and anxiety based on the DP, examples include discussion boards, images, FLCAS etc. Finally, an attempt will be made to criticize and analyse some points that the researchers did not consider such as the nature of the course itself and teacher anxiety. To sum up, this article may help to think richly about students' anxiety and loneliness in Covid-19 and relief about them.
... It has also been claimed that when open book exams are conducted online they lead to more academic dishonesty (Moralista & Oducado, 2020) and students also face challenges because of low internet connectivity (Agormedah et al 2020;Agung et al., 2020;Farooq et al., 2020;Figueroa et al., 2020;Hodgson & Hagan, 2020;Paudel, 2020;Qazi et al., 2020). (Agarwal et al., 2008;Francis, 1982;Gharib et al., 2012;Krarup et al., 1974;Ashri & Sahoo, 2021). ...
Educational deliberations in the recent years have been emphasizing the need for organic modes of assessment rather than limiting the scope to mere testing. Recently the National Education Policy 2020 stressed that curricular design should also create a 'robust system of continuous formative/adaptive assessment to track and thereby individualize and ensure each student's learning' (2020). Educational deliberations emphasize the need for developing critical thinking or higher order thinking as a thrust area and this is being reflected in most of the approaches and methods adapted for classroom teaching. However this may not reach fruition unless the testing and assessment align with this mindset. These hopeful intentions must be converted to realistic implementations for objectives to be realized, and assessment plays a pivotal role in the same. A natural transition from closed book or traditional examination to Open Book Exams may assist in students being able to apply acquired knowledge rather than recalling retained information. This article attempts to build a case for the use of open book exams by investigating it's history, types, formats, challenges and opportunities it provides for better teaching and learning.
... The candidates decide to study less and take exam preparations much less seriously (Parker et al., 2021). Empirical studies confirm students spend less time in exam preparation in THE compared to ICE (Gharib et al., 2012). ...
COVID-19 gave universities and colleges no choice. They had to switch to digital teaching and introduce home-based exams as a substitute for ordinary school exams. At the same time, the ambitions were to maintain the student's learning outcomes and ensure the exam grade measured the students' knowledge and skills. With data from a Norwegian business school, this paper will analyse if home based exams provide other results than traditional school exams with closed books. The chosen method is to compare achievements before and during the pandemic and link the performance to academic skills in other subjects and from upper secondary school. The results suggest that the measurement of grades changed under COVID-19. This applies to the quantitatively oriented subjects and the non-quantitative oriented subjects. This is useful knowledge since students' grades are used for ranking for further studies and professional careers.
... However, no assessment test solution for an online course is perfectly practical. For example, although open-book exams tend to assess thinking skills, it may lead students hunting for answers instead of spending time understanding what they have learned [20][21][22]. As a result, academic honesty and effective assessment pose additional challenges to the quality of online education. ...
Online learning has emerged as a widely used learning mode and will likely supplement traditional learning in the post-pandemic era. The purpose of this study is to present student voices of online school education by investigating students’ online learning experiences during the COVID-19 pandemic in various contexts, and explain why the impacts are important to student learning and well-being. Semi-structured in-depth interviews were conducted with nine students from China, Lebanon, and the United States to gain direct insight into students’ perceptions of each country. The results showed that the online learning environment provided at the national level, such as social conflicts, and the facilities provided at the individual level, such as information access, increase the educational inequity. High-school students experienced numerous psychological changes and encountered academic cheating issues in the home online-learning environment. We recommend that online school education should make significant improvements in pedagogy, students’ mental health, and learning assessment, and consider factors beyond technology solutions.
... Some researchers found no difference in marks [11,17,18]. Other researchers found higher marks among students who took the online test [19,20,12,13]. Anaya, et al. [21] had mixed results, with students in face-to-face tests attaining higher scores for some classes and lower scores for the other classes, and overall results were not signi cant. ...
Background
Access to electronic (E) resources has become an indispensable requirement in medical education and practice.
Objective:
Our objective was to assess the effect of E-resources access on end-course-exam scores of health profession students
Methods:
A quasi-experimental study which included two cohorts of medical (n=106 & 85) and three cohorts of dental students (n= 66, 64 and 69) who took end-course- exams. Each exam was composed of two- parts (Part I and II), that encompassed equal number of questions and duration. Access to E-resources was allowed in part-II only. Items Difficulty Index (DI), Discrimination Index, (DisI), Point Biserial, (PBS) and cognitive level were determined.
Results:
The study included 390 students. The proportion of items at various levels of DI, DisI, and PBS and the average values for item DI, DisI in both parts of each exam were comparable. The average scores in part-II were significantly higher than part-I (P<0.001, <0.001 and 0.04) and lower-order cognitive-level items scores were higher in three exams (P<0.0001, 0.0001, 0.0001). Higher- order cognitive level items scores were comparable between part I and II in all courses. The significant factor for change in marks were questions cognitive level and type of the course
Conclusions:
Access to E-resources does not make a significant difference in scores of higher-order cognitive level items. Question cognitive level and course type were the significant factors for the change in exam scores when accessing E-resources. Time-restricted E-resources accessed tests that examine higher cognitive level item had no significant academic integrity drawback.
... Previous work has shown that students have lower anxiety in open book exams compared with closed book exams (Gharib et al., 2012) and in multiple choice exams compared to open question exams (Hembree, 1988 ...
Background
Online and blended learning need an appropriate assessment strategy which ensures academic integrity. During the pandemic, many universities have chosen for online proctoring. Although some earlier examples suggest that online proctoring may reduce cheating, the potential side‐effects of proctoring are largely unknown.
Objectives
Therefore, this study aims to identify the effects of proctoring on students' self‐reported temptation to cheat and potentially undesirable side‐effects, including test anxiety, perceived exam difficulty, and performance. In addition, we examine which contextual and student characteristics affect test anxiety during online exams.
Methods
For this, we collected four waves of survey data throughout a year of teaching during the pandemic at one faculty of a Dutch university, resulting in a total sample of 1760 students within 105 courses.
Results and Conclusions
Multi‐level analyses showed that while proctoring had no effect on the temptation to cheat, exam difficulty or performance, students reported higher levels of test anxiety. Some learning strategies, internet literacy, access to a reliable technology and a dedicated study space as well as gender and financial stress affect their test anxiety. To conclude, the decision to use online proctoring needs to consider the undesirable side‐effect on test anxiety. We discuss practical implications for university administrators, educational designers and teachers to reduce test anxiety.
... There had previously been no experience with OBEs within the course. Some evidence suggests students prepare less for OBEs (Agarwal & Roediger, 2011;Boniface, 1985;Moore & Jensen, 2007), though other studies found no difference in preparation time (Betts et al., 2009;Gharib et al., 2012) or study tactics (Broyles et al., 2005;Tamblyn et al., 2007). Importantly, these studies did not use data from summative, end-of-year exams and did not investigate the effects of free access to Internet resources. ...
Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom’s taxonomy (‘Remember’, versus ‘Understand/Apply’). We predicted higher performance on the OBE, driven by higher performance on ‘Remember’ questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms ‘Open-Book, ‘Bloom Category’ and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14–2.39), and higher on ‘Remember’ than ‘Understand/Apply’ questions (OR 1.13, 95% CI: 1.09–1.19). The difference in performance between ‘Remember’ and ‘Understand/Apply’ questions was greater in the OBE than the CBE (‘Open-Book’ * ‘Bloom Category’ interaction: OR 1.2, 95% CI: 1.19–1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge.
... For instance, Wiens, Beck et al. (2020) report on the validity and reliability of a video-based assessment design for teachers, and only briefly mention cost and time aspects in their discussion. In the same vein, Gharib, Phillips and Mathew (2012) compare an alternative examination format with existing ones and examine how the new format has impacted student performance. By approaching the subject of examinations in such a way that makes isolated aspects salient, examinations are considered to be neutral tools, and far-reaching questions about their place in the larger system are not part of the analysis. ...
This thesis is based on the investigation of video examinations in teacher education and serves as a case of an innovation process.
... One byproduct of take-home exams can be said to be reduced test anxiety for students. The literature has shown that open-book exams cause less exam-based anxiety than closed-book exams (Atılgan et al., 2009;Gharib et al., 2012). Considering the negative academic connotations of the pandemic on student learning, it is seen as important that take-home exams may also lower levels of exam anxiety for students (Özdin & Bayrak Özdin, 2020). ...
COVID-19 has changed the way we teach. Today, we have become far more experienced in the delivery of distance education and use of online tools. However, the quality of distance education and learning outcomes have become a matter of ongoing debate. Just as higher education aims to develop high-level skills in its students, researchers are seeking ways to perform valid and reliable assessment in distance education. Institutions and educators are also in search of assessment tools that can help prevent instances of cheating and plagiarism. However, performance-based assessment tools may also offer options to measure both high-level skills and in limiting cheating behaviors. In this study, we used the take-home exam as a formative remote assessment tool as a local case in Turkey. We surveyed the views of 43 undergraduate students about the quality of take-home exams as a remote assessment tool. The results showed that participants had a high quality perception about the use of take-home exam, especially with regards to being kept informed about evaluation and scoring, rapid assessment, the provision of feedback, and consistency of scope between assessment and course content. Whereas students highly perceived the use of take-home exams, they reported more moderate views regarding take-home exams increasing the level of interaction with their peers. The results of this study suggest that the use of take-home exams is significantly preferred by higher education students, that it is a reliable and distinctive way to measure students’ academic performance, and may increase student-teacher interaction through its formative use.
... Therefore, it is being concluded that mean marks scored by the students in an OBE is significantly higher than the mean marks scored in a CBE. This is in line with the results of studies conducted in the past, showing that students score higher in an OBE (Agarwal et al., 2008;Francis, 1982;Gharib et al., 2012;Krarup et al., 1974). However, many researchers have also provided empirical evidence that there are no significant differences in the performance of the students (Brightwell et al., 2004;Ioannidou, 1997;Pauker, 1974). ...
Coronavirus disease 19 (COVID-19) pandemic has shaken the higher education sector. Indian academic institutes are well acquainted with the traditional closed book examination; however, the pandemic has forced the institutes to resort to new methods to evaluate the students. Many academic units in India did not take the examination in the light of public health, but in the case of the University of Delhi, it decided to conduct an online open book examination (OBE). Therefore, in the present paper, we aim to evaluate the strength, weaknesses, opportunities, and challenges of conducting online OBE by referring to the vast literature available. We further extended our study by assessing the students’ performance in an OBE and closed book examination. The results unveiled that students tend to score higher marks in the case of an OBE set up compared to closed book examination.
... This is because they encourage deeper learning, as students can spend more time engaging with the material, rather than learning facts [62]. A slight amendment to this could be to use modified closed book examinations, with an adaptation that permits students taking in handwritten note cards or cheat sheets, again preventing the process of rote learning [63,64]. Evidence also suggests that open book exams can also be less stressful for students [65] and that performance is comparable with closed book exams, indicating that this would not be a case of lowering academic standards [66]. ...
This paper reflects on UK mathematics education following the poor performance in the Programme for International Student Assessment (PISA) metric, which compares reading, science, and mathematics across 27 countries. We compared a range of features within secondary school mathematics in the UK with the countries outperforming the UK. We note disparities in the depth of the curriculum and the use of high-stakes testing which could be disadvantaging UK students. We also reflect on key factors that may underpin teacher effectiveness in the UK, including teacher expectations, in part driven by early use of ability sets, a lack of teacher autonomy, and poor continuous professional development. On this basis, we make several recommendations to strengthen UK mathematics education.
... 1 While openbook examinations have existed for many years, 2,3 the use of such examinations as summative assessments in multiple courses in undergraduate pharmacy degrees, to the best of our knowledge, has not been assessed. Reported benefits of open-book examinations include a reduced anxiety level, decreased emphasis on rote memorization, 4 the encouragement of "a deeper engagement with the course material on the part of the students," 5 and "thinking at higher cognitive levels." 1 Moreover, an open-book examination may enhance student understanding in a more realistic context rather than just recalling facts 6 and more closely typifies the experience of real-life health practitioners who commonly refer to resources in order to make clinical decisions. 7 Considering the exponential increase in health and clinical knowledge, it is especially important that students are able to apply knowledge from reputable sources, such as medicines formularies. ...
Objective. To determine whether allowing final-year Bachelor of Pharmacy students to use a medicines formulary during examinations modified their learning behaviors and performance, and to investigate students' perceptions about having this resource available during examinations.Methods. Student performance and examination difficulty (as measured by classification of examination questions as high or low according to Bloom's taxonomy of learning) in second semester examinations (formulary allowed) was compared with first semester examinations (closed book) in successive years. Students completed a survey regarding their study and examination approaches and experiences after both semesters.Results. Examinations in semester two had more questions rated higher on Bloom's taxonomy than did examinations in semester one. Data were collected from student surveys for closed book examinations (response rate of 25% and 19% in 2015 and 2016, respectively) and open book examinations (response rate of 22% and 15% in 2015 and 2016, respectively). Students' study approaches, hours studied per week, and anxiety (all self-reported) did not differ between semesters one and two, but students in semester two spent more time studying with a formulary compared with students in semester one. Qualitative analysis of student comments revealed students preferred the formulary-allowed examinations over the closed-book examinations. The majority of students (68%) agreed with the statement: "Knowing that I will have access to the AMH [Australian Medicines Handbook] during the exams allowed me to pay more attention to higher level skills such as analysis and evaluation."Conclusion. When students were allowed to use a formulary for examinations, they studied more using their formulary prior to the examination. Students performed similarly on examinations with a greater proportion of questions addressing higher levels of Bloom's taxonomy and on closed-book examinations that were comparatively less cognitively challenging.
... Studies have shown that the testing effect is stronger in an open-book condition, especially for complex learning material(Agarwal et al., 2008;Roelle & Berthold, 2017). Open-book testing is nevertheless not a common technique in formal instruction, even if it reduces test anxiety when applied in exams (e.g.,Gharib et al., 2012). However, the self-paced study time of students might decrease because the students are expecting an open-book exam ...
We compared the long‐term effects of generating questions by learners with answering questions (i.e., testing), and restudying in the context of a university lecture. In contrast to previous studies, students were not prepared for the learning strategies, learning content was experimentally controlled, and effects on factual and transfer knowledge were examined. Students’ overall recall performance after one week profited from generating questions and testing but not from restudying. Analyzing the effects on both knowledge types separately, traditional analyses revealed that only factual knowledge appeared to benefit from testing. However, additional Bayesian analyses suggested that generating questions and testing similarly benefit factual and transfer knowledge compared to restudying. The generation of questions thus seems to be another powerful learning strategy, yielding similar effects as testing on long‐term retention of coherent learning content in educational contexts, and these effects emerge for factual and transfer knowledge. This article is protected by copyright. All rights reserved.
... Although these studies indicate the ineffectiveness of constructing crib sheets, most students believe that they are helpful and relieve stress (Dickson & Bauer, 2008). Another study of psychology students investigated the differences between open-book, cheat sheet, or closed-book exams and had a followup retention quiz and preference survey (Gharib, Phillips, & Mathew, 2012). The results indicated that students did slightly better if they had the additional resources and that they preferred open book and cheat sheets to closed book. ...
Many students want to use cheat sheets, or crib notes, on exams. Whether or not those aids actually help them has not been carefully studied. This paper measures 16 students’ notes by scoring the writing density as well as the number of definitions, examples, and mistakes. To consider the effectiveness of the notes, they are matched against exam solutions. This closer look at the content and application of the online students’ notes revealed that they were neither well- made nor well used.
... Less focus on isolated factual knowledge recall could have the benefit of lowering the time and energy students tend to spend on cramming less relevant facts and on being stressed out in overloaded curricula. It seems that open-book assessment might reduce student anxiety and stress in higher education (Gharib, Phillips, & Mathew, 2012;Theophilides & Dionysiou, 1996;Zoller & Ben-Chaim, 1989), and encourage deep learning (Baillie & Toohey, 1997;Eilertsen & Valdermo, 2000;Theophilides & Koutselini, 2000), although these conclusions have been contested (Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008). ...
Admission interviews in higher education may be developed with the intention to select
applicants with specific personal competences not captured by traditional gradebased
admission. In this study, we examined whether the data structure of multiplemini
admission interview scores supported the presence of communication, empathy,
collaboration, and resilience as independent test dimensions. In addition, the associations
between the interview scores and unintended test constructs (station format,
pre-university grades, age, gender) were examined. Confirmatory and exploratory factor
analyses and regression analyses were used to examine interview data from a cohort
of Danish medical school applicants. The proposed multi-dimensionality was not
supported by the data structure. The influence of the unintended constructs examined
was limited or non-existing. These results are in line with the scarce existing literature.
This situation makes a priori claims that the multiple-mini interview can measure multi-
dimensional personal competences inadvisable, and care should be taken about
what is communicated to stakeholders.
... This is called the testing effect, and also holds true for open-book tests. Studies where OBA was compared with CBA also demonstrated testing effects that were roughly similar (Pauker 1974;Agarwal et al. 2008;Gharib, Phillips and Mathew 2012). ...
... Less focus on isolated factual knowledge recall could have the benefit of lowering the time and energy students tend to spend on cramming less relevant facts and on being stressed out in overloaded curricula. It seems that open-book assessment might reduce student anxiety and stress in higher education (Gharib, Phillips, & Mathew, 2012;Theophilides & Dionysiou, 1996;Zoller & Ben-Chaim, 1989), and encourage deep learning (Baillie & Toohey, 1997;Eilertsen & Valdermo, 2000;Theophilides & Koutselini, 2000), although these conclusions have been contested (Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008). ...
Relatively little evidence about the validity threats in open-book multiple-choice tests exist. The aim of this study was to examine validity aspects relating to gener-alization, extrapolation and decision of a multiple-choice test of medical knowledge with aids (open-book and internet access). The theoretical framework was modern validity theory, and the study was designed as a ‘known groups com-parison’ study. Test performances of three known groups of test takers hypothe-sized to have different knowledge levels of the test content were compared, and analysis of pass/fail decisions was used to examine implications of decisions based on test scores. Results indicated that it was possible to discriminate be-tween expert and non-expert test taker groups even with the access to aids. In con-trast, an indefensible passing score was found to be the largest potential threat to test validity.
... Furthermore some researchers Block and Gharib et al. add that the test model with opened books universally reduces the anxiety of students in examinations as well as increasing pleasure in promoting student learning outcomes [6], [7]. More some also Green, Ferrante, and Heppard explained that the opened book model test makes students better prepare themselves for the exam and further increase the frequency of learning [8]. ...
... Weniger offensichtlich ist, dass, unabhängig von der Art der Prüfung, gute Studenten gute Ergebnisse erzielen und schlechte Studenten schlechte Ergebnisse erzielen. Das eigentlich interessante Ergebnis ihrer Studie ist allerdings, dass der Langzeitlerneffekt ebenfalls nicht von der Art der Prüfung abhängig war [AGM12]. In unserem Definitionsrahmen entsprechen Prüfungen mit Identitätsprüfung einer Open-Book-Prüfung. ...
Die Glaubwürdigkeit und Überprüfbarkeit der Zertifikate ist ein wesentlicher Bestandteil von jeglicher Form von zertifiziertem Training. Diese Aussage gilt natürlich auch für MOOCs. In diesem Kontext kommt allerdings erschwerend hinzu, dass eine individuelle, persönliche Beaufsichtigung der Prüfungen tausender TeilnehmerInnen offline nur schwer zu realisieren ist. Es wird daher eine Technik benötigt, diese Beaufsichtigung online durchzuführen, um die Vertrauenswürdigkeit oder Wertigkeit dieser Zertifikate zu erhöhen.
In dieser Studie vergleichen wir verschiedene Spielarten der Online-Aufsicht. Wir stellen die Ergebnisse einiger Umfragen unter unseren TeilnehmerInnen, die sich mit deren Sicht bezüglich der Wertigkeit der Zertifikate befassen, vor und bewerten die Aussagen in unserem Kontext. Schließlich stellen wir ein Experiment vor, das wir mit einer neuen Variante der Online-Aufsicht durchgeführt haben. Anstatt sich auf menschliche Augen zu verlassen, wird ein automatisierter Abgleich des Gesichts vor der Kamera mit einem hinterlegten Bild durchgeführt, um zu überprüfen ob die angemeldete TeilnehmerIn auch die ist, die die Prüfung ablegt.
... Students generally find large final exams stressful and prefer a more modular approach to assessment [35]. However, where an exam-based approach is used, open-book exams have been found to have lower levels of student anxiety as compared to closed-book exams and exams where students may bring in a cheat sheet [17]. ...
Constructive alignment is a student-centred approach to teaching and learning that aims to enhance student learning through a combination of constructivist learning theories and aligned curriculum. This paper presents two case studies where units have been developed to apply the principles of constructive alignment in the area of computer science and software engineering. It outlines the role of formative feedback and delayed summative assessment as a means of embedding constructivist learning theories in the application of constructive alignment. The discussion outlines some of the challenges and advantages gained from the greater focus on formative feedback during the teaching period, and presents some recommendations for others considering applying constructive alignment.
На підставі здійсненого автором аналізу наукового доробку встановлено, що дослідники основними проблемами онлайн оцінювання вважають високий рівень академічної недоброчесності здобувачів (списування та колективну роботу над завданнями), а також складність перевірки в дистанційному режимі досягнутих ними результатів навчання та сформованих компетентностей. На користь цих тверджень свідчить і проведене автором статті опитування науково-педагогічних працівників. Задля вирішення вказаних проблем у статті розглянуто підхід до забезпечення надійності процедур онлайн оцінювання здобувачів, який передбачає методичну та організаційну складові. У межах даного підходу автор пропонує на етапі розробки процедур оцінювання сконцентруватися на підборі тих типів питань, за рахунок яких вдасться унеможливити академічну недоброчесність здобувачів та різнобічно перевірити їх підготовку. У статті дана характеристика низькопорядкових «загальноприйнятих (conventional)» та «концептуальних (conceptual)» типів питань. Розглянуто особливості питань «на впізнавання», «на пригадування», а також «на інтеграцію та використання». Сформульовано рекомендації щодо використання різних типів питань під час онлайн оцінювання здобувачів відповідно до рівнів таксономії Блума. Зокрема для оцінювання найвищих рівнів запропоновано використовувати кейси та ситуаційні задачі. Проведене опитування показало високий рівень залучення викладачів до процедур оцінювання в умовах онлайн навчання, що зменшує їх переваги в порівнянні з очним контролем. Тому серед організаційних засад проведення онлайн контролю автор пропонує обирати інструменти для оцінювання, які дають змогу використовувати автоматизовану перевірку навіть для завдань з «високопорядковими» типами питань. Наголошено на потребі завчасного ознайомлення здобувачів з процедурою онлайн оцінювання, наданню переваги підходу «low-stakes assessment». Також у статті описано досвід автора з використання гугл-форм та платформи «Online Test Pad».
Die Nachfrage nach zukunftsgewandten, problemlösungsorientierten Prüfungsformaten für die Hochschullehre stieg nicht zuletzt aufgrund der COVID-19-Pandemie. Open-Book-Prüfungen, bei denen Unterlagen mitgebracht werden, werden seit Langem als alternatives Prüfungsformat vorgeschlagen. Der vorliegende Beitrag beleuchtet in einer Studie mit 113 Teilnehmenden, ob die Menge und Art mitgebrachter Unterlagen einen Einfluss auf die Performanz in Multiple-Choice-Fragen zu konzeptuellem und prozeduralem Wissen aufweisen. Einen signifikanten, positiven Zusammenhang in Bezug auf die Performanz in beiden Wissensformen wies die Menge an mitgebrachten kommentierten Vorlesungsfolien auf. Diese Befunde werden in Bezug zur ICAP-Theorie gesetzt, die zugrundeliegende Lernaktivitäten beleuchtet.
Trotz massiver Förderungen für die Digitalisierung ist die Präsenzlehre noch immer der Standard an deutschen Hochschulen. Aufgrund des Physical Distancing im Zuge der Corona-Pandemie musste sie jedoch kurzfristig fast vollständig digitalisiert werden. Die Beiträge des Bandes bieten einen multiperspektivischen Zugang zu den damit verbundenen Herausforderungen und beleuchten, wie die verschiedenen Akteur*innen die Umstellung auf digitale Lehr- und Lernformate umgesetzt und erlebt haben. Durch die Zusammenführung der verschiedenen Sichtweisen können die Bedarfe und Wünsche der einzelnen Akteursgruppen zusammengebracht und bei der nachhaltigen Weiterentwicklung der Hochschullehre besser berücksichtigt werden.
Covid-19 pandemic has affected every sphere of life specially the education sector observing a paradigm shift in the nature of pedagogy from offline face-to-face to online-virtual mode of learning. The biggest challenge in online-learning was the conduction of online examination for student's assessment specially in Indian context where digital divide is rampant. Thus, present study examines and compares the challenges faced by the students in two most widely accepted modes of examination by Indian universities and institutes of higher learning, that is, take home/unrestricted/ assignment-based examination (ABE) and highly time restricted/open-book examination (OBE). Primary data was collected through questionnaires prepared by using Google forms to measure adaptability, satisfaction, and challenges using 5-point Likert's scale. Cronbach's α test was performed on question items to check the reliability and internal consistency of the items. χ 2 test has been applied in order to check whether there is a statistically significant relationship between the gender and place of residence in the acceptability of ABE and OBE. The findings suggest that both modes of examination have their own challenges largely governed by the digital and economic divide. The acceptance level of ABE and OBE is not associated with gender. However, we found the level of acceptance association of ABE with the place of residence of the students but not with OBE.
The purpose of this two-year longitudinal study was to examine the performance of veterinary students on exams utilizing crib sheets to determine their effects on exam performance, perceived exam stress, and retention of subject matter knowledge. Scores for individual exams where crib sheets were permitted were compared with exam scores where crib sheets were not permitted utilizing one-sample t-tests. One-sample t-tests were also utilized to determine the influence of crib sheets on subject matter knowledge retention for a cumulative exam. A questionnaire was distributed to capture students’ perceptions about the value of crib sheets. Researchers found crib sheet use enhanced student exam performance, but did not improve retention of subject matter knowledge. Results from the questionnaire indicated students perceived crib sheet use as a way decrease exam anxiety and provide support during studying and testing. Disadvantages surfaced by students included that crib sheets could be used as a crutch and could decrease learning. It is recommended that instruction on strategies for use be implemented in any course using crib sheets. Overall, crib sheet use was perceived as positive by students and could be a viable way to combat high levels of anxiety and depression in veterinary students.
Generative tasks foster the integration of new idea units into coherent mental representations. However, they scarcely foster the consolidation of the new knowledge in memory. Hence, a promising means to optimize generative tasks is incorporating retrieval practice into them. A viable means to do so is implementing generative tasks in a closed-book style in which learners cannot access the learning material while working on the task. We investigated the effects of such closed-book generative tasks in comparison to open-book ones. Furthermore, we analyzed whether a closed/open switch style, in which learners could flexibly switch between the two styles, would be superior to both a pure closed- and a pure open-book style. We found that a closed-book style slightly hindered generative activities but not learning outcomes in comparison to an open-book style. The closed/open switch style yielded better learning outcomes than both a pure closed- and pure open-book style.
The study explored the application of closed-book, open-book and cheat-sheet exams in an undergraduate cost and management accounting course at a university in Ghana. 198 students participated in an exploratory study examining how the different exam types impact on exam scores, pre-exam anxiety and knowledge retention. The study improves on the methods used in prior studies by, simplifying the estimation of pre-exam anxiety, and controlling for general student aptitude (GPA) in the specification of model. ANOVA was used to analyse the data. The results of the study provide compelling support for the application of cheat-sheet exams even though the perceptual views of students regarding this exam type were quite low on several variables and parameters.
Sonography programs continuously explore and evaluate methods to improve quality. The purpose of this mixed-methods study was to investigate the use of open-book examinations (OBEs) in an accredited sonography program. Participants included sonography students enrolled in an academic health sciences center who used an OBE format in two ultrasound physics courses. OBE mean examination scores were compared to students who used a closed-book examination (CBE) format. No statistically significant difference was seen in mean examination scores between the OBE and CBE cohorts with the exception of Spring Examination 1 (P =.0026). All OBE students (n = 17) participated in focus groups. Qualitative analysis showed examination preparation and text usage changed over time. Seventy percent of OBE students reported reduced examination tension and stress. OBEs do not have a significant effect on examination grades, and examination anxiety and stress are reported to be reduced. The OBE cohort noted a preference of OBEs over CBEs.
Allowing the use of student-prepared notecards has been suggested as a way to increase student performance, with the literature showing mixed results. By allowing students to utilize a notecard on some, but not all, tests in principles of macroeconomics, we examine the effect notecards have on student performance. Our analysis utilizes a novel method of controlling for student aptitude and instructor biases in each sample. The results show improvements in performance both as a whole and by level of question based on Bloom’s taxonomy and question difficulty levels. In addition, we find notecard usage by students leads them to become more optimistic and realistic about their test performance, as well as leading students to study more efficiently.
In this article, we propose a student-centred approach to teaching quantum chemistry and spectroscopy at the bachelor-level that extends active learning principles outside course lectures. We seek to elucidate what type of methodology is most appropriate and efficient for this context and student population, and how this incorporation of active learning elements impacts learning. Three quantitative learning indicators are used to measure the effectiveness of the proposed approach, including exercise points obtained by the students, exam results, and the results of a conceptual inventory administered both at the beginning and the end of the course. The proposed model resulted in substantial improvement in learning outcomes compared to a previous class where active learning elements were confined mostly to the course lectures and a traditionally taught class. The model can be generalised to any subject where both quantitative and qualitative understanding is required. Thus, in addition to providing yet another data point supporting the effectiveness of active learning approaches in science, this study shows the benefits of applying these approaches to exercises and other course tasks besides lectures.
Introduction:
A core principle of family medicine is information mastery, or application of principles of evidence-based medicine in clinical practice. While information mastery teaching and assessment are beginning to permeate postgraduate family medicine training programs, and while exciting literature on new open resource assessment methods is emerging, there are no prior descriptions of examinations that specifically assess medical students' information mastery competency.
Methods:
To test information mastery competency, a novel final exam for the family medicine clerkship was developed, implemented, and evaluated. During the timed exam, the competency-based information mastery assessment (IMA) requires students to look up evidence-based information using web resources to answer case-based questions. Exam feasibility was tested with pilot examinees whose reactions were gauged. Student performance on the traditional closed book knowledge assessment (KA) was compared with performance on the open internet IMA. Exam performance was compared with preceptor ratings of students' clinical performance. Low performers were further analyzed for preceptors' ratings of specific student skills in information mastery and self-directed learning.
Results:
An open internet IMA testing knowledge application and information mastery skills is not only feasible but can also be educational. Student performance scores on the open internet IMA do not differ from scores on the closed book KA. Students describe many positive features of this open internet IMA. Student performance on the competency-based IMA correlates with clinical ratings by preceptors and with preceptors' judgment of information mastery skills.
Conclusions:
A novel approach to assessment in family medicine clerkships may be used to assess student competency in information mastery. Further research is needed for enhanced exam validation.
Penelitian ini merupakan studi eksploratori, bertujuan untuk mengkaji persepsi mahasiswa terhadap sistem ujian akhir 'buka buku', dan kemungkinan memberikan kebebasan kepada mahasiswa mengakses informasi online selama ujian. Responden penelitian sebanyak 92 mahasiswa magister, berusia 22 – 60 tahun, 67% berusia antara 28 sampai dengan 50 tahun. Instrumen berupa kuesioner yang terdiri atas 14 pertanyaan tertutup tentang sistem ujian 'buka buku' menggunakan skala Likert 1-5, dan pertanyaan terbuka tentang kebebasan menggunakan internet dalam ujian'buka buku.' Hasil penelitian menunjukkan bahwa pada umumnya (86.8%) mahasiswa mempunyai persepsi yang positif terhadap sistem ini dan sebagian besar (77%) lebih menyukai sistem ujian 'buka buku' daripada 'tutup buku'. Sistem ini dinilai lebih sesuai untuk jenjang pendidikan pascasarjana yang sejalan dengan prinsip pendidikan yang baik karena tidak mengandalkan hafalan, mengurangi kecemasan mahasiswa dalam ujian, dan membuat mahasiswa lebih serius mempersiapkan diri. Penggunaan internet secara bebas dalam ujian sistem'buka buku' dinilai baik, karena mahasiswa dapat melengkapi jawabannya dengan analisis materi internet yang diakses. Penelitian ini menyimpulkan bahwa mahasiswa mempunyai persepsi yang positif terhadap sistem ujian ‘buka buku’ dan penggunaan internet dalam sistem ini, baik karena pertimbangan akademik maupun psikologis. Oleh sebab itu, sistem ujian ini dapat lebih luas digunakan pada berbagai jenjang pendidikan sebagai alternatif sistem ujian. Pada saat yang sama, perlu dilakukan kajian lebih lanjut mengenai pengaruh sistem ini pada
mahasiswa, serta berbagai strategi untuk menjadikan sistem ujian ini efektif.
ResearchGate has not been able to resolve any references for this publication.