No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Establishing a culture of assessment
... Campus leaders explore their role in determining "whether a meaningful assessment culture thrives" (Haviland, 2009). There is no shortage of literature aimed at administrators, faculty, and institutional effectiveness personnel pitching various ways to create a "culture of assessment" that supports accreditation efforts and learning outcomes attainment (see for example Angelo, 2002;Berlanger, 2006;Hill, 2004;Lakos, 2004;Moltz, 2009;Piascik & Bird, 2008;Rothwell & Khera, 2009;Weiner, 2009). ...
... Six factors emerge as salient to the current analysis. One theme that emerges in the culture of quality literature surrounding accreditation in higher education is the necessity for (1.) teamwork and buy-in for accreditation to reach its full organizational potential (Brittingham, 2009;Gose, 2002;Oden, 2009;Piascik & Bird, 2008;Rodrigues, 2002;Weiner, 2009). Shera's (2008) case study of how one department at the University of Toronto successfully created a culture of research found more than just effective leadership was needed; it takes a team: "Creating a coalition for change requires consultation and collaboration with faculty members, students, and community partners" (p. ...
... Cohort two was randomly selected using nth sampling from schools whose reaffirmation visit would take place in five years (2013). Accreditation experts have argued that "to determine whether an assessment culture exists" researchers must "look at the attitudes and behaviors of individuals" within institutions (Weiner, 2009). Accordingly, student workers collected email addresses of SACS steering committee members along with randomly selected email addresses of faculty from universally occurring departments (mathematics, English, history, biology, and business departments), staff in human resources and student affairs, and administrators within each selected college or university. ...
There is an inherent tension in the U.S. system of accreditation. Historically, the system has been oneof self-regulation (Brittingham, 2009). As access to higher education has grown, however, and theconcomitant flow of federal money to colleges and universities has increased, the federal governmentand the taxpayers it represents have called for more and more external reporting of measures ofcollege quality. Critics of the current system would like more external oversight to create what they havetermed variously a "culture of quality" or a "culture of evidence" (Bardo, 2009; Crow, 2009; Kelderman,2009; Understanding, 2001). The most dissatisfied would like to remove regional accrediting approvalas the imprimatur that authorizes federal funds; those critics would delegate the power to authorizespending public funds to some branch of the federal government (Graca, 2009). Defenders of thecurrent system point to the power of self-regulation to establish an ongoing culture of improvement incolleges and universities more effectively than external regulation can achieve (Kelderman, 2009;Oden, 2009). For the purposes of this study, it is important to note that both critics and defenderspredicate their arguments for being the better path to achieving educational quality on the belief that itwill take transformed organizational cultures in higher education to sustain any real overhaul ofeducational outcome attainment. This article considers both sides of the accreditation debate and usesGlaser, Zamanou, and Hacker’s (1987) Organizational Culture Survey (OCS) to create a unique dataset to explore the question: to what extent does participating in regional accreditation affectperceptions of organizational culture for members of those cultures?
... Conversely, a culture of assessment is one in which an organization's "values, beliefs, norms, and behaviors reflect a shared appreciation of assessment practice and its value to institutional advancement" (Barham et al., 2013, p. 73). A culture of assessment may be comprised of 15 elements, as described by Weiner (2009), including general education goals, common assessment language, systematic assessment, sustainable assessment plans, and information sharing. When assessment leaders describe establishing cultures of assessment at their institutions, they detail the importance of understanding underlying beliefs about assessment, considering the language individuals use to describe assessment at the institution, reflecting on assessment practices, gathering accounts of the subcultures within the institution and promoting communication across subcultures, and providing a framework for innovation in assessment . ...
... Another strategy for incorporating ritual into assessment work is formally celebrating the efforts of faculty and administration. Weiner (2009) includes celebration of successes as one of the 15 elements that demonstrates a successful climate for assessment. Celebration of assessment work should reflect the culture of an institution and can begin in small ways such as thanking faculty for completing assessment reports on time and providing them with meaningful feedback on reports. ...
... Applying strategies from Kezar's (2014) cultural change meta-framework can provide assessment leaders with tools to improve their efficacy. When faced with cultures of fear or compliance surrounding assessment, assessment leaders can use strategies outlined in this article to support movement toward a culture of assessment (Weiner, 2009). ...
Increased focus on accountability in higher education has resulted in greater demand for assessment leaders to gather, analyze, and use student learning data. As college and university campuses use student learning data for improvement, assessment professionals are called upon to lead change that challenges underlying beliefs and assumptions about assessment, its purposes, and its uses. Cultural change theories can support assessment professionals by providing strategies for leading change. This article examines how strategies from cultural change theories can be applied by assessment leaders to create cultural change, shifting campus cultures surrounding assessment from that of fear and compliance to celebration.
... These developments have led to an increased focus on institutional conditions that support or hinder the implementation of evidence-based practices related to assessing student learning and institutional effectiveness, what assessment scholars and practitioners have commonly referred to as cultures of assessment. Scholars have widely advocated for the development and maintenance of these cultures to improve student learning (Banta & Palomba, 2014;Bresciani, Gardner, & Hickmott, 2010;Douchy, Segers, Gijbels, & Struyven, 2007;Driscoll, de Noriego, & Ramaley, 2006;Maki, 2010;Suskie, 2009;Weiner, 2009). ...
... Scholarship on cultures of assessment can be broadly separated into two categories: (a) institutional practices suggestive of a culture of assessment and (b) conjectural elements hypothesized as fundamental to a culture of assessment. In regards to institutional practices, Weiner (2009) offered a clear definition built upon to fifteen specific institutional functions indicative of a culture of assessment: (a) general education goals, (b) common use of assessment terms, (c) faculty ownership, (d) ongoing professional development, (e) administrative support and understanding, (f) practical, sustainable assessment plan, (g) systematic assessment, (h) student learning outcomes, (i) comprehensive program review, (j) assessment of co-curricular activities, (k) institutional effectiveness, (l) information sharing, (m) planning and budgeting, (n) celebration of success, and (o) new initiatives. Similarly, Douchy et al. (2007, pp. ...
... These definitions have not taken into account that although such traits might be in place at an institution, practices in a culture of assessment might augment the reasons these traits are present. Stated differently, one institution can be engaging in all 15 of the traits outlined by Weiner (2009) with the primary intent of improving student learning, while another institution also could employ these same traits but might be doing so primarily to satisfy accreditation mandates. Although both institutions would meet Weiner's definition of a culture of assessment, the influences on student learning improvement processes could be drastically different. ...
Although touted as beneficial to student learning, cultures of assessment have not been examined adequately using validated instruments. Using data collected from a stratified, random sample (N = 370) of U.S. institutional research and assessment directors, the models tested in this study provide empirical support for the value of using the Administrators’ Survey of Assessment Cultures as an assessment tool. The resulting first order model provided good fit statistics and included five factors: a) Faculty Perceptions, b) Use of Data, c) Sharing, d) Compliance or Fear Motivators, and e) Normative Purposes for Assessment. Internal consistency estimates were also good (Cronbach’s α = .792 to .922). Use of the Schmid-Leiman solution afforded further support for the higher order factor, Culture of Assessment. Predictive discriminant analysis correctly classified the majority of the respondents (80.7%) as belonging to a student learning or accreditation group based on the linear classification scores. Implications for assessment leadership, practice, and future research are offered.
... These developments have led to an increased focus on institutional conditions that support or hinder the implementation of evidence-based practices related to assessing student learning and institutional effectiveness, what assessment scholars and practitioners have commonly referred to as cultures of assessment. Scholars have widely advocated for the development and maintenance of these cultures to improve student learning (Banta & Palomba, 2014;Bresciani, Gardner, & Hickmott, 2010;Douchy, Segers, Gijbels, & Struyven, 2007;Driscoll, de Noriego, & Ramaley, 2006;Maki, 2010;Suskie, 2009;Weiner, 2009). ...
... Scholarship on cultures of assessment can be broadly separated into two categories: (a) institutional practices suggestive of a culture of assessment and (b) conjectural elements hypothesized as fundamental to a culture of assessment. In regards to institutional practices, Weiner (2009) offered a clear definition built upon to fifteen specific institutional functions indicative of a culture of assessment: (a) general education goals, (b) common use of assessment terms, (c) faculty ownership, (d) ongoing professional development, (e) administrative support and understanding, (f) practical, sustainable assessment plan, (g) systematic assessment, (h) student learning outcomes, (i) comprehensive program review, (j) assessment of co-curricular activities, (k) institutional effectiveness, (l) information sharing, (m) planning and budgeting, (n) celebration of success, and (o) new initiatives. Similarly, Douchy et al. (2007, pp. ...
... These definitions have not taken into account that although such traits might be in place at an institution, practices in a culture of assessment might augment the reasons these traits are present. Stated differently, one institution can be engaging in all 15 of the traits outlined by Weiner (2009) with the primary intent of improving student learning, while another institution also could employ these same traits but might be doing so primarily to satisfy accreditation mandates. Although both institutions would meet Weiner's definition of a culture of assessment, the influences on student learning improvement processes could be drastically different. ...
Although touted as beneficial to student learning, cultures of assessment have not been examined adequately using validated instruments. Using data collected from a stratified, random sample (N = 370) of U.S. institutional research and assessment directors, the models tested in this study provide empirical support for the value of using the Administrators’ Survey of Assessment Cultures as an assessment tool. The resulting first order model provided good fit statistics and included five factors: a) Faculty Perceptions, b) Use of Data, c) Sharing, d) Compliance or Fear Motivators, and e) Normative Purposes for Assessment. Internal consistency estimates were also good (Cronbach’s α = .792 to .922). Use of the Schmid-Leiman solution afforded further support for the higher order factor, Culture of Assessment. Predictive discriminant analysis correctly classified the majority of the respondents (80.7%) as belonging to a student learning or accreditation group based on the linear classification scores. Implications for assessment leadership, practice, and future research are offered.
... Yet, they are pivotal to the establishment and success of these processes aimed at enhancing student learning while also meeting internal and external accountability requirements. Weiner (2009) underscored the necessity of engaging faculty in all stages of assessment, asserting that without faculty ownership, the process is likely to fail. Ebersole (2009) advocated for a culture of assessment by actively involving faculty in setting outcome objectives, utilizing data for decision-making, and communicating results. ...
... Research demonstrates the vital role of faculty in both the assessment of student achievement of institutional learning outcomes and the use of assessment data to inform curricula and program development as well as improvements in teaching and learning practices (Ebersole, 2009;Lindauer & Coward, 2021;Weiner, 2009). SAIL 3.0 underscored the vital role faculty play in ensuring the assessment of student achievement of learning outcomes is accurate, contextualized, meaningful, and actionable. ...
The primary aim of institutional learning outcomes assessment is the creation of a culture of assessment where faculty use evidence-based data to validate and improve teaching and learning for the benefit of students. Faculty are key to these processes and yet, they are often woefully disengaged from them. This paper presents findings from an action research project that utilized a collaborative self-study approach to engage faculty in the strategic assessment of institutional learning (SAIL). SAIL is an immersive professional development opportunity that bridged quality assurance with meaningful improvements in the classroom. Findings indicated that cross-disciplinary dialogue about assessment increased faculty awareness of the (mis)alignment between course, program, and institutional learning aims while also identifying and informing potential gaps in curriculum and program design. SAIL is an excellent mechanism to engage faculty in an immersive assessment of student achievement that may then lead to meaningful improvement in teaching and learning.
... Además, implica el trabajo colaborativo de los actores educativos en todos procesos sistemáticos que estén encaminados a mejorar la calidad académica estudiantil. Weiner (2009) Son las actitudes y conductas predominantes que caracterizan el funcionamiento del apoyo institucional hacia la evaluación de los resultados educativos de los estudiantes. ...
... En cuanto a lo primero, si bien la literatura ofrece diversas definiciones conceptuales sobre cultura de la evaluación (como Barham et al., 2013;Bolseguí & Fuguet, 2006;Corti et al., 2011;Valenzuela et al., 2011;Weiner, 2009), están orientadas en términos institucionales, es decir, a nivel macro. Por tanto, era necesario desarrollar una conceptualización que integrara elementos que pudieran describir la cultura de la evaluación del propio docente (más allá del marco institucional); pues la cultura, de acuerdo con Geertz (1992) se configura también con los significados de cada individuo. ...
... Additionally, successful institutions have the support of administration for the commitment of required resources. 4,[9][10][11] In order to start building a culture of assessment, an institution must start with good organizational and governance-related structures that will support assessment activities. Auburn College of Pharmacy described the development of their Office of Teaching, Learning, and Assessment (OTLA) in 2007. ...
... Assessment results can illuminate areas of need and may be helpful to demonstrate that investments (e.g., in classroom advancements) have led to better outcomes (e.g., in student learning/ performance). 11 The importance of leadership in creating a successful assessment environment as well as a culture change has been previously described. 8 While leadership has been cited as an important factor, establishing a culture of assessment also depends on the maturation of the process. ...
Introduction:
Pharmacy education has standards for assessment that must be met, prompting many programs to develop cultures of assessment. This study assesses the extent to which assessment personnel perceive that a culture of assessment has been established within their schools/colleges of pharmacy (S/COPs) and determines whether a relationship exists between assessment structure and the perceived strength of the culture of assessment.
Methods:
An electronic survey was administered to assessment personnel at 134 S/COPs. Respondents were asked about changes within the past five years pertaining to assessment personnel/committees and factors prompting these changes. They were also asked to rate items related to support for institutional assessment (SIA scale) and faculty engagement with assessment (FEA scale). The reliability of each scale was estimated using Cronbach alpha and bivariate correlations were conducted to estimate the relationships among the two scales and other survey items.
Results:
The S/COP response rate was 84% (113/134). Most (58%) noted the formation or re-structuring of the assessment office, with 26% creating one or more new assessment positions. The most common source of change was new accreditation standards (55%) followed by a new leadership position created in assessment (40%). Cronbach alpha for the SIA scale was 0.79 and 0.84 for the FEA scale. Program size was strongly negatively correlated with school age (rs = -0.58) and the FEA scale (r = -0.21).
Conclusions:
There was a strong relationship between SIA and FEA. Future studies are warranted to determine the interdependency of support for assessment and faculty engagement with assessment.
... Assessment structure as well as leadership responsible for assessment are included in the literature as key elements for successful assessment in higher education. [16][17][18][19][20] Furthermore, a commitment to the provision of resources is vital for well-functioning assessment programs. Examples of resources include personnel with expertise in assessment, use of assessment tools, and general resources for assessment planning and budgets. ...
... Examples of resources include personnel with expertise in assessment, use of assessment tools, and general resources for assessment planning and budgets. [16][17][18][20][21][22] Our findings suggest that S/COPs have allocated considerable resources toward support of assessment efforts, as evidenced by the number and types of assessment positions, committees, and software. The vast majority of schools (96%) had at least one to two faculty or staff positions responsible for assessment, and those positions were rarely shared with other programs outside the school/ college. ...
Introduction:
Expectations for assessment in higher education have increased in recent decades, prompting institutions to invest additional resources in this area. This study aimed to determine the resources, structure, and perception of assessment resources in United States schools and colleges of pharmacy (S/COPs).
Methods:
Assessment personnel in S/COPs were surveyed electronically. Information collected included S/COP demographics, composition of assessment positions, experience and training of assessment personnel, and structure and responsibilities of committees engaged in assessment. Respondents' perception of their S/COPs having sufficient assessment personnel, recent changes in assessment, and the factors that prompted assessment changes were also surveyed.
Results:
Respondents included individuals from 113 S/COPs (84% response rate). Most S/COPs had 1-2 assessment positions and 1-2 assessment-related committees. The most common assessment position titles were assistant/associate dean, director, coordinator/specialist, and administrative assistant. Dean-level administrators typically had worked in assessment the longest, whereas directors were more likely to have formal assessment training. Most respondents (75%) agreed they had sufficient assessment personnel to meet the 2007 Accreditation Council for Pharmacy Education Standards. Nearly two-thirds of respondents agreed they had sufficient personnel to meet the 2016 Standards and support their current assessment plan/process. Most S/COPs had a formal assessment committee (93%) and an average of two committees overseeing assessment.
Conclusion:
Most S/COPs reported having sufficient resources to support assessment activities. Although there were some consistent themes, there does not appear to be a single model for structuring assessment resources or committees. Effectiveness of various assessment structures represents an area for future research.
... The modern definition of organizational culture includes the following elements: organizational values, dominant leadership styles, languages and symbols, practices and routines, and definitions of success (Quinn & Cameron, 2011;Hofstede Insights, n.d.). According to Weiner (2009), analyzing people's attitudes and behaviors is one way to understand the assessment culture in a given institution. The same is true according to Cameron and Quinn (2011, p. 22) who stated that "an organization's culture is reflected by what is valued, the dominant leadership styles, the language and symbols, the procedures and routines, and the definitions of success that make an organization unique." ...
... Wehlburg (2008), for instance, proposes an integrated and transformative assessment model where faculty are embedded in the process and empowered to own their program assessment rather than asked to respond to external mandates that amount to checking the boxes. Other research suggests doing so involves creating a culture of assessment (e.g., Farkas, 2013;Ndoye & Parker, 2010;Weiner, 2009). Central to doing so is to build assessment programs and processes from the ground up so to speak. ...
Learning outcome assessment is a fairly recent trend in higher education that began in the 1980s (Lubinescu et al., 2001). Today, many faculty perceive assessment reporting to be tedious, time-consuming, and irrelevant busywork (Wang & Hurley, 2012). Unfortunately, this systematic process created to use empirical evidence to measure, document, and improve student learning has in many cases lost sight of this central goal. As a result, faculty may be justified in their opinions about it. This essay proposes a framework for addressing this thorny issue via WISER. WISER is an acronym for five content pillars of the communication discipline faculty can use to ensure their assessment efforts achieve the goal of not only documenting but also improving student learning. WISER stands for writing, immersive experiences, speaking, ethical communication, and research as programmatic assessment categories. These WISER categories extend the National Communication Association (NCA)-endorsed domains of communication learning in ways that make them functional for assessment.
... Assessment culture is 'the deeply embedded values and beliefs collectively held by members of an institution who influence assessment practices on their campuses' (Holzweiss, Bustamante, and Fuller 2016, 20). It is said that assessment cultures are connected to quality (Kalu and Dyjur 2018;Ylonen, Gillespie, and Green 2018;Cardoso et al. 2019), continuous improvement (Goncalves et al. 2018;Stanny 2018) and accountability (Weiner 2009). The empowerment of stakeholders and shared purpose have been common recommendations for developing a quality-oriented assessment culture and navigating the tension between assessment for learning and assessment for accountability (Seagraves and Dean 2010;Eastberg 2011;Meyer-Beining 2020). ...
A series of worldwide projects concerning assessment, student outcomes and quality in higher education has revealed the need for a change in how higher education institutions assess student outcomes; however, many academics remain unconvinced. The success of assessment change arguably depends on the assessment culture within the institution. This qualitative investigation of assessment cultures draws on the perspectives of 35 academics from Australia, Canada and Sweden. Data were analysed through a socio-cultural lens, with results supporting assessment cultures related to institutional structures and collegial relationships. The results also suggest the existence of assessment microcultures embedded in disciplines. This study provides concurrent validity to previous studies of assessment cultures, evidenced from institutional leadership perspectives, through the analysis of academic practitioners’ viewpoints. Synthesis of findings observed that a combination of agency for change and policy levers effectively stimulated change. Included are suggestions to address the perceived barriers of entrenched disciplinary practices, institutional systems and logistical constraints. There is limited empirical research on the impact of assessment culture on assessment practices; this study addresses this shortcoming and provides a new aspect to add to the literature on assessment cultures in higher education.
... The reference section points to just a few of the more recent and relevant works relating to this effort. 3,4,5,6,7,8,9 The fundamental problem has been identified and summarized as follows: ...
... Opportunities to celebrate success constitute an important part of the assessment process and should not be ignored (Weiner, 2009). Too often, assessment efforts are seen as means to focus on areas requiring improvement such that accomplishments are not properly recognized. ...
Creating advising curricula through backward design ensures that learning objectives remain central to the process and enables those in advising units to design comprehensive assessment plans for continued curricular improvement. By incorporating measures to observe student learning directly, advisors can evaluate their curriculum objectively to ensure students achieve desired learning outcomes. An advising unit created a proactive advising curriculum for academically at-risk students through backward design that includes multiple assessment measures. Students in four categories of academic risk were targeted for intervention. Through the evaluation of direct-learning evidence gathered through assessment, the advising unit improved the advising curriculum, showing the process for intentional curriculum design and assessment to improve student learning.
... For the purpose of this paper, assessment culture is defined as the attitudes, behaviors, and beliefs of individuals within an institution that support "the assessment of student learning outcomes" (Weiner 2009). In our system, faculty are both clinical preceptors and AAs, and the AAs are preceptors for both their own advisees and other residents. ...
Purpose: Adopting CBME is challenging in medicine. It mandates a change in processes and approach, ultimately a change in institutional culture with stakeholders ideally embracing and valuing the new processes. Adopting the transformational change model, this study describes the shift in assessment culture by Academic Advisors (AAs) and preceptors over three years of CBME implementation in one Department of Family Medicine.
Methods: A qualitative grounded theory method was used for this two-part study. Interviews were conducted with 12 AAs in 2013 and nine AAs in 2016 using similar interview questions. Data were analyzed through a constant comparative method.
Results: Three overarching themes emerged from the data: (1) specific identified shifts in assessment culture, (2) factors supporting the shifts in culture, and (3) outcomes related to the culture shift.
Conclusions: In both parts of the study, participants noted that assessment took more time and effort. In Part 2, however, the effort was mitigated by a sense of value for all stakeholders. With support from the mandate of regulatory bodies, local leadership, department, faculty development and an electronic platform, a cultural transformation occurred in assessment that enhanced learning and teaching, use of embedded standards for performance decisions, and tracking and documentation performance.
... Institutionalizing a diversity plan, however, cannot occur without a culture of assessment. Weiner (2009) identified several components of a culture of assessment in higher education: (a) core competencies and learning outcomes that all students are expected to demonstrate; (b) faculty, staff, and administrators engaged in planning, developing, and implementing assessment; (c) institutional leadership reviewing and using assessment results to inform budgeting and strategic planning; (d) campus climate assessment to explore institutional progress toward its mission and goals; and (e) cross-disciplinary forums to share assessment results and challenges, and celebrate success. Even more critical is the measurement and assessment of campus climate research and using the results to effect change (Worthington, 2008). ...
Organizational change around diversity and inclusion does not occur rapidly in higher education. As institutions strive to deliver on their espoused goals, values, and mission to reflect the social and cultural demographics representative of the state, society, and world, many argue the importance of strategic diversity plans. While recruitment and retention initiatives for faculty, staff, and students remain a priority for institutions, it is becoming increasingly evident, particularly for the public good and sustainability, that diversity plans evidence progress and success. The chief diversity officer often leads these organizational change efforts. This paper is a case study of a public research-intensive university’s successes and challenges with institutionalization of a diversity plan, including implications for effecting change in the campus organizational culture.
... The utility of review teams for ensuring the quality of examination items to increase reliability and validity evidence is well documented (Abozaid, Park, & Tekian, 2017;Downing & Haladyna, 1997;Herndon, 2006;Weiner, 2005), but review panels typically include subject matter experts who evaluated item content alongside item structure. The human resources and infrastructure necessary to complete a comprehensive review of items are an important consideration in undertaking such a process. ...
The sentiment that there is more work to be done than there is time is pervasive among faculty members at most academic institutions. At health science centers, faculty members often balancing teaching responsibilities, clinical loads, and research endeavors. Creative use of educational support staff may provide institutions an avenue for accomplishing goals related to quality improvement, curriculum revision, and accreditation tasks. One such task is the maintenance of a bank of multiple-choice examination items that are free of structural flaws. This study measured the effects of a systematic approach to revising structural flaws in multiple-choice questions on the psychometric properties of the items. Structural flaws were identified by educational support staff instead of the faculty experts who authored the items and were responsible for teaching the content knowledge the items were intended to assess. Two-way ANOVA was used to measure the outcome of the revision project and structural flaw type on the psychometric qualities of existing conventional multiple-choice examination items. Neither variable had a statistically significant effect on the psychometric qualities of the items. Nonetheless, efforts to remove structural flaws from multiple-choice items may lead to stronger reliability estimates, enhanced validity evidence, and an improved test-taking experience for students. Adviser: Allen Steckelberg
... But if the empirical documentation of "two cultures" rivaling C. P. Snow's (1959) famed treatise on epistemological bifurcation in the Academy occasions no great surprise when attitudes toward assessment are examined, does this automatically portend "bad news" for the dueling narratives and for the inclusive community of higher education of which they are a part? Are we forced to follow the implication of Wendy Weiner's (2009) judgment that the creation of plentiful and productive cultures of assessment exists only in the form of pipe dreams absent adequate faculty buy-in? On the one hand, such a reading seems inescapable if the foregoing findings are shared with a breadth commensurate to their depth. ...
... For many years researchers and practitioners have promoted the benefits of cultivating and sustaining institutional cultures of assessment as essential to ensuring optimum student learning in US colleges and universities (Burke 2004;Ewell 2004;Ndoye and Parker 2010;Weiner 2009). Other scholars have highlighted barriers to establishing such a culture. ...
In this study, results are presented from a rigorous content analysis of responses to two open-ended questions included in the Administrators’ Survey of Assessment Culture. A sample of 302 US higher education administrators provided 566 narrative responses addressing (1) the primary reason they conducted
assessment on campus, and (2) how they would characterize their campus assessment cultures. Analysis revealed two meta-themes: “Institutional Structures,” including procedures, data usage, and accountability; and “Organizational Culture,” administrators’ descriptions of rituals, artifacts, discourse, values, and change related to assessment. Implications are shared for reframing and cultivating notions of institutional cultures of assessment.
... (for example : Whittle, Pell et. al., 2010;Anonymous, 2011) Closing the Loop --Part I Weiner (2009) emphasizes that the success of an assessment program depends on having a faculty-led team who plans the program, develops tools for and implements it, and uses the data obtained. Accordingly, at CSU Stanislaus, undergraduate assessment efforts are coordinated by the Undergraduate Assessment Committee, consisting of three members of the business faculty. ...
Assurance of student learning through effective assessment has become increasingly important over the past decade as accrediting agencies now require documented efforts to measure and improve student performance. This paper presents the methodology used by the College of Business Administration at California State University, Stanislaus to assess its students quantitative reasoning skills and the resulting actions taken to improve student performance. The critical role that faculty play in developing meaningful assessment instruments and evaluating and acting on the results is emphasized.
... But if the empirical documentation of "two cultures" rivaling C. P. Snow's (1959) famed treatise on epistemological bifurcation in the Academy occasions no great surprise when attitudes toward assessment are examined, does this automatically portend "bad news" for the dueling narratives and for the inclusive community of higher education of which they are a part? Are we forced to follow the implication of Wendy Weiner's (2009) judgment that the creation of plentiful and productive cultures of assessment exists only in the form of pipe dreams absent adequate faculty buy-in? On the one hand, such a reading seems inescapable if the foregoing findings are shared with a breadth commensurate to their depth. ...
The “Assessment Movement” in higher education has generated some of the most wide-ranging and heated discussions that the academy has experienced in a while. On the one hand, accrediting agencies, prospective and current clientele, and the public-at-large have a clear vested interest in ensuring that colleges and universities actually deliver on the student learning outcomes that they promise. Anything less would be tantamount to a failure of institutional accountability if not outright fraud. On the other hand, it is no secret that efforts to foster a “culture of assessment” among institutions of higher learning have frequently encountered resistance, particularly on the part of faculty unconvinced that the aspirations of the assessment movement are in fact achievable. One consequence of this tension is the emergence of an embryonic literature devoted to the study of processes that monitor, enhance, or deter the cultivation of a “culture of assessment” with sufficient buy-in among all institutional stakeholders, faculty included. Despite employment of a wide-ranging host of research methods in this literature, a significant number of large unresolved issues remain, making it difficult to determine just how close to a consensual, culture of assessment we have actually come. Because one critical lesson of extant research in this area is that “metrics matter,” we approach the subjective controversy over outcomes assessment through an application of Q methodology. Accordingly, we comb the vast “concourse” on assessment that has emerged among stakeholders recently to generate a 50 item Q sample representative of the diverse subjectivity at issue. Forty faculty and administrators from several different institutions completed the Q-sort which resulted in two strong factors: the Anti-Assessment Stalwarts and the Defenders of the Faith. Suggestions are offered regarding strategies for reconciling these “dueling narratives” on outcomes assessment.
... The culture of assessment in higher education has usually meant the increased use of measures to document the direct effect of, for example, a learning intervention in a given course or to show how a particular configuration of skills coalesce at the program level to create a well-rounded undergraduate student. The broad shift toward greater accountability through the increased use of assessment tools has been a slow, sustained effort among institutions of higher education and the same is true for libraries (Moltz, 2009;Ndoye and Parker, 2010;Weiner, 2009). The problem for librarians is that rather than being fully integrated or embedded into the course and being involved with its design, objectives, and content, they are often in the position of having to wait for patrons to contact them before they can add value. ...
Purpose
– The purpose of this paper is to identify an important area for librarians to positively impact student retention.
Design/methodology/approach
– This programmatic and conceptual piece describes how embedding librarians into the growing enterprise of undergraduate research experiences (UREs) lays a framework for a context in which libraries and librarians directly contribute to the retention of undergraduate students.
Findings
– Librarians are capable of directly contributing to the retention of students. While their efforts, it is contended, contribute routinely and to the actual retention of students, it is difficult for their efforts to register in the assessment of retention used by administrators. This discrepancy can be solved if librarians play a more explicit (and quantifiable) role in retaining students.
Research limitations/implications
– UREs are a growing, but generally untapped trend for librarians; however, because UREs generally correlate with academic success and student retention, they offer librarians a useful entry point to contribute to the academic mission of colleges and universities, and in a measurable way.
Practical implications
– Embedded librarianship poses a number of hurdles for its practitioners; however, it also has the potential for libraries and librarians to become more explicitly connected to overall institutional goals and strengthen their positions in the academy more broadly.
Social implications
– Improving the scientific literacy of undergraduate students and aiding them on their path toward graduation is meaningfully enhanced through the embedding of librarians into the college curriculum.
Originality/value
– Systematically embedding librarians into UREs is not strongly represented in the literature.
The history of assessment and evaluation in education has been characterized by a continual search for objective indicators, absent the critical role played by subjective factors that affect the education process. This chapter presents three separate Q studies to increase understanding of subjective issues. Study 1 examines perspectives of faculty and staff on the purpose of a college education and reveals how divergent views on the purpose of education problematizes the development of a meaningful assessment program. Study 2 examines the culture of assessment of faculty and administrators from different institutions with suggestions offered regarding strategies for reconciling dueling narratives on outcomes assessment. Study 3 involves students in a research methods class given the task of assisting two community organizations in doing a survey to provide data on the need for affordable housing in the local community. This study examines how student essays provided the basis for the Q study revealing how the students experienced the class differently.
Much of the focus of the vast body of literature on assessing student learning is on how to assess and implement reporting processes on learning, whether at the classroom, cocurricular, program, general education, or institution level. Yet what is often not acknowledged within the literature or considered in the various recommendations for implementing new or revised approaches for how to assess student learning are considerations of the who—the individual people engaged in assessment. While there has been movement recently toward humanizing the work of assessment by elevating the individuals and people involved in the process of assessing student learning, this article argues that in order to cultivate and sustain meaningful engagement of people in assessment, assessment professionals need to take up a new line of inquiry: themselves. As individuals engaged in change processes, assessment staff can benefit from examination of their identity development as assessment professionals and that identity tempers their interactions with others. Implications for culture(s) of inquiry and the practice of assessment are presented at the end.
While outcomes assessment is commonplace in colleges of veterinary medicine, no information is published on how veterinary colleges resource, administer, and view assessment. Consequently, this article has the following objectives: (a) to determine the current level of resources (personnel, committees, software) allocated toward education assessment and program evaluation in colleges of veterinary medicine, (b) to characterize any common organizational structures within colleges of veterinary medicine for assessment, (c) to determine assessment personnel (faculty and staff) perceptions regarding whether existing assessment resources and structures are sufficient, and (d) to examine the perceived strength of the culture of assessment. Our survey found that most assessment professionals had been in their position for 4 years or less and over 50% did not have formal assessment training. A majority of respondents agreed that assessment was encouraged and supported at their institution, but there was much less agreement on items related to formal plans and structures. For example, only one quarter of respondents reported that assessment was connected to planning and budgeting, and only one third reported having a formal assessment plan. We hope that our survey will be a resource tracking the development of assessment resources and climate at American colleges of veterinary medicine.
The peer-reviewed and psychometrically validated Test of Scientific Literacy Skills developed by Gormally et al. was used to assess the strengths and weaknesses of a general education natural science program. By comparing the scores of students who had already taken at least one course in this area with the scores of those who had not, and by relating scores to students' perceptions of learning about science, a richer understanding of our program has emerged. The ability to focus on particular skill sets and areas of proficiency has allowed us to identify specific areas to target for improvement.
In this study, results are presented from a rigorous content analysis of responses to two open-ended questions included in the Administrators' Survey of Assessment Culture. A sample of 302 US higher education administrators provided 566 narrative responses addressing (1) the primary reason they conducted assessment on campus, and (2) how they would characterize their campus assessment cultures. Analysis revealed two meta-themes: “Institutional Structures,” including procedures, data usage, and accountability; and “Organizational Culture,” administrators' descriptions of rituals, artifacts, discourse, values, and change related to assessment. Implications are shared for reframing and cultivating notions of institutional cultures of assessment.
This article discusses how a faculty-driven process utilized the mantra of “simple, transparent, and less burdensome” to reform assessment at a regional Midwestern university. We detail how an assessment task force transformed an unwieldy, cumbersome, and disjointed assessment scheme into a systematic, specific, and actionable assessment plan that also began to cultivate a culture of assessment that focuses on improving student learning instead of assessment focused on compliance, that is, meeting accreditation requirements.
Our Doctoral Program in Educational Leadership aims to develop leaders whose fundamental leadership identity is grounded in the Catholic, Dominican tradition. The values, studium, and COR questions are detailed as part of the Dominican ethos. The chapter describes a process by which the comprehensive exams were replaced with a qualitative program assessment focused on building three identities: academic writer, scholarly researcher, and an Edgewood Leader. The Edgewood Leader identity is built upon the Dominican ethos and is the focus of this study. Three models of assessment and the literature on leadership development were used to implement this assessment system. The findings related to leadership identity growth are detailed. Conclusions regarding program themes, strengths and weaknesses are described. The discussion connects the Dominican culture and heritage operating at the institution and the leadership development literature to the program assessment system currently in place.
Given increasing emphasis on assessment and accountability across the globe in higher education, it is clear that individual institutions must take action and provide evidence in response to these concerns. This article provides an example of how one university has developed a central office for assessment, facilitated the development of an assessment process across the university and in departments and programmes, and provides assessment services to the institution that include both academic and administrative areas.
Too often, innovative designs for educational enhancement fail to achieve desired outcomes because they embodied the unintended consequence of built-in obsolescence. This ineffectiveness results from several issues, including a static view of learning and teaching styles, significant personnel dependence, an inability to manage changes in program size, and/or a lack of portability and adoption by the larger educational community. To counter these problems, faculty members at Drexel University are developing a system to disseminate innovative teaching (instructional, curricular, assessment, and operational) methodologies and support tools to automate manual processes. The approach is similar to Apple, Inc.'s highly successful iPhone approach. Drexel University's EduApps Portal collects, evaluates and disseminates novel approaches to educational situations. By creating a Web-based applications portal, Drexel University ensures that new innovations are critically evaluated and widely disseminated in a useable format that provides for ongoing feedback and continuous improvement. This paper outlines the EduApps approach, discusses several examples and provides a blueprint for implementation.
The peer-reviewed and psychometrically validated Test of Scientific Literacy Skills developed by Gormally et al. was used to assess the strengths and weaknesses of a general education natural science program. By comparing the scores of students who had already taken at least one course in this area with the scores of those who had not, and by relating scores to students’ perceptions of learning about science, a richer understanding of our program has emerged. The ability to focus on particular skill sets and areas of proficiency has allowed us to identify specific areas to target for improvement.
Purpose
The purpose of this paper is to explore the idea of using Kotter's eight‐step model for change leadership to create a culture of assessment that is embedded in the organizational culture of an academic library.
Design/methodology/approach
This is a conceptual paper that examines the potential application of a business change model to an academic library change context. Each step of Kotter's model is described within the context of building a culture of assessment, supported by examples and suggestions from the literature of libraries, higher education, organizational behavior and change leadership.
Findings
While building a culture of assessment is critical for libraries in the current environment, it is a difficult and complex process that requires grassroots initiative, top‐down support and strong leadership. Kotter's model for change management provides a pragmatic structure for culture change through behavioral change and change leadership regardless of positional authority.
Practical implications
This paper includes best practices for facilitating a culture of assessment, including necessary resources and possible approaches.
Originality/value
While the literature contains articles about building a culture of assessment, no studies have applied models of change leadership from the management literature to this unique cultural process. This is the first paper that suggests concrete steps for libraries towards building a culture of assessment, even in the absence of an ideal organizational culture. The focus is also on enabling change leadership regardless of whether the change agents have positional authority.
ResearchGate has not been able to resolve any references for this publication.