Technical ReportPDF Available

The Impact of Higher Education Leadership, Management and Governance Research: Mining the 2014 Research Excellence Framework Impact Case Studies. London: Leadership Foundation for Higher Education.

Authors:
  • Research Support NI

Abstract and Figures

The impact agenda is moving at pace through higher education. Impact is increasingly central to universities’ research strategies. Whilst the term ‘impact’ is bandied about and we think we know what it means, there is little consensus in the sector about how impact is achieved, what kind of organisational conditions support and nurture it, or what success looks like. https://www.lfhe.ac.uk/en/research-resources/publications/index.cfm/S5-02 Impact is a priority for higher education research but there are very few anchors or compasses to orientate university leaders and the research community towards it. To find direction through this complex territory, the Leadership Foundation for Higher Education has commissioned this study of impact case studies from leadership, governance and management (LGM) research submitted to the 2014 Research Excellence Framework (REF) assessment. The REF impact case study database provides, for the first time, a landmark resource for learning about research impact. This study set out to gain a conceptual handle on LGM research impact in terms of the processes and mechanisms involved. In particular, we were interested in mapping out how LGM research in these case studies led to change, made a difference or had impact. The results will help universities think through issues of impact: what the rationale is for understanding impact, why impact is important, and how it can be shown and maximised in its different contexts. The report provides practical advice for the academic community about finding routes to impact. In particular, the Adaptive Systems Framework for Advancing Research (AS-FAR) will be useful for researchers in their impact planning and reporting. It can contribute to the development of pathways to impact.
Content may be subject to copyright.
A preview of the PDF is not available
... Four main categories entitled 'Social-political-economic factors', 'research user factors', 'researcher factors' and 'research output factors'. (Morrow, 2016) Impact processes. Eleven types of impact process including 'stakeholder engagement', 'planning impact ', 'communication', 'and networking' and 'framing/translating outputs'. ...
... Twenty different mechanisms for impact including 'stimulating interest', 'keying into interests' and 'creating spaces for exchange'. (Morrow, 2016) Therefore, processes of engagement, collaboration and partnership between researchers and research users are important elements to promote implementation of research in policy and practice. However, both Morton (2015b) and Morrow (2016) highlight the importance of the research user e.g. ...
... (Morrow, 2016) Therefore, processes of engagement, collaboration and partnership between researchers and research users are important elements to promote implementation of research in policy and practice. However, both Morton (2015b) and Morrow (2016) highlight the importance of the research user e.g. members of the public, businesses, commissioners of health/public services and government to determine whether they will use the findings of the research and therefore its subsequent impact. ...
Article
Researchers have always recognised the importance of disseminating the findings of their work, however, recently the need to proactively plan and drive the impact of those findings on the wider society has become a necessity. Firstly, this is because funders require evidence of return from investment and secondly and crucially because national research assessments are becoming powerful determinants of future funding. In research studies associated with nursing, impact needs to be demonstrated by showing the effect on a range of stakeholders including service users, patients, carers, the nursing workforce and commissioners. Engaging these groups is a well-known challenge influenced by lack of access to academic journals, lack of time to read long complex research papers and lack of opportunities to interact directly with the researchers. This needs to be addressed urgently to enable nursing research to increase the impact that it has on health delivery and the work of clinical practitioners. Social media is potentially a novel way of enabling research teams to both communicate about research as studies progress and to disseminate findings and research funders are increasingly using it to publicise information about research programmes and studies they fund. A search of the healthcare literature reveals that advice and guidance on the use of social media for research studies is not well understood or exploited by the research community. This paper, therefore, explores how using social networking platforms, notably Twitter™ offers potential new ways for communicating research findings, accessing diverse and traditionally hard-to-reach audiences, knowledge exchange at an exponential rate, and enabling new means of capturing and demonstrating research impact. The paper discusses approaches to initiate the setup of social networking platforms in research projects and considers the practical challenges of using Twitter™ in nursing and healthcare research. The discussion is illuminated with examples from our current research. In summary, we suggest that the use of social media micro-blogging platforms is a contemporary, fast, easy and cost effective way to augment existing ways of disseminating research which helps drive impact.
... Patient and public involvement & engagement in the dissemination of research Morton (2015:1) Morrow (2016) and Morton (2015;2015a) assert that engagement and dissemination of research to its 'users' [including patients and the public] is essential for the implementation and uptake of research findings. They focus on research participants or patients which may consist of involvement, engagement or both but not the wider non-professional community. ...
... education for educational staff. Some of our followers also expressed an interest in advising or working on research with the team and this is a clear benefit for patient and public involvement in research generally (Schnitzler et al, 2016;Morrow, 2016;Morton, 2015Morton, , 2015aRyan, 2013;NIHR, 2014;INVOLVE, 2015). recommends taking care to consider the amount of time and commitment to monitor and manage such a group effectively. ...
Article
Full-text available
Background: The social media platform Facebook boasts of having more than 1,284 million daily active users globally. A large proportion of adults use the internet to seek health-related information. Aim: To critically analyse the use of social media to engage parents of children with attention deficit hyperactivity disorder (ADHD) with the findings of clinical research. Discussion: Observation and qualitative content analysis combined with Facebook Audience Insights were used to evaluate the levels of engagement and interaction with different types of research information. More than 1,100 people from 41 nations engaged with the group. Sharing information through a range of Facebook functions was found to successfully achieve engagement and reach this demographic nationally and internationally. Conclusion: Lay research users are eager to engage and understand clinical research. Social media platforms are an appropriate way to disseminate research. Implications for practice: This paper presents a much-needed evidence-based framework that nursing and health researchers can use for effective communication.
... IWH is building up a small database of case studies, and it may be useful to analyze them further. Morrow 35 completed an evaluation of the Research Excellence Framework case studies collected in the UK. They found that the case studies were useful in determining trends in research impact. ...
Article
Full-text available
Research organizations, governments and funding agencies are increasingly interested in the impact of research beyond academia. While a growing literature describes research impacts in healthcare and health services, little has focused on occupational health and safety research. This article describes a research impact model that has been in use for over a decade. The model was developed to track and describe the impact of research conducted by a mid‐sized institute that focuses on work and health. Model development was informed by existing models, with the goal of contextualizing the institute's case studies describing three types of research impact: evidence of the diffusion of research; evidence of research informing decision‐making; and evidence of societal impact. A logic model describes research actions and outcomes, as well as key audiences and knowledge transfer approaches. A unique element is its indication of the level of difficulty in determining types of impact. The model compares well with current research impact models developed or used in healthcare and health services research, and it has been useful in guiding a mid‐sized research organization's process for tracking and describing the impact of its research. It may be useful to other small and mid‐sized research organizations that focus on workplace health and safety.
... incorporate other potential influences such as public opinion, media reporting of issues, economic climate, policy infrastructure, political ideology and priorities, stakeholder interests, expert advice, industry perspectives, health professional regulatory body standards, prescriber environment and resources, and patient needs, values, and preferences. 3,21,22 The CNODES Knowledge Translation Team works with CNODES researchers and staff, DSEN, and decision makers to develop and strengthen linkages, trust, respect, and partnerships to facilitate their collective contribution to promote drug safety for Canadians. ...
Article
Full-text available
Purpose The Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context. Methods Stakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context. Results A theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing. Conclusion A flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.
... Such ''logic models'' have their place, but they are particularly unfit for purpose for assessing the interactions, negotiations, and activities of an unstable and organically evolving research system in which the chain of causation for any particular outcome is diffuse and contested (Greenhalgh, Jackson, Shaw and Janamian 2016: 409) This analysis points to the arguably underdeveloped definitions and methods for measuring impact-at least in the humanities-and the problems with a wider 'audit culture' approach to capturing change. However, nowhere in the literature are academic perceptions of research impact discussed as one part of a paradigm shift, with most hinged on the REF2014 and its outcomes (Grant 2015 1 ;Morrow 2016). While important for a better critical understanding of academic culture, structures and economy, such research neglects perceptions of the incremental rise of impact over the very long term. ...
Article
Full-text available
Since the Research Excellence Framework of 2014 (REF2014) ‘impact’ has created a conceptual conundrum gradually being pieced together by academics across the Higher Education sector. Emerging narratives and counter-narratives focus upon its role in dictating institutional reputation and funding to universities. However, not only does literature exploring impact, rather than ‘REF2014 impact’ per se, seldom see it as part of a changing sector, but it often also treats it as a new phenomenon within the political and social sciences. Here, we draw upon academic perceptions of impact set in motion in the UK during the 1970s, we critique the underlying assumption that impact is new. We argue three key points to this end. Firstly, contrary to much of the literature examining academic perceptions of impact, it is a long-standing idea. Secondly, within such accounts, the effect of academic research on policy and society (which is long-standing) and the instrumentalisation of impact as a funding requirement (which is relatively new) are conflated. Thirdly, this conflation creates a novelty effect. In the context of a wider sea change to Higher Education, we examine different forms of consent, acceptance, endorsement and resistance surrounding the ‘new’ impact agenda to argue that this ‘novelty effect’ masks an important transitory process of acclimatisation among academics.
Article
Research is increasingly under the spotlight to demonstrate impact as well as ‘World Class’ quality. Impact measures were introduced into the United Kingdom’s Research Excellence Framework in 2014, and are being adopted in other countries. However, impact is a concept that is both loosely applied and often contested. It needs unpacking to build understanding about how it can be effectively evaluated. This article uses findings from a subsample of 1309 research-based case studies in leadership, governance and management submitted to the Research Excellence Framework. The mixed-method study used Complex Adaptive Systems as a lens to explore perspectives of impact as a consequence of research, as a process and as an emerging concept. We describe some of the rich patterns of impact practices, mechanisms for exchange, connections with context, and types of measures, used to evidence impact. The article helps to illuminate the complexity of impact and implications for its evaluation.
Article
Research impact and its measurement are of increasing importance. This is particularly significant for agricultural science, which is expected to produce solutions to future challenges that will arise from population growth, climate change and ecosystem degradation. Much econometric effort has been devoted to analysis of investment in agricultural research and its effects on farm productivity. This analysis, reviewed here, has produced a consensus suggesting that returns are high, although they are achieved only after long lags. However, policy-makers perceive the occurrence of impacts as too few, and poorly targeted with respect to their needs. An attribution gap between the outcomes of agricultural research and how they reach farmers has motivated evaluation of the process of transmission and translation of agricultural research outputs into ultimate impacts. This gap can be narrowed by Participatory Impact Pathway Analysis, implemented mostly so far in low income countries. However, it is a costly and cognitively complex approach. Content analysis of the UK's 2014 REF Impact Case Studies uncovers the mind set of researchers and their managers regarding the description of impact and how it is supposed to occur. This reveals a nascent conservatism that focuses on research that can be shown to have impact, rather than research impact itself. From the overall discussion it can be concluded that the impact evaluation of agricultural science raises more profound issues than simply efficiency or transparency. Confirmation bias threatens impact evaluation, principally by distracting from other important stories about how and why the ultimate effects occur, but also by transforming the nature of the process itself. Methodological pluralism, with greater integration and triangulation between different evaluation approaches, is a promising means of resolving these problems.
Article
Full-text available
Businesses today are experiencing profound pressures to reform and improve stakeholder-related practices and their impacts on stakeholders and the natural environment - in short, to manage responsibly as well as profitably. Pressures for expanding the emphasis on profits to managing responsibly derive from three general sources: primary stakeholders such as owners, employees, customers, and suppliers; secondary stakeholders such as non-governmental organizations (NGOs), activists, communities, and governments; and general societal trends and institutional forces. The latter include a proliferation of "best of" rankings, the steady emergence and development of global principles and standards that are raising public expectations about corporate responsibility, and new reporting initiatives emphasizing the socalled triple bottom lines of economic, social and environmental performance. To respond to these pressures, many multinational corporations (MNCs) in particular are developing what we have called total responsibility management (TRM) systems approaches for managing their responsibilities to stakeholders and the natural environment. In this article we outline the dominant pressures pushing the evolution of total responsibility management and present a managerial framework that highlights the three main components of TRM approaches - inspiration (vision), integration, and improvement/innovation - with the indicators inherent to a responsibility measurement approach.
Book
Addressing the question of how leadership can work most successfully in universities, Engaging Leaders strengthens the sense of shared professional knowledge and capability amongst leaders in higher education. Presenting a narrative of change which not only spells out why universities need to work differently, this book also takes the reader through clear practical steps which any practising leader can take in order to build a collaborative professional culture which supports and challenges all members of an academic community. Using a selection of case studies from UK and international universities, the book explores why working collectively remains a fundamental challenge for many higher education institutions and looks at the benefits of creating a collective commitment throughout universities. It covers: • action learning and professional learning communities • emotionally-intelligent organisational cultures • feedback as an intrinsic tool • building partnerships and collaborations • Higher Education and public value • cultures of innovation and sustainable practices. Engaging Leaders is for those who see themselves as leaders in higher education, whether or not this is recognised in their job title. It provides stimulating perspectives on what they might do to become more engaged and engaging, and includes examples of inspiring practice which is already making a difference in universities, colleges and new private providers. It will be of value to established managers as well as those new in post, and also for those participating in postgraduate programmes.
Chapter
This paper critically analyses the nature and state of decision support systems (DSS) research. To provide context for the analysis, a history of DSS is presented which focuses on the evolution of a number of sub-groupings of research and practice: personal DSS, group support systems, negotiation support systems, intelligent DSS, knowledge management-based DSS, executive information systems/business intelligence, and data warehousing. To understand the state of DSS research an empirical investigation of published DSS research is presented. This investigation is based on the detailed analysis of 1,020 DSS articles published in 14 major journals from 1990 to 2003. The analysis found that DSS publication has been falling steadily since its peak in 1994 and the current publication rate is at early 1990s levels. Other findings include that personal DSS and group support systems dominate research activity and data warehousing is the least published type of DSS. The journal DSS is the major publishing outlet; US ‘Other’ journals dominate DSS publishing and there is very low exposure of DSS in European journals. Around two-thirds of DSS research is empirical, a much higher proportion than general IS research. DSS empirical research is overwhelming positivist, and is more dominated by positivism than IS research in general. Design science is a major DSS research category. The decision support focus of the sample shows a well-balanced mix of development, technology, process, and outcome studies. Almost half of DSS papers did not use judgement and decision-making reference research in the design and analysis of their projects and most cited reference works are relatively old. A major omission in DSS scholarship is the poor identification of the clients and users of the various DSS applications that are the focus of investigation.