Nordic Institute for Studies in Innovation, Research and Education

Grant peer review processes are pivotal in allocating substantial research funding, yet concerns about their reliability persist, primarily due to low inter-rater agreement. This study aims to examine factors associated with agreement among peer reviewers in grant evaluations, leveraging data from 134,991 reviews across four Norwegian research funders. Using a cross-classified linear regression model, we will explore the relationship between inter-rater agreement and multiple factors, including reviewer similarity, experience, expertise, research area, application characteristics, review depth, and temporal trends. Our findings are expected to shed light on whether similarity between reviewers (gender, age), their experience, or expertise correlates with higher agreement. Additionally, we investigate whether characteristics of the applications—such as funding amount, research area, or variability in project size—affect agreement levels. By analyzing applications from diverse disciplines and funding schemes, this study aims to provide a comprehensive understanding of the drivers of inter-rater agreement and their implications for grant peer review reliability. The results will inform improvements to peer review processes, enhancing the fairness and validity of funding decisions. All data and analysis scripts will be publicly available, ensuring transparency and reproducibility.

Lack of broad public support for climate policies is a major barrier to transitioning to a low‐carbon economy. Multiple studies report higher support for emission‐reducing policies among left‐leaning and climate‐concerned individuals, but the literature is mute about the interaction effects between the two factors. Do left‐leaning individuals with low climate concern support climate policies, and what about right‐leaning individuals with high concern? The answers to these questions will have important implications for political mobilization for climate policies: Increasing concerns for climate change, for instance by highlighting the consequences of climate change, might be a forceful strategy for some audiences but not for others. We examine the potential interaction between climate concern and a composite measure of political value orientation with statistical analyses of Norwegian survey data for the period 2018–2020. As outcome variables, we examine support for both demand‐ and supply‐side policy – fossil fuel pricing and reduced oil production – to provide more general results. Our study indicates a positive relationship between climate concern and policy support across the full political spectrum. Right‐leaning political value orientation moderates but does not deter the relationship between climate concern and support for climate policies. These findings hold for both fossil fuel pricing and reduction in oil production.

The importance of evidence‐based policymaking is widely recognized, but how science influences policy remains insufficiently explored. This study aims to examine how policy documents cite research articles, thereby tracing the complex impact process of scientific research on policymaking. A conceptual model is proposed to classify four types of citation pathways by distinguishing between direct and indirect impacts and observing whether a reinforcement effect is present. To operationalize this model, we collected nearly 10 thousand policy documents related to artificial intelligence (AI) and over 1.6 million links between these policies and their referenced articles. A large‐scale data analysis and a case study were conducted. Results exhibit distinct citation pathways among specific types of institutions, geopolitical areas, and policy areas. Indirect influences emerge as an important mechanism. Research articles from EU countries primarily serve the policymaking of inter‐governmental organizations (IGOs) and the EU, while research articles from the USA significantly support both domestic and foreign policymaking. Notably, IGOs serve as key intermediaries, facilitating the indirect influence of research on policymaking. In addition, while the knowledge from the social sciences provides substantial support for policies in various areas, an increasing involvement of the natural sciences in the development of AI‐related policies is found.

While initiatives that aim to achieve excellence within research in higher education are quite common, initiatives focusing on excellence within education is a more recent phenomenon. This article reports on a document study of calls for proposals, applications and reports from the Norwegian ’Centres for Excellence in Education’ Initiative. Based on the central literature in the field, the study investigates the notions of ‘quality’ and ‘excellence’ that can be identified. The findings revealed four groups of notions of educational quality and excellence: ‘subject-specific’, ‘education-centred’, ‘sector-targeted’ and ‘sector-wide’ notions. The analysis also revealed a skewed distribution between statements pointing inwards to the institutions or subject areas versus outwards to the higher education sector and society, in favour of the inbound notions. The increased impact and participation of students within the centres stood out as an example of ‘whose quality’ has influenced the identified notions. The article concludes by questioning the prominence of excellence within the initiative, asking whether excellence is more of an attitude than something the centres have managed to express or something to reach for rather than something to achieve.

The Covid-19 pandemic resulted in significant disruptions to both international student mobility and labour markets. Against this background, this scoping review documents findings from 35 studies, published between January 2020 and February 2024, that focused on international student mobility, Covid-19, labour market outcomes, and related regulations. The review shows that the Covid-19 pandemic negatively impacted (1) the labour market outcomes of international students before studying abroad, through changes in visa regulations, (2) skill acquisition during their mobility, (3) the duration of their studies, and (4) the availability of jobs during and after their mobility period. These negative impacts occurred at a time when international students’ needs for work were heightened. This ‘double whammy’ was more pronounced among international students from lower socio-economic backgrounds, increasing pre-existing inequalities. It was also affected by students’ residency status and policy changes. These findings underline the critical role that temporality plays in shaping the returns to international student mobility. However, the existing literature on the topic is relatively sparse and has primarily relied upon qualitative approaches and rapid research. Future studies are needed to better understand the impact of Covid-19 on the labour market outcomes of international students.

Many national research and innovation systems include higher education institutions and public research organizations (PRO) with different mandates and tasks. This paper investigates what happens to the relationship between a university and a PRO when they are increasingly pushed towards fulfilling similar tasks and functions. We investigate this through a historical case study of the relationship between a university and a PRO within the field of science and technology and draw on concepts from the institutional logics and institutional complexity literatures to frame the study. We find that in the early phase of the relationship, institutional complexity was handled through a strategy of “structural differentiation” where the university outsourced the commercial logic to the PRO, but in practice the two operated as integrated organizations. In the later phases, growing external demands and internal developments led to a “blending” strategy where the university reincorporated the commercial science logic. This had a fundamental impact on the relationship between the two organizations in terms of increased competition and blurring of roles, which led to increased structural and operational separation. Our study underlines the importance of considering wider, and often unintended, ramifications of policies on the research and innovation system as a whole.

Governments across Europe have implemented mergers between higher education institutions (HEIs) to reduce costs and to increase HEIs’ performance. However, few studies have empirically confirmed the efficiency of such mergers for faculty publication performance. This study addresses this gap by analysing a major reform in the Norwegian higher education system, where institutions were merged with goals such as fostering high-quality research, creating robust academic communities, and developing world-leading academic environments. We specifically analyse the pre- and post-merger publication performance of academics, using a high-quality bibliometric database with longitudinal data and survey results on job characteristics, organizational change, leadership, and commitment. We compare the performance of academics across different types of mergers: mergers between university colleges, mergers of university colleges with universities, and non-merged institutions. The study shows that improvements in research performance have only been partially realized. The mergers did not benefit the performance of existing staff; however, merged institutions appear to have attracted new personnel with stronger research qualifications and higher research output.

Should young children be tested? Proponents view early testing as a necessary instrument for early targeting. Others consider it detrimental to child mental health and with little impact on educational performance. We exploit variation in low-stakes test-taking in mathematics amongst primary school children in Norway, traditionally a low-testing environment. We examine the introduction of difficult mathematics tests. These tests were introduced to provide information for a larger randomised controlled trial (RCT) in Norwegian primary schools. We demonstrate zero effects of testing exposure on educational attainment, but evidence of heterogenous effects across gender and educational background of children. We find no negative effects on student welfare, but testing improves student perceptions of teaching practices, feedback and engagement.

This paper examines the connections between Isaac Newton, Robert Simson, and Adam Smith, highlighting the influence of the Newtonian scientific method. Smith, influenced by Simson’s teachings, regarded Simson as a leading mathematician of their time. Simson’s innovative application of ancient porisms to explain Newtonian fluxions challenged existing perspectives and had a profound impact on the Scottish Enlightenment, which in turn shaped Smith’s seminal work, The Wealth of Nations. This paper clarifies how Smith integrated the Newtonian method into his philosophical and economic theories, emphasizing the interconnections among these influential figures and their lasting contributions.

Drawing on the debate about sustainable academic careers in European countries, this article investigates patterns of time-use on academic activities for female and male researchers in Germany and Norway. More specifically, we investigate different gendered conditions for academic work related to patterns across career stages, disciplines and citizenship. We use data from the German Scientists Survey 2019 and the time-use survey of academic staff in Norway 2021 conducting descriptive and bivariate analyses. Gendered patterns in academic time-use appear to differ according to career stage, country context and disciplines in both countries. For citizenship , gendered patterns are moderate, but gendered patterns across career stage and discipline are reproduced. By simultaneously investigating female and male researchers across career stage, discipline and country, we can show more nuanced patterns of academic time-use. Thus, findings provide implications for further research combining various indicators for female and male researchers to inform policies in both countries.

This article explores the idea that technologies may be considered shapeshifting as they move between and are domesticated in different social worlds. We employ a material-semiotic approach with a domestication and social worlds framework to study the case of Trampe, a bicycle lift built in Trondheim, Norway, in 1993. The study is based on interviews, observation, documents, and news media analysis. We show how the lift shifted shape: although originally intended as bicycling infrastructure, it became domesticated as a tourist attraction in the social world of tourists, and a symbol of the city's efforts to stimulate cycling in the social world of the local government. Trampe has remained the only bike lift in the world and would have been closed long ago due to maintenance costs, but its shapeshifting qualities have prolonged its sociotechnical life after its functional abilities were no longer sufficient. Our study suggests that more attention should be paid to these shapeshifting dynamics when analyzing technologies and their sociotechnical lives. Combining domestication theory with a social worlds perspective helps identify these shapeshifting features.

The paper argues that the empirical study of public policy making almost exclusively deals with structural arrangements and power relations, while giving insufficient attention to how policy entrepreneurs in government develop policy initiatives through venue selection, framing, and dialogue. Drawing on insight from theories of public policymaking and organizational decision-making, public management, and on data from a case study of higher education merger reform in Norway, the article provides a conceptual and empirical contribution by suggesting the significance of managerial dialogue — highlighting a specific combination of dialogue, power asymmetries, managerialism, and venue selection as important elements in higher education reform policymaking.

The speed and volume of scientific publishing is accelerating, both in terms of number of authors and in terms of the number of publications by each author. At the same time, the demand for knowledge synthesis and dissemination is increasing in times of upheaval in the education sector. For systematic reviewers in the field of education, this poses a challenge in the balance between not excluding too many possibly relevant studies and handling increasingly large corpora that result from document retrieval. Efforts to manually summarise and synthesise knowledge within or across domains are increasingly running into constraints on resources or scope, but questions about the coverage and quality of automated review procedures remain. This article makes the case for integrating computational text analysis into current review practices in education research. It presents a framework for incorporating computational techniques for automated content analysis at various stages in the traditional workflow of systematic reviews, in order to increase their scope or improve validity. At the same time, it warns against naively using models that can be complex to understand and to implement without devoting enough resources to implementation and validation steps.

The proliferation of digital tools in higher education has resulted in new possibilities for students’ professional learning and development. Consequently, there is a need for more knowledge regarding the practical incorporation of digital technology in professional learning designs, the rationale behind their selection, and the perceived advantages and disadvantages of their use. In this paper, we discuss findings from a qualitative case study of two academic subjects selected from professional study programmes at Norwegian higher education institutions: occupational therapy and nautical studies. Deploying a constant comparative method of analysis inspired by a grounded theory approach, we discuss how the integration of digital technology in professional learning designs may facilitate students’ professional learning and development. By exploring the data material and connecting the findings to existing literature and theory on professional learning, we show how the thoughtful integration and use of digital technology into a professional learning design may open up new ways of fostering epistemic fluency in future professionals. By focusing on the suitability – rather than the advancedness – of the technology, integrated into a reflective and holistic learning design, the use of technology may facilitate the development of epistemic fluency both within and outside a situated professional context.

The Paris Agreement on climate change is built around a pledge-and-review system, wherein countries submit nationally determined pledges of mitigation commitments. While the agreement’s flexible design has attracted broad participation, its lenient informational requirements for pledges have also engendered considerable ambiguity in countries’ commitments. What are the implications of commitment ambiguity on the credibility of compliance and treaty effectiveness more broadly? This article sheds light on these questions by evaluating the relationship between commitment ambiguity and ambition in countries’ Paris pledges. We introduce a model that disentangles commitment ambiguity into two distinct forms: structural and strategic. We argue that structural ambiguity, which arises from constraints on the information necessary for governments to accurately assess their commitment potential, dampens the ambition levels of pledges. This prudence effect is driven by compliance concern: the anticipation of audience costs in case of non-compliance induces policymakers to adjust ambition downwards. Our empirical analysis of countries’ Nationally Determined Contributions demonstrates that ambiguous pledges exhibit lower ambition than precise ones. In line with our theory, this prudence effect of ambiguity is mainly pronounced among democracies, which face higher domestic audience costs in case of non-compliance with pledges than autocracies. This article contributes a novel theory of ambiguity and ambition in international institutions, which shows how audience-based compliance concerns can discipline cheap talk. Moreover, the article’s empirical findings provide insight into the credibility of countries’ climate pledges under the Paris Agreement, with implications for the treaty’s overall effectiveness prospects.

Research funding allocation varies significantly among countries. Is there an optimal balance between direct allocation (basic funding) and competitive funding schemes? In this chapter, we explore this question using scholarly literature and comparative data on R&D funding.

Societal impact of research does not occur primarily as unexpected, extraordinary incidents of particularly useful breakthroughs in science. Is it more often a result of normal everyday interactions between organisations that need to create, exchange, and make use of new knowledge to further their goals. This chapter discusses how to assess and improve the cocreation and use of research in normal research–society relations.

Since being released in July 2022, an Agreement on Reforming Research Assessment has been signed by more than 700 research performing and funding organisations within and outside of Europe. It is intended to guide a reform and mutual learning process within a coalition of its signatories, CoARA. This chapter analyses the agreement critically and provides recommendations for further development.