Ismael Rafols

Leiden University | LEI · Center for Science and Technology Studies

This article reviews Loet Leydesdorff’s contributions to science mapping. It explains how over the years, his mapping techniques evolved from journal mapping to overlays over global maps of science and finally towards interactive interfaces portraying multiple classifications and ontologies. It then critically reviews the challenges of current appr...

Current modes of developing Open Science are not leading to expected outcomes: evidence is growing that, first, in some dimensions they are leading to more inequity and second, their societal impact is unclear or limited. In the face of these insights, I argue that the conceptualisation and promotion of Open Science needs rethinking to serve its go...

For science and technology to contribute to social justice, scientometrics analyses need to be able to produce descriptions that are appropriate to specific contexts and values. Given that the participation of stakeholders in the analyses is crucial for these perspectives to be realized, this requires research information analyses that are “open”....

La ciencia, la tecnología y la innovación han devenido en un verdadero motor de transformaciones de toda índole en escalas locales y globales. Sin embargo, las capacidades desigualmente distribuidas a nivel mundial han generado desarrollos desproporcionales en estas distribuciones. No es tan solo un problema de la ciencia en sí, sino del modo en có...

chapter in report for UNDP

Background Following a flurry of policies for Open Science (OS), there is now a wave of initiatives to monitor its adoption. However, the great diversity of understandings and activities related to Open Science makes monitoring very challenging. There is a danger that by focusing on what can be readily observed (e.g. publications) many other OS act...

This study examines whether and how India’s research efforts address the country’s health needs. It does so by investigating the degree of alignment between the burden of diseases in India and research efforts being undertaken in health and biomedical sciences by Indian institutions. WHO data from Global Health Estimates on Disability-Adjusted Life...

Background: Following a flurry of policies for Open Science (OS), there is now a wave of initiatives to monitor its adoption. However, the great diversity of understandings and activities associated related to Open Science makes monitoring very challenging. There is a danger that by focusing on what can be readily observed (e.g. publications) many...

A current issue in mission-oriented research policy is the balance of priorities in research portfolios. In parallel, in health policies, there is a debate on shifting research away from biomedical treatments towards health promotion and well-being. In this study, we examine if research agendas are responsive to these demands in cardiometabolic and...

This study examines whether India's research efforts address the country's health needs. It does so by investigating the degree of alignment between burden of diseases in India and knowledge production in health and biomedical sciences by Indian institutions. WHO data from Global Health Estimates on Disability-Adjusted Life Years (DALYs) is used as...

This special issue presents a small selection of contributions from Latmétricas 2021, among those submitted by authors. This Conference brought together more than 500 practitioners and researchers from every corner of Latin America and the Caribbean. There were sessions on new metrics, measuring open access effects on productivity and impact, the r...

The reform of evaluation proposed at European level in the recent Agreement on Reforming Research Assessment represents a window of opportunity to reform the Spanish evaluation system, at a moment when the new Organic Law of the University System (LOSU) is being debated. This agreement establishes the primacy of peer review based on qualitative cri...

A current issue in mission-oriented research policy is the distribution of priorities in research portfolios. In parallel, in health policies there is a debate on shifting research away from biomedical treatments towards health promotion and well-being. In this study we examine if research agendas are responsive to these demands in cardiometabolic...

This report aims to map and provide information on current cardiometabolic research to inform and help funding agencies reflect on their research and innovation priorities. The emphasis lies in showing the distribution of current research across fields and approaches against perceived societal needs.

This report aims to map and provide information on current research in mental health to inform and help funding agencies reflect on their research and innovation priorities. The emphasis lies in spotting misalignments and gaps of current research efforts against perceived societal needs and demands in light of shifting mental health needs.

En una carta publicada hace seis meses (Delgado-López-Cózar, et al., 2021), pocas semanas después de que la Agencia Española de Evaluación se adhiriera a la declaración dora (dora, 2012), hacíamos un llamamiento a las autoridades científicas españolas para que abandonaran las políticas de evaluación de la investigación basadas en un uso excesivo e...

This letter is a call to the Spanish scientific authorities to abandon current research evaluation policies, which are based on an excessive and indiscriminate use of bibliometric indicators for nearly all areas of scientific activity. This narrow evaluation focus is especially applied to assess the individual performance of researchers. To this en...

Analysts are rapidly developing methods to map publications to SDGs in the face of policy demands. However, as reported by Armitage et al. (2020), a high degree of inconsistency is found when comparing the bibliometric corpora obtained with different approaches. These inconsistencies are not due to minor technical issues, but instead they represent...

Synthesis centers are a form of scientific organization that catalyzes and supports research that integrates diverse theories, methods and data across spatial or temporal scales to increase the generality, parsimony, applicability, or empirical soundness of scientific explanations. Synthesis working groups are a distinctive form of scientific colla...

Although the main bibliometric databases (Web of Science (WoS) and Scopus) claim to includejournals on the basis of scientific and publication standards, there have long been concerns thatits coverage is biased in favour of journals from industrialised countries and towards topicsrelevant to these countries. In this article, we investigate this cla...

The use of indicators is generally associated with a reduction of perspectival diversity in evaluation that often facilitates making decisions along dominant framings – effectively closing down debate. In this chapter we will argue that while this is indeed often the case, indicators can also be used to help support more plural evaluation and foste...

In this chapter we build upon Moed’s conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalisation of higher education presents important challenges to institutions worldwide, w...

Growth in collaborative research raises difficulties for those tasked with research evaluation, particularly in situations where outcomes are slow to emerge. This article presents the ‘Diversity Approach to Research Evaluation’ (DARE) as a novel way to assess how researchers engaged in knowledge creation and application work together as teams. DARE...

Pandemic politics highlight how predictions need to be transparent and humble to invite insight, not blame. Pandemic politics highlight how predictions need to be transparent and humble to invite insight, not blame.

Background: It has been well established that research is not addressing health needs in a balanced way - much more research is conducted on diseases with more burden in high-income countries than on those with more burden in lower-income countries. In this study, we explore whether these imbalances persist and inquire about the possible influence...

Research question: What are the current research trends in human assisted reproduction around the world? Design: An analysis of 26,000+ scientific publications (articles, letters and reviews) produced worldwide between 2005 and 2016. The corpus of publications indexed in PubMed was obtained by combining the Medical Subject Heading (MeSH) terms:...

In this chapter we build upon Moed's conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalization of higher education presents important challenges to institutions worldwide, w...

Study question: What are the current trends of research in Human Assisted Reproduction around the world? Summary answer: USA is the leading country, followed by the UK, China, France and Italy. The largest research area is “laboratory techniques”, although other areas such as “public health”, “quality, ethics and law” and “female factor” are gainin...

Synthesis centers are a recently-developed form of scientific organization that catalyzes and supports a form of interdisciplinary research that integrates diverse theories, methods and data across spatial or temporal scales, scientific phenomena, and forms of expertise to increase the generality, parsimony, applicability, or empirical soundness of...

The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify ‘high quality’ journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards...

The use of indicators in research policy and evaluation is widely perceived as problematic. Responding to demands for explicit normative framings in STI governance, I propose an agenda for transforming the place and role of indicators in policy. Given that expert advice should not separate knowledge formation from decision-making under conditions o...

To what extent is scientific research related to societal needs? To answer this crucial question systematically we need to contrast indicators of research priorities with indicators of societal needs. We focus on rice research and technology between 1983 and 2012. We combine quantitative methods that allow investigation of the relation between ‘rev...

In this article, we develop a method that uses altmetric data to analyse researchers' interactions, as a way of mapping the contexts of potential societal impact. In the face of an increasing policy demand for quantitative methodologies to assess societal impact, social media data (altmetrics) have been presented as a potential method to capture br...

Since outbreaks in 2003, avian influenza has received a considerable amount of funding and become a controversial science policy issue in various respects. Like in many other global and multidisciplinary societal problems fraught with high levels of uncertainty, a variety of perspectives have emerged over how to “tackle” avian influenza and public...

The spread of bibliometric indicators in research evaluation and policy has been accompanied by an increasing realisation that indicators' use is often problematic and/or inappropriate. In this article, we propose a parsimonious framework to analyse the conditions under which the use of indicators become problematic. We propose that indicators' use...

The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify “high quality” journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards...

Scientometric techniques have been remarkably successful at mapping science but they face important difficulties when mapping research for societal problems possibly because they they are derived only from scientific documents and thus do not rely on non-academic expert knowledge. Here we aim to explore how ontologies can be used in science mapping...

The purpose of this study is to modernize previous work on science overlay maps by updating the underlying citation matrix, generating new clusters of scientific disciplines, enhancing visualizations, and providing more accessible means for analysts to generate their own maps. We use the combined set of 2015 Findings appear in the form of visuals t...

In many countries research evaluations confer high importance to mainstream journals, which are considered to publish excellent research. Accordingly, research evaluation policies discourage publications in other non-mainstream journals under the assumption that they publish low quality research. This approach has prompted a policy debate in low an...

Visualization of research activities can contribute to identification of research opportunities, program management, and science policy. Science overlay maps have been applied over the past years in a number of scientometric studies to elucidate disciplinary engagement. This paper updates the underlying citation matrix, generates new clusters of sc...

Science policy is increasingly shifting towards an emphasis in societal problems or grand challenges. As a result, new evaluative tools are needed to help assess not only the knowledge production side of research programmes or organisations, but also the articulation of research agendas with societal needs. In this paper, we present an exploratory...

In this article, we develop a method that uses altmetric data to analyse researchers' interactions, as a way of mapping the contexts of potential societal impact. In the face of an increasing policy demand for quantitative methodologies to assess societal impact, social media data (altmetrics) has been presented as a potential method to capture bro...

Visual analytics has been increasingly used to help to better grasp the complexity and evolution of scientific and technological activities over time, across science and technological areas and in organisations. This chapter presents general insights into some important fields of expertise such as mapping, network analysis and visual analytics appl...

The evaluation of interdisciplinary research is complicated by ambiguity about what interdisciplinarity is and what it should be. The question is topical, as evaluation plays an important role in how science is being shaped and changed today. The chapter performs a meta-analysis of the concept of interdisciplinarity in research evaluation, and give...

In many countries research evaluations confer high importance to mainstream journals, which are considered to publish excellent research. Accordingly, research evaluation policies discourage publications in non-mainstream journals under the assumption that they publish low quality research. This approach has prompted a policy debate in low and midd...

Over the last decade, the desire to address complex societal problems, achieve impact, and create value from working across disciplines has led to an increased emphasis on interdisciplinary research (IDR) among researchers, funders, and higher education institutions (HEIs) in the UK (Nurse, 2015; British Academy, 2016). Simultaneously, there has be...

There has been a growing focus, among both policymakers and researchers, on ideas of responsible research and innovation, as a way to ensure that new sources of public value are captured. The EC defines RRI as " a process where all societal actors (researchers, citizens, policy makers, business, third sector organisations etc) work together during...

Current evaluation frameworks in research policy were designed to address: 1) life and natural sciences, 2) global research communities, and; 3) scientific impact. This is problematic, as they do not adapt well to SSH scholarship, to local interests, or to consider broader societal impacts. This paper discusses three different evaluation frameworks...

In the last two decades many countries in the periphery of the science system have developed policies to improve the ‘quality’ of their science. These policies for ‘catching up’ with developed countries include policy instruments based on notions of ‘quality’ or ‘excellence’ – either explicitly or implicitly. An analysis of these policy instruments...

EDITOR'S SUMMARY Focusing on graphene, a material made from carbon atoms, researchers constructed a map illustrating the material in the context of scientific developments and patent activity over time. Two basemaps reflecting general science and patent activity were created, in addition to two overlay maps reflecting scientific publication and pat...

The perception that many promising results from basic biomedicine have not systematically contributed to medical treatments and, ultimately, health care improvements, has led to a wide range of publicly funded initiatives aiming at facilitating the ‘translation' of scientific discoveries into beneficial applications and practices. Many of these ini...

A variety of quantitative techniques have been used in the past in future-oriented technology analysis (FTA). In recent years, increased computational power and data availability have led to the emergence of new techniques that are potentially useful for foresight and forecasting. As a result, there are now many techniques that might be used in FTA...

This paper examines the use of scientometric overlay mapping as a tool of ’strategic intelligence’ to aid the governance of emerging technologies. We develop an integrative synthesis of different overlay mapping techniques and associated perspectives on technological emergence across the geographical, social, and cognitive spaces. To do so, we long...

This article analyses the effect of degree of interdisciplinarity on the citation impact of individual publications for four different scientific fields. We operationalise interdisciplinarity as disciplinary diversity in the references of a publication, and rather than treating interdisciplinarity as a monodimensional property, we investigate the s...