Lutz Bornmann

Max Planck Society | MPG · Administrative Headquarters, Science Policy and Strategy Department

Research funding systems fundamentally influence how science operates. This paper aims to analyze the allocation of competitive research funding from different perspectives: How reliable are decision processes for funding? What are the economic costs of competitive funding? How does competition for funds affect doing risky research? How do competit...

Following Funk and Owen-Smith (Manag Sci 63:791–817, 2017), Wu et al. (Nature 566:378–382, 2019) proposed the disruption index (DI1) as a bibliometric indicator that measures disruptive and consolidating research. When we summarized the literature on the disruption index for our recently published review article (Leibel and Bornmann in Scientometri...

Following Funk and Owen-Smith (2017), Wu et al. (2019) proposed the disruption index (DI1) as a bibliometric indicator that measures disruptive and consolidating research. When we summarized the literature on the disruption index for our recently published review article (Leibel & Bornmann, 2024), we noticed that the calculation of disruption score...

ChatGPT has arrived in quantitative research evaluation. With the exploration in this Letter to the Editor, we would like to widen the spectrum of the possible use of ChatGPT in bibliometrics by applying it to identify disruptive papers. The identification of disruptive papers using publication and citation counts has become a popular topic in scie...

In science and beyond, quantifications are omnipresent when it comes to justifying judgments. Which scientific author, hiring committee-member, or advisory board panelist has not been confronted with page-long publication manuals, assessment reports, evaluation guidelines, calling for p-values, citation rates, h-indices, or other numbers to judge a...

Sonification uses sound to display larger datasets as an alternative to graphic displays. While it has some advantages over visualization, sonification is still largely limited to a few specialized applications and public outreach projects.

The Disruption index has attracted significant media attention after a study revealed a trend of decreasing disruptiveness in papers and patents. Critiques of the index question these results and highlight issues of inconsistency, time-sensitive biases, and data-induced biases. [Image: see text]

The visualization of publication and citation data is popular in bibliometrics. Although less common, the representation of empirical data as sound is an alternative form of presentation (in other fields than bibliometrics). In this representation, the data are mapped into sound and listened to by an audience. Approaches for the sonification of dat...

Reference Publication Year Spectroscopy (RPYS) is a method to reveal the historical roots and landmark papers of a research field based on cited references data. RPYS visualizes how frequently cited references occur in the cited reference lists of papers published in the research field. Previous RPYS studies revealed various historical aspects of t...

In evaluative bibliometrics and higher education studies, one is frequently confronted with the task of comparing institutions with similar institutions. In this Letter to the Editor, a simple approach is discussed which applies ChatGPT. Although the approach seems to produce promising results (tested with an example at the level of research instit...

Research funding systems are not isolated systems-they are embedded in a larger scientific system with an enormous influence on the system. This paper aims to analyze the allocation of competitive research funding from different perspectives: How reliable are decision processes for funding? What are the economic costs of competitive funding? How do...

Only a few scientists are able to publish a substantial number of papers every year; most of the scientists have an output of only a few publications or no publications at all. Several theories (e.g., the “sacred spark” theory) have been proposed in the past to explain these productivity differences that are complementary and focus on different asp...

Purpose Differences in annual publication counts may reflect the dynamic of scientific progress. Declining annual numbers of publications may be interpreted as missing progress in field-specific knowledge. Design/methodology/approach In this paper, we present empirical results on dynamics of progress in economic fields (defined by Journal of Econo...

The purpose of this paper is to provide a review of the literature on the original disruption index (DI1) and its variants in scientometrics. The DI1 has received much media attention and prompted a public debate about science policy implications, since a study published in Nature found that papers in all disciplines and patents are becoming less d...

This paper compares two measures of organizational size of Higher Education Institutions widely used in the literature: the number of Academic Personnel (AP) measured according to definitions from international education statistics, and the Scientific Talent Pool (STP), i.e. the number of unique authors affiliated to the HEI as derived from the Sco...

We plan to empirically study the assessment of scientific papers within the framework of the anchoring-and-adjustment heuristic. This study is a follow-up study which is intended to answer open questions from the previous study with the same topic Bornmann (2021) and Bornmann (2023). The previous and follow-up studies address a central question in...

The Brazilian Federal Agency for the Support and Evaluation of Graduate Education (CAPES) funds research projects in the framework of their international cooperation programs. It is theobjectiveof the programs toimprove the quality of Brazilian research by fosteringinternational scientific exchangewithinfunded joint projects between researchersfrom...

Bibliometric studies are increasingly employed for research assessment by using statistical methods to analyze scientific outputs and impacts. The main objective of this study is to investigate the research performance of the Kingdom of Saudi Arabia (KSA) with a focus on KSA’s most productive universities. The study describes the research profile o...

The purpose of this paper is to provide a review of the literature on the original disruption index (DI1) and its variants in scientometrics. The DI1 has received much media attention and prompted a public debate about science policy implications, since a study published in Nature found that papers in all disciplines and patents are becoming less d...

We compared field-normalized citation scores from the freely available bibliographic database OpenAlex with those from three commercial databases (Web of Science, Scopus, and Dimensions). The scores have been calculated for nearly 335,000 publications published by 48 German universities in four OECD subject areas between 2013 and 2017. We found var...

L. Wu, Wang, and Evans (2019) introduced the disruption index (DI) which has been designed to capture disruptiveness of individual publications based on dynamic citation networks of publications. In this study, we propose a statistical modelling approach to tackle open questions with the DI: (1) how to consider uncertainty in the calculation of DI...

A key goal of public policy and public administration research is to inform policy decisions. It is not clear, however, to what extent this is the case. In this study, therefore, citations from policy documents to public policy and administration research were analyzed to identify which research contributed most to policy reports and decisions. Add...

John J. Mearsheimer is one of the most renowned theorists of International Relations and is often regarded as one of the primary representatives of the theoretical school of neorealism. To be able to understand and classify his positions, it is necessary to investigate his (theoretical) backgrounds and academic roots. The aim of this paper is, ther...

In our study, we have empirically studied the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether the assessment of a paper can be influenced by numerical information that act as an anchor (e.g. citation impact). We have undertaken a survey of corresponding authors wit...

In this study, we propose a new index for measuring excellence in science which is based on collaborations (co-authorship distances) in science. The index is based on the Erd\H{o}s number - a number that was introduced several years ago. We propose to focus with the new index on laureates of prestigious prizes in a certain field and to measure co-a...

Reference Publication Year Spectroscopy (RPYS) is a bibliometric method originally introduced to reveal the historical roots of research topics or fields. RPYS does not identify the most highly cited papers of the publication set being studied (as is usually done by bibliometric analyses in research evaluation), but instead it indicates most freque...

Climate change is an ongoing topic in nearly all areas of society since many years. A discussion of climate change without referring to scientific results is not imaginable. This is especially the case for policies since action on the macro scale is required to avoid costly consequences for society. In this study, we deal with the question of how r...

Can the male citation advantage (more citations for papers written by male than female scientists) be explained by gender homophily bias, i.e., the preference of scientists to cite other scientists of the same gender category? Previous studies report much evidence that this is the case. However, the observed gender homophily bias may be overestimat...

The governments of most ‘emerging market’ (EM) countries (e.g., India, Mexico, and Brazil) have invested significantly in scientific research over the past decades. These investments are reflected in the countries’ research volume, impact, and international collaboration activities. The EMs’ research support justification delivers the need to evalu...

The normative theory of citing considers citations as rewarding tools to acknowledge the influence of scientific works, while the social constructivist theory of citing considers citations, for example, as persuasion tools used by authors to support their claims, and convince the scientific community that those claims are valid. Other citation theo...

Although it is bibliometric standard to employ field normalization, the detailed procedure of field normalization is not standardized regarding the handling of the document types. All publications without filtering the document type can be used or only selected document types. Furthermore, the field-normalization procedure can be carried out with r...

Identification of young talented individuals is not an easy task. Citation-based data usually need too long to accrue. In this study, we proposed a method based on bibliometric data for the identification of young talented individuals. Three different indicators and their combinations were used. An older cohort with their first publication between...

RPYS is a bibliometric method originally introduced in order to reveal the historical roots of research topics or fields. RPYS does not identify the most highly cited papers of the publication set being studied (as is usually done by bibliometric analyses in research evaluation), but instead it indicates most frequently referenced publications - ea...

Changes in the number of publications in a certain field might reflect the dynamic of scientific progress in this field, since an increase in the number of publications can be interpreted as an increase in the field-specific knowledge. In this paper, we present a methodological approach to analyse the dynamics of science on lower aggregation levels...

In course of the organization of Workshop III entitled “Cited References Analysis Using CRExplorer” at the International Conference of the International Society for Scientometrics and Informetrics (ISSI2021), we have prepared three reference publication year spectroscopy (RPYS) analyses: (i) papers published in Journal of Informetrics; (ii) papers...

In this article, we revisit the analysis of Laband and Tollison (Appl Econ 38(14):1649–1653, 2006) who documented that articles with two authors in alphabetical order are cited much more often than non-alphabetized papers with two authors in the American Economic Review and the American Journal of Agricultural Economics . Using more than 120,000 mu...

Research groups spend time and resources in the process of applying for funding. This issue raises important questions regarding inefficiency and whether the currently used funding mechanisms are adequate. This article aims to identify ways of reducing the inefficiency and the waste of resources when making research funding calls. We look at four w...

Climate change is an ongoing topic in nearly all areas of society since many years. A discussion of climate change without referring to scientific results is not imaginable. This is especially the case for policies since action on the macro scale is required to avoid costly consequences for society. In this study, we deal with the question of how r...

Twitter data are used as alternative metrics (altmetrics) to measure the impact or attention of research. Tweets are used to communicate about papers. However, Twitter data can only be used for research evaluation purposes, if biases do not influence tweet decisions on papers. The existence of biases can only be reasonably investigated using an exp...

Bornmann and Marewski (2019) have adapted the concept of fast-and-frugal heuristics to scientometrics in order to study and guide the application of bibliometrics in research evaluation. Bibliometrics-based heuristics (BBHs) are simple decision strategies for evaluative purposes based on bibliometric indicators. One aim of the heuristics research p...

Usage of field-normalized citation scores is a bibliometric standard. Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field...

Changes in the number of publications in a certain field might reflect the dynamic of scientific progress in this field, since an increase in the number of publications can be interpreted as an increase in the field-specific knowledge. In this paper, we present a methodological approach to analyse the dynamics of science on lower aggregation levels...

Usage of field-normalized citation scores is a bibliometric standard. Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field...

While previous research has mostly focused on the “number of mentions” of scientific research on social media, the current study applies “topic networks” to measure public attention to scientific research on Twitter. Topic networks are the networks of co-occurring author keywords in scholarly publications and networks of co-occurring hashtags in th...

National culture is among those societal factors which could influence research and innovation activities. In this study, we investigated the associations of two national culture models with citation impact of nations (measured by the proportion of papers belonging to the 10 % and 1 % most cited papers in the corresponding fields, PPtop10% and PPto...

Science and technology develop not only along historical trajectories, but also as next-order regimes that periodically change the landscape. Regimes can incur on trajectories which are then disrupted. Using citations and references for the operationalization, we discuss and quantify both the recently proposed “disruption indicator” and the older i...

Controlling for confounding factors is one of the central aspects of quantitative research. While methods like linear regression models are common, their results can be misleading under certain conditions. We demonstrate how statistical matching can be utilized as an alternative that enables the inspection of post-matching balancing. This contribut...

Growth of science is a prevalent issue in science of science studies. In recent years, two new bibliographic databases have been introduced, which can be used to study growth processes in science from centuries back: Dimensions from Digital Science and Microsoft Academic. In this study, we used publication data from these new databases and added pu...

Research on heat waves (periods of excessively hot weather, which may be accompanied by high humidity) is a newly emerging research topic within the field of climate change research with high relevance for the whole of society. In this study, we analyzed the rapidly growing scientific literature dealing with heat waves. No summarizing overview has...

In over five years, Bornmann, Stefaner, de Moya Anegon, and Mutz (2014b) and Bornmann, Stefaner, de Moya Anegón, and Mutz (2014c, 2015) have published several releases of the www.excellencemapping.net tool revealing (clusters of) excellent institutions worldwide based on citation data. With the new release, a completely revised tool has been publis...

In our planned study, we shall empirically study the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether citation decisions are (mainly) driven by the quality of cited references. The design of our study is oriented towards the study by Teplitskiy, Duede [10]. We shall...

The second quantum technological revolution started around 1980 with the control of single quantum particles and their interaction on an individual basis. These experimental achievements enabled physicists, engineers, and computer scientists to utilize long-known quantum features—especially superposition and entanglement of single quantum states—fo...

In course of the organization of Workshop III entitled "Cited References Analysis Using CRExplorer" at the International Conference of the International Society for Scientometrics and Informetrics (ISSI2021), we have prepared three reference publication year spectroscopy (RPYS) analyzes: (i) papers published in Journal of Informetrics, (ii) papers...

We have organized Workshop III entitled "Cited References Analysis Using CRExplorer" at ISSI2021. Here, we report and reflect on this workshop. The aim of this workshop was to bring beginners, practitioners, and experts in cited references analyses together. A mixture of presentations and an interactive part was intended to provide benefits for all...

This study focuses on a recently introduced type of indicator measuring disruptiveness in science. Disruptive research diverges from current lines of research by opening up new lines. In the current study, we included the initially proposed indicator of this new type (Funk & Owen-Smith, 2017; Wu, Wang, & Evans, 2019) and several variants with DI1:...

Research on heat waves (periods of excessively hot weather, which may be accompanied by high humidity) is a newly emerging research topic within the field of climate change research with high relevance for the whole of society. In this study, we analyzed the rapidly growing scientific literature dealing with heat waves. No summarizing overview has...

In-text citation analysis is one of the most frequently used methods in research evaluation. We are seeing significant growth in citation analysis through bibliometric metadata, primarily due to the availability of citation databases such as the Web of Science, Scopus, Google Scholar, Microsoft Academic, and Dimensions. Due to better access to full...

Recently, Google Scholar added a new section to the Google Scholar Author Profiles called “Public Access”, with information on funded (and unfunded) publications. This Letter to the editor discusses the advantages and disadvantages of the new section.

The second quantum technological revolution started around 1980 with the control of single quantum particles and their interaction on an individual basis. These experimental achievements enabled physicists and engineers to utilize long-known quantum features - especially superposition and entanglement of single quantum states - for a whole range of...

Our previous mapping technique of the journal set included in the Journal Citation Reports (JCRs) is further elaborated into sophisticated computer routines.

Methodological mistakes, data errors, and scientific misconduct are considered prevalent problems in science that are often difficult to detect. In this study, we explore the potential of using data from Twitter for discovering problems with publications. In this case study, we analyzed tweet texts of three retracted publications about COVID-19 (Co...

In over five years, Bornmann, Stefaner, de Moya Anegon, and Mutz (2014) and Bornmann, Stefaner, de Moya Anegon, and Mutz (2014, 2015) have published several releases of the www.excellencemapping.net tool revealing (clusters of) excellent institutions worldwide based on citation data. With the new release, a completely revised tool has been publishe...

The disruption index (DI) based on bibliographic coupling and uncoupling between a document and its references was first proposed by Funk & Owen-Smith (2017) for citation relations among patents and then adapted for scholarly papers by Wu et al. (2019). However, Wu & Wu (2019) argued that this indicator would be inconsistent. We propose revised dis...

Open Science is an umbrella term that encompasses many recommendations for possible changes in research practices, management, and publishing with the objective to increase transparency and accessibility. This has become an important science policy issue that all disciplines should consider. Many Open Science recommendations may be valuable for the...

In this study we determined whether Twitter data can be used as social-spatial sensors to show how research on COVID-19/SARS-CoV-2 diffuses through the population to rea