Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Background
The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades. Moreover, recent studies indicate that...
Contexts in source publication
Context 1
... recent years there has been a surge in the number of published academic papers, with >7 million new papers each year and >1.8 million papers with ≥5 references (see Fig. 1).(There is a decline in the number of papers after 2014, probably due to missing papers in the MAG dataset, which was released in 2016.) Additionally, by analyzing the language of the papers' titles, we observed an increase in papers with non-English titles (see Fig. ...
Context 2
... mean number of references per paper (see Fig. S9). In addition, by analyzing the reference number distributions grouped by publishing decade, we can observe that higher numbers of references have become increasingly common. For example, in 1960 few papers had >20 references, but by 2010 many papers had >20 references, and some >40 references (see Fig. ...
Context 3
... also examined how self-citation trends have changed, and we observed that both the total number of self-citations and the percentage of papers with self-citations increased substantially (see Fig. S12). Also, the mean number of self-citations per paper, as well as the maximal number of self-citations in each year, increased sharply (see Fig. 7). For example, ∼3.67% of all papers in 1950 contained ≥1 self-citation, while 8.29% contained self-citations in 2014 (see Fig. S12). Moreover, the maximal number of self-citations in a single ...
Context 4
... percentage of papers with self-citations increased substantially (see Fig. S12). Also, the mean number of self-citations per paper, as well as the maximal number of self-citations in each year, increased sharply (see Fig. 7). For example, ∼3.67% of all papers in 1950 contained ≥1 self-citation, while 8.29% contained self-citations in 2014 (see Fig. S12). Moreover, the maximal number of self-citations in a single paper increased sharply from 10 self-citations in a paper published in 1950 to >250 self-citations in a paper published in 2013 (see Fig. ...
Context 5
... that the percentage of papers with no citations other than selfcitations after 5 years decreased (see Fig. 9). Nevertheless, 72.1% of all papers published in 2009, and 25.6% of those with ≥5 references, were still without any citations after 5 years (see Fig. 9). Moreover, the total number of papers without any citations increased sharply (see Fig. ...
Context 6
... by analyzing the citation distributions of papers published in different decades, we discovered that citation distributions changed notably over time (see Fig. ...
Context 7
... analyzing the number of new authors each year, we discovered a sharp increase over time, with several million new authors publishing each year in recent years (see Fig. S14). (However, it is possible that the same author has several MAG author IDs.) Additionally, when analyzing the trends grouped by the authors' academic birth decades, we discovered a substantial increase in the mean number of published papers for the later birth decades (see Fig. 11). For example, researchers who started their careers in ...
Context 8
... million new authors publishing each year in recent years (see Fig. S14). (However, it is possible that the same author has several MAG author IDs.) Additionally, when analyzing the trends grouped by the authors' academic birth decades, we discovered a substantial increase in the mean number of published papers for the later birth decades (see Fig. 11). For example, researchers who started their careers in 1950 published on average 1.55 papers in a period of 10 years, while researchers who started their careers in 2000 published on average 4.05 papers in the same time frame. Furthermore, we observed that authors who started their careers after 1990 tended to publish more in ...
Context 9
... 10 years, while researchers who started their careers in 2000 published on average 4.05 papers in the same time frame. Furthermore, we observed that authors who started their careers after 1990 tended to publish more in conferences in the first years of their career than their more senior peers who started their careers in the 1950s or 1970s (see Fig. S15). For example, researchers who started their careers in the 1970s published on average ∼2 conference papers and 1.65 journal papers after 10 years; researchers who started their careers in the 2000s published ∼4 conference papers and 2.59 journal papers in the same time ...
Context 10
... can also observe that the mean number of co-authors has considerably increased over the decades (Fig. 12). Moreover, we note that researchers who started their careers in the 1950s and 1970s had on average only a few co-authors over a period of 25 years, while researchers who started their careers in the 1990s had >60 co-authors in the same career length of 25 years (see Fig. 12). Last, by exploring how author sequence numbers evolved, we ...
Context 11
... mean number of co-authors has considerably increased over the decades (Fig. 12). Moreover, we note that researchers who started their careers in the 1950s and 1970s had on average only a few co-authors over a period of 25 years, while researchers who started their careers in the 1990s had >60 co-authors in the same career length of 25 years (see Fig. 12). Last, by exploring how author sequence numbers evolved, we discovered that with seniority, the researchers' median sequence number increased (see Fig. S16). Additionally, with seniority, the percentage of published papers with the researcher listed as the first author decreased (Fig. 13). Moreover, by looking at the decade ...
Context 12
... and 1970s had on average only a few co-authors over a period of 25 years, while researchers who started their careers in the 1990s had >60 co-authors in the same career length of 25 years (see Fig. 12). Last, by exploring how author sequence numbers evolved, we discovered that with seniority, the researchers' median sequence number increased (see Fig. S16). Additionally, with seniority, the percentage of published papers with the researcher listed as the first author decreased (Fig. 13). Moreover, by looking at the decade researchers started their careers, we can see a sharp decline in the percentages of first authors (Fig. 13). Overall, early-career researchers are publishing more in ...
Context 13
... >60 co-authors in the same career length of 25 years (see Fig. 12). Last, by exploring how author sequence numbers evolved, we discovered that with seniority, the researchers' median sequence number increased (see Fig. S16). Additionally, with seniority, the percentage of published papers with the researcher listed as the first author decreased (Fig. 13). Moreover, by looking at the decade researchers started their careers, we can see a sharp decline in the percentages of first authors (Fig. 13). Overall, early-career researchers are publishing more in their careers but appear as first authors much less than in previous ...
Context 14
... with seniority, the researchers' median sequence number increased (see Fig. S16). Additionally, with seniority, the percentage of published papers with the researcher listed as the first author decreased (Fig. 13). Moreover, by looking at the decade researchers started their careers, we can see a sharp decline in the percentages of first authors (Fig. 13). Overall, early-career researchers are publishing more in their careers but appear as first authors much less than in previous ...
Context 15
... analyzing journal trends using the SJR and MAG datasets, we discovered that the number of journals increased greatly over the years, with 20,975 active ranked journals in 2016 (Fig. 14). Furthermore, we observed that hundreds of new ranked journals were published each year (see Figs S17 and S18). In addition, we discovered that the number of published papers per journal increased sharply, from a mean of 74.2 papers in 1999 to a mean of 99.6 papers in 2016 (Fig. 14). We also observed that in recent years, journals that ...
Context 16
... analyzing journal trends using the SJR and MAG datasets, we discovered that the number of journals increased greatly over the years, with 20,975 active ranked journals in 2016 (Fig. 14). Furthermore, we observed that hundreds of new ranked journals were published each year (see Figs S17 and S18). In addition, we discovered that the number of published papers per journal increased sharply, from a mean of 74.2 papers in 1999 to a mean of 99.6 papers in 2016 (Fig. 14). ...
Context 17
... over the years, with 20,975 active ranked journals in 2016 (Fig. 14). Furthermore, we observed that hundreds of new ranked journals were published each year (see Figs S17 and S18). In addition, we discovered that the number of published papers per journal increased sharply, from a mean of 74.2 papers in 1999 to a mean of 99.6 papers in 2016 (Fig. 14). We also observed that in recent years, journals that publish thousands of papers have become more common. For example, in 2016, according to the SJR dataset, 197 journals published >1,000 papers ...
Context 18
... exploring how various metrics have changed over time, we discovered the following: First, over the past 18 years, the number of papers published in Q1 and Q2 journals more than doubled, from 550,109 Q1 papers and 229,373 Q2 papers in 1999 to 1,187,514 Q1 papers and 554,782 Q2 papers in 2016 (Fig. 15). According to the SJR dataset, in 2016, 51.3% of journal papers were published in Q1 journals and only 8.66% were published in Q4 journals. Second, the h-index decreased over recent years from a mean value of 37.4 and median of 23.0 in 1999 to a mean value of 31.3 and median of 16 in 2016 (see Fig. S19). Third, we noted that the SJR ...
Context 19
... Q1 papers and 554,782 Q2 papers in 2016 (Fig. 15). According to the SJR dataset, in 2016, 51.3% of journal papers were published in Q1 journals and only 8.66% were published in Q4 journals. Second, the h-index decreased over recent years from a mean value of 37.4 and median of 23.0 in 1999 to a mean value of 31.3 and median of 16 in 2016 (see Fig. S19). Third, we noted that the SJR and the mean number of citations measures both increased considerably during the past 18 years (see Figs 16 and ...
Context 20
... the number of papers in top journals doubling between 2000 and 2014, the number of authors increased substantially (see Fig. S21). (The total number of authors each year was determined by summing the number of authors in each published paper.) Additionally, by calculating the mean academic career ages of first and last authors, we discovered that in recent years the mean academic age has increased notably (Fig. 17). Moreover, when looking at first and last ...
Context 21
... 2014, the number of authors increased substantially (see Fig. S21). (The total number of authors each year was determined by summing the number of authors in each published paper.) Additionally, by calculating the mean academic career ages of first and last authors, we discovered that in recent years the mean academic age has increased notably (Fig. 17). Moreover, when looking at first and last authors who previously published in one of the selected top-30 journals, we discovered that over time the percentage of returning authors increased substantially (see Fig. 18). By 2014, 46.2% of all published papers in top-30 selected journals were published by last authors who had published ≥1 ...
Context 22
... academic career ages of first and last authors, we discovered that in recent years the mean academic age has increased notably (Fig. 17). Moreover, when looking at first and last authors who previously published in one of the selected top-30 journals, we discovered that over time the percentage of returning authors increased substantially (see Fig. 18). By 2014, 46.2% of all published papers in top-30 selected journals were published by last authors who had published ≥1 paper in a top-30 selected journal before (Fig. ...
Context 23
... and last authors who previously published in one of the selected top-30 journals, we discovered that over time the percentage of returning authors increased substantially (see Fig. 18). By 2014, 46.2% of all published papers in top-30 selected journals were published by last authors who had published ≥1 paper in a top-30 selected journal before (Fig. ...
Context 24
... S23); e.g., in Cell, the last authors' ca- reer ages increased from ∼4.5 years in 1980 to ∼20 years in 2014 (see Fig. S23); and (iii) the percentage of returning authors in the vast majority of the selected journals increased drastically; e.g., in Nature Genetics, in 86.6% of 2014 papers, ≥1 of the authors had published in the journal before (see Fig. 19). ...
Context 25
... it was >33 (see r A big variance in each L0 field's mean and median number of citations after 5 years. For example, for 2009 papers in the fields of computer science and political science, the median number of citations 5 years after publication was 4, while in biology and environmental science, the median was 9 and 13 citations, respectively (Fig. ...
Context 26
... repeating the above analysis for the L1 subfields of biology and for the L2 subfields of genetics, we uncovered similar differences among fields of study. Namely, we observed the following for subfields in the same hierarchal level: (i) significant variance in the mean number of papers (see Figs S27 and S28), (ii) notable variance in the mean number of authors (see Figs S29 and S30), (iii) noteworthy variance in the mean number of references (see Figs S31 and S32), and (iv) vast variance in median citation numbers (see Figs S33 and S34). ...
Context 27
... have become targets, and now they are no longer good measures. By making papers shorter and collaborating with more authors, researchers are able to produce more papers in the same amount of time. Moreover, we observed that the majority of changes in papers' properties are correlated with papers that receive higher numbers of citations (see Fig. S13). Authors can use longer titles and abstracts, or use question or exclamation marks in titles, to make their papers more appealing. Thus, more readers are attracted to the paper, and ideally they will cite it, i.e., academic clickbait [45]. These results support our hypothesis that the citation number has become a target. Consequently, ...
Context 28
... 25% of all papers with ≥5 references) have no citations at all (see Fig. 9). Obviously, many resources are spent on papers with limited impact. The lack of citations may indicate that researchers are publishing more papers of poorer quality to boost their total number of publications. Additionally, by exploring papers' citation distributions (see Fig. 10), we can observe that different decades have very different citation distributions. This result indicates that comparing citation records of researchers who published papers during different periods can be challenging. Fifth, by exploring trends in authors (see Results of Author Trends section and Figs 11, 12, 13, S14, S15, and S16), ...
Context 29
... in authors (see Results of Author Trends section and Figs 11, 12, 13, S14, S15, and S16), we observed an exponential growth in the number of new researchers who publish papers. We also observed that young career researchers tend to publish considerably more than researchers in previous generations, using the same time frames for comparison (see Fig. 11). Moreover, young career researchers tend to publish their work much more in conferences in the beginning of their careers than older researchers did in previous decades (see Fig. S15). We also observed that young career researchers tend to collaborate considerably more in the beginning of their careers than those who are older (see ...
Context 30
... observed that young career researchers tend to publish considerably more than researchers in previous generations, using the same time frames for comparison (see Fig. 11). Moreover, young career researchers tend to publish their work much more in conferences in the beginning of their careers than older researchers did in previous decades (see Fig. S15). We also observed that young career researchers tend to collaborate considerably more in the beginning of their careers than those who are older (see Fig. 12). Furthermore, we see that the mean percentage of researchers as first authors early in their career is considerably less than those in previous generations (see Fig. 13). In ...
Context 31
... (see Fig. 11). Moreover, young career researchers tend to publish their work much more in conferences in the beginning of their careers than older researchers did in previous decades (see Fig. S15). We also observed that young career researchers tend to collaborate considerably more in the beginning of their careers than those who are older (see Fig. 12). Furthermore, we see that the mean percentage of researchers as first authors early in their career is considerably less than those in previous generations (see Fig. 13). In addition, authors' median sequence numbers typically increase over time, and the rate is typically faster for young career researchers (see Fig. S16). These ...
Context 32
... decades (see Fig. S15). We also observed that young career researchers tend to collaborate considerably more in the beginning of their careers than those who are older (see Fig. 12). Furthermore, we see that the mean percentage of researchers as first authors early in their career is considerably less than those in previous generations (see Fig. 13). In addition, authors' median sequence numbers typically increase over time, and the rate is typically faster for young career researchers (see Fig. S16). These results emphasize the changes in academia in recent years. In a culture of "publish or perish," researchers publish more by increasing collaboration (and being added to more ...
Context 33
... those who are older (see Fig. 12). Furthermore, we see that the mean percentage of researchers as first authors early in their career is considerably less than those in previous generations (see Fig. 13). In addition, authors' median sequence numbers typically increase over time, and the rate is typically faster for young career researchers (see Fig. S16). These results emphasize the changes in academia in recent years. In a culture of "publish or perish," researchers publish more by increasing collaboration (and being added to more author lists) and by publishing more conference papers than in the past. However, as can be observed by the overall decline of researchers as first ...
Context 34
... by analyzing journal trends (see Results of Journal Trends section), we see a rapid increase in the number of ranked active journals in recent years (see Fig. 14). Moreover, on average, journals publish more papers than in the past, and dozens of journals publish >1,000 papers each year (see Figs 14 and S17). With the increase in the number of active journals, we observed rapid changes in impact measures: (i) the number of papers published in the first and second quartiles (Q1 and Q2) has ...
Context 35
... by analyzing journal trends (see Results of Journal Trends section), we see a rapid increase in the number of ranked active journals in recent years (see Fig. 14). Moreover, on average, journals publish more papers than in the past, and dozens of journals publish >1,000 papers each year (see Figs 14 and S17). With the increase in the number of active journals, we observed rapid changes in impact measures: (i) the number of papers published in the first and second quartiles (Q1 and Q2) has increased sharply, and today the vast majority of papers are published in these quartiles (see Fig. 15); (ii) the journals' mean and median h-index have decreased sharply (see Fig. S18); and (iii) both the SJR and the mean number of citations have increased considerably (see Figs 16 and S20). ...
Context 36
... of journals publish >1,000 papers each year (see Figs 14 and S17). With the increase in the number of active journals, we observed rapid changes in impact measures: (i) the number of papers published in the first and second quartiles (Q1 and Q2) has increased sharply, and today the vast majority of papers are published in these quartiles (see Fig. 15); (ii) the journals' mean and median h-index have decreased sharply (see Fig. S18); and (iii) both the SJR and the mean number of citations have increased considerably (see Figs 16 and S20). With these substantial changes, it is clear that some measures, such as the use of quartiles and the h-index, are rapidly losing meaning and ...
Context 37
... in the number of active journals, we observed rapid changes in impact measures: (i) the number of papers published in the first and second quartiles (Q1 and Q2) has increased sharply, and today the vast majority of papers are published in these quartiles (see Fig. 15); (ii) the journals' mean and median h-index have decreased sharply (see Fig. S18); and (iii) both the SJR and the mean number of citations have increased considerably (see Figs 16 and S20). With these substantial changes, it is clear that some measures, such as the use of quartiles and the h-index, are rapidly losing meaning and value. Moreover, with the abundance of journals, researchers can "shop around" for a ...
Context 38
... on average, journals publish more papers than in the past, and dozens of journals publish >1,000 papers each year (see Figs 14 and S17). With the increase in the number of active journals, we observed rapid changes in impact measures: (i) the number of papers published in the first and second quartiles (Q1 and Q2) has increased sharply, and today the vast majority of papers are published in these quartiles (see Fig. 15); (ii) the journals' mean and median h-index have decreased sharply (see Fig. S18); and (iii) both the SJR and the mean number of citations have increased considerably (see Figs 16 and S20). With these substantial changes, it is clear that some measures, such as the use of quartiles and the h-index, are rapidly losing meaning and value. ...
Context 39
... some fields have hundreds of thousands of papers published yearly, others have only thousands published yearly (see Figs 20 and S22). Moreover, similar large differences are reflected in other examined fields' features, such as the mean number of references and the mean and median citation numbers (see Figs 21 and S35). ...
Context 40
... we have revealed in this study, to measure impact with citation-based measures-that have now become targetsclearly has many undesirable effects. The number of papers with limited impact has increased sharply (see Fig. S11), papers may contain hundreds of self-citations (see Fig. 7), and some top journals have become "old boys' clubs" that mainly publish papers from the same researchers (see Figs 17 and 18). Moreover, using citation-based measures to compare researchers in different fields may have the dangerous effect of allocating more resources to ...
Context 41
... we have revealed in this study, to measure impact with citation-based measures-that have now become targetsclearly has many undesirable effects. The number of papers with limited impact has increased sharply (see Fig. S11), papers may contain hundreds of self-citations (see Fig. 7), and some top journals have become "old boys' clubs" that mainly publish papers from the same researchers (see Figs 17 and 18). Moreover, using citation-based measures to compare researchers in different fields may have the dangerous effect of allocating more resources to high-citation domains, shortchanging other domains that are equally important. ...
Context 42
... complex features of the specified object in a straightforward manner. For example, with only a few and relatively simple lines of Python code, we were able to calculate the mean number of co-authors per author in a specific year for authors who started their career in a specific decade. An overview of our code framework is presented in Fig. ...
Similar publications
Introduction:
Our goal in this study was to determine female representation on editorial boards of high-ranking emergency medicine (EM) journals. In addition, we examined factors associated with gender disparity, including board members' academic rank, departmental leadership position, h-index, total publications, total citations, and total publis...
Objectives:
To analyse the coverage and main bibliometric indicators of Insights into Imaging in Scopus and the Emerging Sources Citation Index (ESCI) from the Web of Science Core Collection database.
Methods:
The evolution of journal production in the Scopus database was analysed according to document types, collaboration indexes between author...
Приведены статистические (наукометрические) данные публикационной активности журнала «Динамика окружающей среды и глобальные изменения климата» (ДОСиГИК). Кроме того, сделана попытка проанализировать, какие материалы оказались наиболее полезны читателям. В дополнение к ранее вышедшему обзору за 10 лет проведена оценка работы журнала за последние пя...
Braun et al. (2006) recommended using the h-index as an alternative to the journal impact factor (IF) to qualify journals. In this paper, a Bayesian-based sensitivity analysis is performed with the aid of mathematical models to examine the behavior of the journal h-index to changes in the publication/citation counts of journals. Sensitivity of the...
Yiqin Alicia Shen, Jason M. Webster, Yuichi Shoda, and Ione Fine Department of Psychology, University of Washington Past research has demonstrated an under-representation of female editors and reviewers in top scientific journals, but less is known about the representation of women authors within original research articles. We collected research ar...
Citations
... Evaluating scientific impact necessitates precise measurement of individual articles' impact, which is commonly assessed through metrics reliant on citation counts [1,2]. However, these metrics are subject to limitations, notably susceptibility to manipulation within the scholarly community [3]. Recently, there has been a shift towards utilizing knowledge distilled from citation graphs rather than relying solely on citation counts [4]. ...
This paper explores methods for building a comprehensive citation graph using big data techniques to evaluate scientific impact more accurately. Traditional citation metrics have limitations, and this work investigates merging large citation datasets to create a more accurate picture. Challenges of big data, like inconsistent data formats and lack of unique identifiers, are addressed through deduplication efforts, resulting in a streamlined and reliable merged dataset with over 119 million records and 1.4 billion citations. We demonstrate that merging large citation datasets builds a more accurate citation graph facilitating a more robust evaluation of scientific impact.
... Agentic LLM based systems have shown potential of outperforming PhD researchers and postdocs on short-horizon scientific tasks like question answering, summarization and contradiction detection in various domains (Skarlinski et al., 2024; Asai et al., 2024). These advancements have spurred new opportunities of LLMs accelerating scientific discovery, which is essential given the exponential growth of scientific publications (Landhuis, 2016;Fire and Guestrin, 2019). ...
The rapid advancement in capabilities of large language models (LLMs) raises a pivotal question: How can LLMs accelerate scientific discovery? This work tackles the crucial first stage of research, generating novel hypotheses. While recent work on automated hypothesis generation focuses on multi-agent frameworks and extending test-time compute, none of the approaches effectively incorporate transparency and steerability through a synergistic Human-in-the-loop (HITL) approach. To address this gap, we introduce IRIS: Interactive Research Ideation System, an open-source platform designed for researchers to leverage LLM-assisted scientific ideation. IRIS incorporates innovative features to enhance ideation, including adaptive test-time compute expansion via Monte Carlo Tree Search (MCTS), fine-grained feedback mechanism, and query-based literature synthesis. Designed to empower researchers with greater control and insight throughout the ideation process. We additionally conduct a user study with researchers across diverse disciplines, validating the effectiveness of our system in enhancing ideation. We open-source our code at https://github.com/Anikethh/IRIS-Interactive-Research-Ideation-System
... godine varirao od 2,28 u političkim do 5,39 u medicinskim naukama; razlike u prosečnom broju citata, u mnogim humanističkim disciplinama prosečan članak skupi manje od jednog citata za deset godina, u poređenju sa više od 40 u nekim biomedicinskim poljima; razlike u prosečnom broju referenci po članku itd. (Aksnes, Langfeldt, and Wouters 2019;Fire and Guestrin 2019). Jedan od razloga za ove nesrazmere je to što scijentometrija ima problem sa sveobuhvatnošću i tačnošću izvora podataka koje analizira. ...
... Prosečan broj citata ne može da bude validna predstava stvarnosti u kojoj je mali broj visoko citiranih članaka a, recimo, čak 72 posto članaka objavljenih 2009. godine nisu bili citirani ni posle pet godina (Fire and Guestrin 2019). U takvim okolnostima, nekoliko visoko citiranih članaka mogu da podignu rang časopisa (Bornmann and Leydesdorff 2014), dajući vrednost naučnog uticaja radovima koji će biti citirani retko ili nikada. ...
... Onda politika kvantifikacije može da se pravda, ne time što brojevi tačno opisuju društvo, već time što u njemu podstiču promene koje mere. Međutim, prema Gudhartovom zakonu, jednom kada mera postane meta, ona više nije upotrebljiva kao mera -upravo zato što joj ljudi prilagođavaju svoje ponašanje (Fire and Guestrin 2019). Kada "dobri brojeviˮ postanu zamena za društvene vrednosti i uslov opstanka, to stvara posticaje za manipulativno ponašanje s brojevima koje neumitno kruni integritet merene aktivnosti. ...
This paper is a critique of scientometrics as both a meta-science and a scientific policy within the context of the ongoing methodological crisis in science. Scientometrics emerged as a quantitative social ‘science of science’, under the assumption that citations reflect the value scientists attribute to others’ work and that their quantity serves as an objective measure of ‘scientific quality’. However, during the ‘replication crisis’ it became apparent that many widely cited statistical studies from various scientific fields could not be replicated; that the prevailing research practice of statistical hypothesis testing is riddled with methodological flaws and abuses; and also that falsified research continues to be cited unabated even after unsuccessful replication attempts. These issues prove that scientometrics cannot be justified as a science that objectively describes scientific value. Earlier critics argued that using quantification for the purpose of evaluating articles across different disciplines implies abstracting their content, and thus, the citation motives, which otherwise vary empirically. However, when it was introduced into science policy, scientometrics transformed into an applied science that could ignore these theoretical problems because the purpose of citation analysis was not the objective evaluation of scientific work. Starting from the fact that the spreading of the competitive market logic across all aspects of society is the essence of neoliberal political projects, this paper argues that scientometric indicators in science policy serve as a quasi-scientific technology for simulating and legitimizing competition among academic entities. In this way, they stimulate rational behavior with scarce resources among actors in the process of producing scientific ‘goods’ for the academic ‘market of ideas’. As a form of neoliberal management of society through numbers, evaluative scientometrics falls prey to Goodhart's Law. When ‘good numbers’ are set as the goal of an activity, actors focus on them at the expense of the integrity of the original activity. Scientists change and adapt their behavior, including the interpretation and application of methodological standards, to meet quantitative criteria and ensure their professional existence. This explains the current methodological and confidence crisis in science.
... Cabe señalar, a este respecto, que el año 2023 se cerró con un récord de artículos científicos retractados, más de 10.000, y algunos expertos en integridad científica piensan que eso es simplemente "la punta del iceberg" (Van Noorden, 2023a). De forma paralela a como se ha incrementado exponencialmente el número de trabajos científicos que se han publicado en las últimas décadas (Fire y Guestrin, 2019), la preocupación por la veracidad de la información contenida en algunos de ellos ha aumentado (Van Noorden, 2023b). Es por ello por lo que algunas instituciones, y la comunidad científica en general, están reclamando una mayor transparencia derivada de los hallazgos científicos que favorezca la toma de decisiones en el contexto de la gestión de políticas públicas (Batet, 2022;Editorial Nature, 2022). ...
Existen trabajos de naturaleza Internacional sobre las prácticas cuestionables de investigación, pero estos no son numerosos y ninguno de ellos ha abordado esta problemática en la comunidad científica hispanohablante. Este estudio explora estas prácticas en este sector de la comunidad científica. Una muestra plurinacional (19 países) de 1254 participantes respondió a una encuesta electrónica en la que se planteaban 28 prácticas cuestionables de investigación. Los resultados muestran que la citación coercitiva, las prácticas cuestionables de liderazgo, omitir la lectura completa de los artículos científicos y la inclusión injustificada de personas en las listas de autoría son las prácticas más prevalentes en este estrato de la comunidad científica. Además, las motivaciones extrínsecas relacionadas con la promoción social, económica o laboral son las que más se asocian con la realización de estas prácticas cuestionables. Este estudio puede ser de utilidad frente al desarrollo de estrategias que minimicen este tipo de comportamientos.
... At this point, one might also refer to Goodhart's Law, which suggests that 'when a measure becomes a target, it ceases to be a good measure.' This notion has been extensively discussed in the context of academia (e.g.,Fire and Guestrin, 2019). ...
Entrepreneurial approaches and privatisation practices have been widely embraced by academic and professional leadership teams in UK universities, arguably to ensure that the existing chasm between universities and society is bridged. Departments specialising in economics and business have transformed into mechanisms for disseminating knowledge reconfigured to meet the social and economic demands of the contemporary ‘entrepreneurial’ university. This paper, through a comprehensive review of the extant literature, argues that the entrepreneurial practices and performance-driven metrics adopted by UK universities have largely suppressed academic pluralism, theoretical development, and heterodox thinking. We are of the view that market practices, in conjunction with managerial-type approaches aimed at satisfying specific institutional and individual performance metrics, raise ethical concerns that undermine the established role of academia. The preservation of the university’s traditional role as an institution that promotes intellectual inquiry and pluralism, seeking factual and new knowledge by cultivating virtues and creativity, requires renunciation of the current model, which has transformed universities into ‘businesses’, and academics into ‘entrepreneurs’. Several alternative propositions are offered which, if considered, may help restore the sacrosanct role of the university as an institution of paideia.
... Так, в 2019 году количество документов, индексируемых Google Scholar превысило 389 млн [1], а по состоянию на февраль 2023 года в Scopus было проиндексировано более 90 млн публикаций 1 . Важной особенностью является ускорение этого роста: если за 1980 год было опубликовано в совокупности около одного миллиона статей, то к концу 2010-х число ежегодно публикуемых статей превысило семь миллионов [2]. ...
The exponential growth in scientific publications has heightened the need for robust tools to organize and retrieve research effectively. The Universal Decimal Classification (UDC) serves as a valuable framework for categorizing articles by subject area. However, manual assignment of UDC codes is often prone to inaccuracies or oversimplification, limiting its utility. In this study, we present a novel approach for the automated assignment of UDC codes to scientific articles using BERT-based models. Our methodology was trained and evaluated on a dataset comprising over 19,000 articles in mathematics and related disciplines. To address the hierarchical structure of UDC, we developed two specialized evaluation metrics: hierarchical classification accuracy and hierarchical recommendation accuracy. We also explored multiple strategies for flattening hierarchical labels. Our results demonstrated a hierarchical recommendation accuracy of 0.8220. Furthermore, blind expert evaluation revealed that discrepancies between reference and predicted labels often stem from errors in the original UDC code assignments by article authors. Our approach demonstrates strong potential for automating the classification of scientific articles and can be extended to other hierarchical classification systems.
... The aim of this literature review was to identify new aids to improve fire search-andrescue operations for firefighters. The author used Scopus as the main database due to its high-quality selection of articles and journals [11]. ...
This study aimed to shorten firefighter search times during indoor fires, allowing more people to be rescued, by enhancing disaster-prevention capabilities using building technologies. In indoor fires, fatalities are often caused by the failure of firefighters to rescue individuals in a timely manner. The question of how to effectively increase the probability of survival while waiting for rescue behind closed doors warrants in-depth research and analysis. Therefore, to ensure that people live in safe environments, there is an urgent need to develop a building door panel material with an emergency call function to prevent such incidents from occurring. Utilizing the PRISMA method, we conducted a comprehensive review of the existing literature to identify the key issues and limitations associated with the current search-and-rescue techniques. Subsequently, the identified primary factors were analyzed using the TRIZ method to determine the key factors that influence the success of rescuing trapped individuals, and a notification system was designed to address this issue. Based on the premise that it is advisable to wait for rescue during a fire, we utilized a smartphone to scan a QR code and transmit the exact location information to the fire department. Through extensive participation and feedback from firefighters, we developed a rescue notification door panel and obtained a patent for it. This system can significantly reduce the time required for search-and-rescue operations in fire incidents. The experimental results show a reduction of one-third in search times.
... Between 1900 and 2015, the number of scientists and their papers rapidly increased (Dong et al. 2017). In 2014, the number of scientific journals with a peer-review system exceeded 34,000, the number of published papers surpassed 7,000,000 (Fire and Guestrin 2019), and it continues to grow as the volume of scientific literature tends to double roughly every twelve years (Price 1986). As it is impossible to read millions of papers, one person cannot be up-to-date even with last year's research results. ...
This paper presents an easy-to-use and layperson-friendly general demarcation for recognising scientific claims. The research was inspired by Gerd Gigerenzer’s approach to heuristics, which consists of a few yes or no questions and various ideas from social epistemology, such as Longino’s venues, Goldman’s notion of cognitive expertise, Anderson’s hierarchy of experts, and Lackey’s concept of acquiring knowledge through others’ testimony. The whole procedure is divided into two tasks. The first establishes whether a given testimony source S is an expert by checking S’s h-index on the Google Scholar search engine. The second is to search Google Scholar for any documented evidence (e.g., reports and textbooks) that proves that there is a consensus among experts that S’s claim is correct.
... Advances in technology have made it possible to access information quickly, and the amount of accessible information has increased daily and exponentially (Fire & Guestrin, 2019;X. Hu et al., 2020;Ioannidis et al., 2018). ...
... Every year, as mentioned above, the number of articles increases exponentially compared to the previous years (Fire & Guestrin, 2019;X. Hu et al., 2020;Ioannidis et al., 2018). ...
In academic performance assessments, citations and citation-based metrics play a pivotal role. Among the elements that influence the success of an article, the title stands out as the first point of contact for editors, reviewers, and readers. Consequently, the title length-citation relationship is an extensively studied issue. While the dominant language used in the databases is English, rhetorical discourses are dominant in the non-English literature about the appropriate length of titles. This study investigates the relationships between title lengths, article downloads, and citations for educational research articles in the Turkish context. Specifically, it focuses on articles indexed in TRIndex and Google Scholar that were published in the Turkish Journal of National Education between 2010 and 2015. The parametric and nonparametric tests using different approaches (median and quarterly classification, correlation analysis) reveal a low negative correlation between title lengths and citations and downloads, a moderate positive correlation between downloads and citations, and a high positive correlation between the citations in the databases. It has been observed that the articles with short titles are read and cited more than the ones with longer titles. These findings align with previous research conducted in the English language. The results suggest that the optimal title length for educational research coincides with “the magical number seven, plus or minus two,” often referred to as Miller’s Law, which represents the number of objects an individual can retain in their short-term memory. The study provides practical implications for authors, editors, and researchers on optimizing title length for better visibility and impact.
... One reason behind using these measures is the fierce competition that scientists face when competing for limited resources such as grants, laboratory space, tenure, and the brightest graduate students 10 , the allocation of which requires a measure of performance 11 . In this process, bibliometric data such as citation counts have become an indispensable component of scientific evaluation for at least the past four decades 5 , and has become one of the primary metrics which scientists strive to maximize 12 . While some may consider such a competitive environment to be a force for good 13 , others argue that it creates perverse incentives for scientists to "game the system", with many resorting to various methods to boost their citation counts, e.g., via self-citations 14,15 , citation cartels [16][17][18][19] , and coercive citations [20][21][22][23] . ...
Citations are widely considered in scientists’ evaluation. As such, scientists may be incentivized to inflate their citation counts. While previous literature has examined self-citations and citation cartels, it remains unclear whether scientists can purchase citations. Here, we compile a dataset of ~1.6 million profiles on Google Scholar to examine instances of citation fraud on the platform. We survey faculty at highly-ranked universities, and confirm that Google Scholar is widely used when evaluating scientists. We then engage with a citation-boosting service, and manage to purchase 50 citations while assuming the identity of a fictional author. Taken as a whole, our findings bring to light new forms of citation manipulation, and emphasize the need to look beyond citation counts.
