No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Research Grants: Conform and Be Funded
Abstract
Too many US authors of the most innovative and influential papers in the
life sciences do not receive NIH funding, contend Joshua M. Nicholson
and John P. A. Ioannidis.
... At the same time, some scholars have expressed concerns that an undue focus on grants may contribute to unintended negative consequences, such as a decreased scholarly creativity and research quality (Gallup & Svare, 2016;Ioannidis, 2016;Lilienfeld, 2017;Thyer, 2011). For instance, several critics have argued that the "grant culture" encourages conformity given that proposed research projects that are perceived to be too novel or risky may be less likely than other projects to be funded (Nicholson & Ioannidis, 2012). Others contend that the relentless chase of grant dollars inadvertently incentivizes questionable research practices, such as p-hacking and HARKing (hypothesizing after results are known), given the pressure to obtain positive results. ...
... In one study, approximately 60% of USA-based biomedical investigators who were sole, first, or last (senior) authors on widely cited articles -those cited 1000 or more times -were not principal investigators on grants from the National Institutes of Health (NIH). Conversely, although a random sample of 100 NIH study section members held current NIH funding, only 1 of them had ever been sole, first, or last author on a paper cited 1000 or more times (Nicholson & Ioannidis, 2012). A more recent analysis revealed that only 36% of United Kingdom (U.K.) biomedical scientists who had published widely cited (again, 1000 or more citations) articles currently held grants from any of three major U.K. medical funders (Stavropoulou et al., 2019). ...
... First, to what extent have eminent psychologists received public or private funding for what they regarded as their most significant published empirical article contribution? Previous research suggests that a sizeable minority of eminent scholars across disciplines did not receive large grants directly pertaining to their landmark publication (Tatsioni et al., 2010) or were not first, sole, or last author on a major federal grant (Nicholson & Ioannidis, 2012). These results raise the possibility that a sizeable minority of eminent psychologists received no external funding for their most important/prestigious article (see also Stavropoulou et al., 2019). ...
The role that grant funding should play in the conduct and evaluation of psychological science is controversial, largely because few data are available to directly inform this issue. We sought to gain insights on grant funding in psychological science by examining the extent to which grant funding influenced eminent psychologists in their research pursuits. Participants were recruited from a compiled, published list of the most eminent psychologists of the modern era (N = 108). Participants were asked about their history of grant funding, including whether their most significant empirical publication received grant funding; participants were also asked about the perceived impact of grant funding on their scholarly productivity and creativity. Virtually all participants had career histories of external funding and reported that they received external funding for their most important published study. Whereas a small minority reported that funding had hindered their creativity, most reported that funding facilitated both their scholarly productivity and creativity. Open-ended comments revealed more nuance to these findings – a slim majority reported that grant funding positively impacted their scholarship whereas an important minority reported that grant funding negatively impacted their scholarship. In sum, our results indicate that grant funding is an important, albeit fallible, marker of eminence in psychological science. Still, even among successful, eminent psychologists, perceptions of grant funding are not invariably positive. Future research is needed to clarify the extent to which grant funding is biased in favor of eminence and to examine the necessity of grant funding for influential psychological research.
... Many scholars and practitioners engaged in strategy and innovation research have identified a number of biases in project evaluation and selection processes (Boudreau et al. 2016, Li 2017, Tomkins et al. 2017, Pier et al. 2018, particularly in a direction that favors conservative or risk-averse projects (Galasso and Simcoe 2011, Hirshleifer et al. 2012, Nicholson and Ioannidis 2012, Boudreau et al. 2016, Criscuolo et al. 2017. Although biases in project selection are not entirely surprising, given the challenge of identifying the long-run potential of complex and uncertain ideas with high failure rates (Arrow 2011, Azoulay and Li 2020, Lane et al. 2021, Scott et al. 2020, it is critical to identify the sources of potential biases due to the trillions of dollars spent each year on funding new projects (Li and Agha 2015). ...
... The relationship between information sharing and conservative selections has potentially economy-wide implications. Expert evaluation panels are used across economic domains, from academic science (Pier et al. 2018) to boards (Zhu 2013) to industrial research and development (Criscuolo et al. 2017), and stakeholders often perceive that evaluators prefer conservative ideas (Nicholson and Ioannidis 2012). However, hitherto, the connection between evaluation format and outcomes had been unclear. ...
... A persistent concern in the evaluation and selection of novel projects is the tendency of decision makers to favor conservative projects (Nicholson and Ioannidis 2012, Wagner and Alexander 2013, Criscuolo et al. 2017. Here, we take conservative projects to broadly mean those with few weaknesses ("safe" projects) rather than the best balance of strengths to weaknesses (for example "high risk, high reward" projects). ...
The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant-funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects, resulting in 761 proposal-evaluation pairs and more than $250,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change their initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own rather than raise them after seeing more favorable scores. Qualitative coding of the evaluators’ justifications for score changes reveals that exposures to lower scores were associated with greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with increased emphasis on nonevaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure rather than maximizing success.
This paper was accepted by Alfonso Gambardella, business strategy.
... Of course, as the authors discussed, previous supporting work in each author's line of research may have been funded and indirectly contributed to the Nobel Prize-winning (Ahead of Print) signature paper (Tatsioni et al., 2010). Nicholson and Ioannidis (2012) analyzed a random sample of 200 papers, with at least 1,000 citations, in the life and health sciences, and reported that 60% of the authors did not currently have NIH funding as principal investigators (PIs). Nicholson and Ioannidis (2012) concluded that, because the NIH's mission is to fund "the best science, by the best scientists," this goal was not being met. ...
... Nicholson and Ioannidis (2012) analyzed a random sample of 200 papers, with at least 1,000 citations, in the life and health sciences, and reported that 60% of the authors did not currently have NIH funding as principal investigators (PIs). Nicholson and Ioannidis (2012) concluded that, because the NIH's mission is to fund "the best science, by the best scientists," this goal was not being met. Unencumbered by what funding bodies want researchers to study, nonfunded scientists can exert their academic freedom, and as experts in their discipline, determine what is important to study. ...
... One of the strongest findings was that a number of granting agencies stifle innovative and creative research (Guthrie, Ghiga, & Wooding, 2017). Other researchers have made similar points in various commentaries (Carlson, 2008;Lilienfeld, 2018;Nicholson & Ioannidis, 2012;Peck, 2008). This finding suggests that creative research that might buck the trend of "normal science" and contribute to a potential paradigm shift (Kuhn, 1962) is likely to come from unfunded research. ...
Grants play a major role in higher education, including kinesiology. However, critical commentaries on the role of external funds appear nonexistent in kinesiology. Therefore, the purpose of this paper is to outline the most common criticisms of grants to stimulate a conversation in kinesiology. First, I first discuss benefits of grants. Second, I examine the role of grants in higher education. Third, I discuss how external funds are not required to contribute meaningful research. Fourth, I examine how a major reason for grants, to produce research publications, often goes weakly filled. Fifth, I show how the development of grant applications (especially unsuccessful applications) are an inefficient expenditure of resources. Sixth, I discuss how pursuing grants can be detrimental to other important academy goals. Seventh, I examine how grants may negatively influence faculty and administrator morale and quality of life. Eighth, I report on some common criticisms of the grant review process and discuss some alternative reviewing systems. Finally, I end with a brief summary and some recommendations.
... Thus it is not easy to effectively evaluate the value of the cited documents. The development of natural language processing technology has greatly encouraged research on citation context analysis [1][2][3][4][5][6] . The recognition of citation sentiment embodied in citation context has become an important research topic in citation context analysis. ...
... The template reflects the co-occurrence of aspect words and the polarity of evaluation words, and ISP/NOP + ISA pattern. S 4 Word + part-of-speech; aspect word + polarity of evaluation word; the person of subject ...
... On the basis of P 1 , negative words are added to reflect their effect on polarity classification. P 4 Word + part-of-speech; aspect word + polarity of evaluation word + scope of negation On the basis of P 1 , the feature of negation scope is added to reflect its influence on polarity classification and the ISP/ISW + ISE + ISA pattern. P 5 Word + part-of-speech; aspect word + polarity of evaluation word + scope of negation; adversative word ...
... Analysts can typically observe the winning projects and papers but not the ones the institutions rejected. The several exceptions to this pattern consist of studies of funding competitions, and many conclude that the funders tend to favor conservative projects (10,(23)(24)(25) and scientists (26). For example, in one study biomedical scientists applying for seed grants with very novel ideas were less likely to get funding than those with moderately novel ideas, despite having similar track records (27). ...
... Contemporary research on scientific funding competitions provides some support for Planck's view, showing that they tend to select against novelty (16,(24)(25)(26)(27). Yet, if novelty is widely disincentivized, why do researchers pursue novel work at all? ...
There are long-standing concerns that peer review, which is foundational to scientific institutions like journals and funding agencies, favors conservative ideas over novel ones. We investigate the association between novelty and the acceptance of manuscripts submitted to a large sample of scientific journals. The data cover 20,538 manuscripts submitted between 2013 and 2018 to the journals Cell and Cell Reports and 6,785 manuscripts submitted in 2018 to 47 journals published by the Institute of Physics Publishing . Following previous work that found that a balance of novel and conventional ideas predicts citation impact, we measure the novelty and conventionality of manuscripts by the atypicality of combinations of journals in their reference lists, taking the 90th percentile most atypical combination as “novelty” and the 50th percentile as “conventionality.” We find that higher novelty is consistently associated with higher acceptance; submissions in the top novelty quintile are 6.5 percentage points more likely than bottom quintile ones to get accepted. Higher conventionality is also associated with acceptance (+16.3% top–bottom quintile difference). Disagreement among peer reviewers was not systematically related to submission novelty or conventionality, and editors select strongly for novelty even conditional on reviewers' recommendations (+7.0% top–bottom quintile difference). Manuscripts exhibiting higher novelty were more highly cited. Overall, the findings suggest that journal peer review favors novel research that is well situated in the existing literature, incentivizing exploration in science and challenging the view that peer review is inherently antinovelty.
... However, other work has suggested that among scientists with extremely cited papers (those with >1000 citations), a large share are not funded by NIH (5). That analysis focused on extremely cited papers; however, it would be useful to obtain evidence on a large, comprehensive sample of highly-cited scientists considering their entire career impact as well as their recent citation impact. ...
... However, it is likely that there is genuine imbalance in funding that has been driven by the funding agency leadership choices, the ability of some subfields to make stronger claims for the support of their work, the existence (or perceived existence) of major opportunities for discoveries, and the fact that study section members tend to prefer funding work that they feel more familiar with. 5 The extremely low rates of funding of some subfields focused on psychology, cognitive sciences, gender studies, legal and forensic medicine, and human factors is problematic, because these fields may also have difficulty to attract other funding (e.g. from industry). What may emerge may be a strengthening of some disciplines, while others (with crucial potential contributions) are abandoned. ...
Both citation and funding metrics converge in shaping current perceptions of academic success. We aimed to evaluate what proportion of the most-cited USA-based biomedical scientists are funded by biomedical federal agencies and whether funded scientists are more cited than not funded ones. We linked a Scopus-based database on top-cited researchers (n=75,316 USA-based) and the NIH RePORTER database of 33 biomedical federal agencies (n=204,603 grant records) with matching based on name and institution. The 40,887 USA-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. The proportion of USA-based top-cited biomedical scientists (based on career-long citation impact) who had received any federal funding from biomedical research agencies was 63% for any funding (1996-2022), 21% for recent funding (2015-2022), and 14% for current funding (2021-2022). Respective proportions were 65%, 31%, and 21%, when top-cited scientists based on recent single year impact were considered. There was large variability across scientific subfields. No subfield had more than 31% of its top-cited USA-based scientists (career-long impact) currently funded. Funded top-cited researchers were overall more cited than non-funded top-cited scientists, e.g. mean (median) 14,420 (8983) versus 8,445 (4613) (p<0.001) and a substantial difference remained (, after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields. Overall, biomedical federal funding has offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists have current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.
... However, there is a long-standing concern, and growing evidence, that the conditions of mainstream funding schemes have adverse effects on (some) researchers' creativity and, thus, on the long-term potential for scientific breakthroughs. Several studies have empirically examined this concern by means of bibliometric indicators of impact, novelty, and disruption (Nicholson and Ioannidis 2012;Wang, Veugelers, and Stephan 2017;Wang, Lee, and Walsh 2018;Wu, Wang, and Evans 2019) and by analyzing the peer-reviewed selection of grant applications (Lane et al. 2021;Luukkonen 2012). They argue most public and private funders employ similar selection criteria that favor relatively safe, conventional research at the expense of risky, novel research. ...
... Critics question the process of deciding which projects are deemed promising and deserving of a concentration of resources. This process is accused of being conservative and risk-averse, rewarding projects with a preliminary proof-of-concept and an overly rationalistic research plan with step-by-step milestones and pre-specified outcomes (Nicholson and Ioannidis 2012;Wang, Veugelers, and Stephan 2017). Based on a large, mixed-method project, Heinze et al. (2009) conclude: ...
Increasingly, the conditions of mainstream competitive research funding shape the conduct of research in ways that stifle researchers' creativity with adverse effects on their job satisfaction and retention and, thus, on the potential for scientific breakthroughs. However, alternative grants are emerging. In this article, we interviewed 24 recipients of such a grant, namely the Danish 'Villum Experiment' (VE), which seeks to promote unorthodox research within the natural and technical sciences. We unravel how researchers feel about the conditions of grants and their reflections on how these conditions shape their research. Our interviewees consistently juxtaposed their experiences of doing research funded by mainstream grants with that of doing their VE projects. To them, mainstream funding is characterized by conservatism, control, and path-dependency, whereas VE facilitates creativity, flexibility, and scientific joy. Conceptually, we outline a 'two modes of science' model: The industrial mode vis-à-vis the entrepreneurial mode of science. By contextualizing our empirical analysis in this model, we show that given the prevalence of funding, anchored in and promoting the industrial mode of science, alternatives, like VE, become a genuine welcome escape from the production line of big science. In emotional terms, the interviewees express that VE fosters a different research experience and that which initially led them to become scientists: playfulness, exploration and passion. Therefore, we propose that funders start taking researchers' emotions seriously, because how the conditions of grants make scientists feel likely shape the research that they do and, consequently, the progress of science.
... Thus, once a practice for originality emerges in an inbred community, it can be sustained over generations. In contrast, in a more open career system that exposes junior academics to career competition, academics pursuing originality can be penalised by mainstream peers (Azoulay et al., 2011;Nicholson & Ioannidis, 2012;Wang et al., 2017), and thus, the socialisation for originality may be compromised. ...
... Such a system has been considered desirable as it is transparent and seems to facilitate early independence (MEXT, 2009). Nonetheless, studies also suggests a disadvantage in pursuing originality, which can be penalised by mainstream peers (Azoulay et al., 2011;Nicholson & Ioannidis, 2012;Wang et al., 2017). This can be aggravated when career decisions involve broader members, who collectively prioritise mainstream high-performers to niche and unorthodox visionaries. ...
en Producing original knowledge is the foundation of scientific progress. Originality is associated with certain skills and practices that can be trained and socialised. This study investigates how inbreeding as a career practice influences the socialisation of originality. An analysis based on a sample of mid‐career life scientists in Japan finds that originality and associated practices are transferred to junior academics from their PhD supervisors, and that the inter‐generational transfer of the practices favourable for originality is reinforced when junior academics are inbred. Hence, if senior academics have orientation towards originality, inbred junior academics are likely to succeed the same orientation; whereas if supervisor lack orientation towards originality, inbred juniors also lack the orientation. Thus, inbreeding can be a double‐edged sword in developing originality.
概要
ja 独創的知識の生成は科学進歩の源泉である。「独創性」に要するスキルや習慣はトレーニングや社会化を通じて獲得可能であるが、本研究では独創性の社会化に対してインブリーディングが及ぼす影響を検討した。日本の生命科学研究者を対象とした分析の結果、独創性とそれに関わる習慣は博士課程の指導教官からその学生へと移転されること、この移転はインブリーディングによって強化されることが示唆された。即ち、独創性の高い指導教官の下でインブリーディングされた学生は高い独創性を獲得し、独創性の低い指導教官の下でインブリーディングされた学生は低い独創性を獲得することが示唆された。
... Public support for edge science, even if failure is likely, can help establish appropriate incentives for novel work in biomedical science. However, when setting its priorities, the NIH considers many factors, including scientific opportunity, disease burden, and availability of private funding (13,14), and many have questioned the NIH's ability to fund groundbreaking work and innovative science in particular (15)(16)(17)(18)(19)(20)(21)(22). Both scientific and political considerations may lead the NIH to underfund the trying out of new ideas. ...
... The NIH faces pressure to deliver short-term successes that can be at odds with a systematic commitment to edge science. A particular temptation is to focus funding priorities based on the likelihood of producing high-impact publications (as measured by citations) without regard for whether the work represents novel or more traditional science (15)(16)(17)(18)(19)(20)(21)(22). But a focus on high-impact science alone runs the risk of undermining more explorative work that often fails, in the sense that it is not rewarded with many citations, but which lays the foundation for breakthroughs that arrive later as the ideas mature. ...
The National Institutes of Health (NIH) plays a critical role in funding scientific endeavors in biomedicine. Funding innovative science is an essential element of the NIH’s mission, but many have questioned the NIH’s ability to fulfill this aim. Based on an analysis of a comprehensive corpus of published biomedical research articles, we measure whether the NIH succeeds in funding work with novel ideas, which we term edge science. We find that edge science is more often NIH funded than less novel science, but with a delay. Papers that build on very recent ideas are NIH funded less often than are papers that build on ideas that have had a chance to mature for at least 7 y. We have three further findings. First, the tendency to fund edge science is mostly limited to basic science. Papers that build on novel clinical ideas are not more often NIH funded than are papers that build on well-established clinical knowledge. Second, novel papers tend to be NIH funded more often because there are more NIH-funded papers in innovative areas of investigation, rather than because the NIH funds innovative papers within research areas. Third, the NIH’s tendency to have funded papers that build on the most recent advances has declined over time. In this regard, NIH funding has become more conservative despite initiatives to increase funding for innovative projects. Given our focus on published papers, the results reflect both the funding preferences of the NIH and the composition of the applications it receives.
... No wonder, they are "recruited," for the most part, from ordinary researchers apt to "rectify" knowledge rather than do innovatory works, let alone breakthrough discoveries. Less than 1 percent of them (72 out of 8517) have ever published a very influential paper, with 1000 or more citations (Nicholson & Ioannidis, 2012;34 -35). Though, even the abundance of citations a research has gathered does not evidence exactly its high value or quality. ...
The paper debunks the deep-rooted belief that proper appraisal of research is a job within reach of the average scientist. The main qualities of an individual expert that allow him to reveal the value of a research are profound knowledge of the subject, broad scientific outlook, and ability for conceptual linking. As the same person rarely combines them in full measure, judgments pretty often turn out to be imprecise and superficial. The scientific community as a whole is surely capable of redressing the defects and weaknesses of its individual members, but copes with the task quite satisfactorily chiefly in case of “standard” research with a small degree of novelty, while with pioneering works, it is mostly mistaken. Collective appraisals are bounded by the conceptual horizon of the age they are given in, the institutional involvement of those making them on its behalf, and the unification of attitudes that is conditioned by the “communal” nature of science as a social institution. That is why the scientific community virtually never attaches high value to breakthrough discoveries forthwith. And without transformation of the existing practice of “measuring” the value of research, this bias can scarcely be overcome.
... Large enough data sets, on the other hand, show that there are criteria independent of individual article quality that can influence an average citation rate. With that stated, the citation count has been proposed as a major criterion for assessing the impact of research, which ultimately dictates the distribution of resources by funding organizations (35). However, it is still debatable whether citation frequency is important when allocating resources (36). ...
Background and aim:
Von Willebrand disease (VWD) is considered the most prevalent inherited bleeding disorder. The current study aims to demonstrate the research status and trends on VWD worldwide.
Methods:
Bibliometric analysis was used to investigate the global research productivity and trends on VWD. The publications on VWD from 1956 to 2021 were extracted using the Web of Science database. In the VWD domain, a total of 3,643 records were analyzed for authorship and collaboration patterns, yearly productivity, highly cited documents, relevant source of publication, most prolific scholars, productive countries, and organizations.
Results:
The most productive journal, author, organization, and country were 'Haemophilia' with 439 publications, 'Favaloro EJ' with 119 publications, the 'University of Milan' with 192 publications, and the United States of America (USA) with 1,048 publications, respectively. The document with the highest citations was 'Srivastava A, 2013, Haemophilia,' which received 1,154 citations in total. In 2016, the highest number of publications shared by two author patterns was 28. With 199 publications, the year 2021 remained on the top, while the citation-wise analysis identified 2006 as the top year with 5,379 citations.
Conclusions:
Research productivity and publication trends on VWD revealed that the USA emerged as the most significant contributing country. The 'University of Milan' was the most significant contributing organization, while 'Favaloro EJ' was the most significant author. 'Hemophilia' was found to be the most significant journal in the field of VWD. It is recommended that researchers from countries with significant contributions to the field should collaborate with researchers from Asian countries and other countries that lack behind in research in the domain of VWD.(www.actabiomedica.it).
... Whether a grant applicant's number of publications is a useful barometer of future research impact is unclear. Highly influential scientists may not be funded when publication count is a key metric [17]. Alternative approaches include funding all applicants, funding applicants randomly, automated approaches, rewarding citizenship, or alternative rubrics for evaluation of promising science [18]. ...
Background
Publications may be a modifiable factor toward research project grant (RPG) funding decisions, the objective was to determine the association of publication record with later RPG receipt.
Methods
This was a retrospective cohort study of recipients of K01, K08, or K23 US career development awards (CDAs) starting from 2000–2015. Exposures were CDA awardees’ first-, middle-, and last-author publication counts, and the quartile of awardees’ highest and mean publication impact factors. The independent association of each exposure with time to RPG (R01 or equivalent) was determined using a Cox model, after adjustment for CDA type, awardee change in institution, and institutional CDA count. The proportion of CDA recipients with later independent funding was also determined by publication count.
Results
Among 6744 CDA awardees, 3943 obtained an RPG. The median time to RPG was 5.6 years (interquartile range 4.2–7.5). The number of first-authorships was associated with a shorter time to RPG (1–4 versus 0: hazard ratio [HR] 1.22, 95% confidence interval [CI] 1.10–1.36; 5–9: 1.59, 95% CI 1.40–1.79; 10–24: 1.78, 95% CI 1.54–2.07; 25+: 2.40, 95% CI 1.61–3.56). Last-authorships were associated with a shorter time to RPG (1–4 versus 0: HR 1.99, 95% CI 1.83–2.16; 5–9: 2.72, 95% CI 2.45–3.03; 10–24: 3.17, 95% CI 2.78–3.62; 25+: 3.12, 95% CI 2.17–4.50). Higher maximum impact factor was associated with a shorter time to RPG (Q2 versus lowest: HR 1.28, 95% CI 1.12–1.46; Q3: 1.45, 95% CI 1.24–1.70; Q4: 1.67, 95% CI 1.39–2.02). Mean impact factor and middle-authorships were not associated with time to RPG. Among 687 CDAs with zero associated first- or last-authorships, 158 (23%) achieved later RPG funding. Among those with at least 10 total first- or last-authorships, 1288/1554 (83%) obtained a later RPG.
Conclusions
A higher number and impact of publications was associated with later independent funding.
... Developed into the familiar, citation-centred form through arduous manual extraction by Garfield and Price in the mid-20th century (2,3), its indicators have proliferated in the Internet age. They now dominate the research landscape, routinely informing major funding decisions and academic staff recruitment worldwide (4)(5)(6)(7)(8). ...
The value of biomedical research—a $1.7 trillion annual investment—is ultimately determined by its downstream, real-world impact, whose predictability from simple citation metrics remains unquantified. Here we sought to determine the comparative predictability of future real-world translation—as indexed by inclusion in patents, guidelines, or policy documents—from complex models of title/abstract-level content versus citations and metadata alone. We quantify predictive performance out of sample, ahead of time, across major domains, using the entire corpus of biomedical research captured by Microsoft Academic Graph from 1990–2019, encompassing 43.3 million papers. We show that citations are only moderately predictive of translational impact. In contrast, high-dimensional models of titles, abstracts, and metadata exhibit high fidelity (area under the receiver operating curve [AUROC] > 0.9), generalize across time and domain, and transfer to recognizing papers of Nobel laureates. We argue that content-based impact models are superior to conventional, citation-based measures and sustain a stronger evidence-based claim to the objective measurement of translational potential.
... However, realistically facing the situation is a prerequisite for improvement. Pretending that everything is fine will not move the field forward 156 , nor will conforming and producing more similar findings 157 . ...
Mental disorders represent a worldwide public health concern. Psychotherapies and pharmacotherapies are recommended as first line treatments. However, evidence has emerged that their efficacy may be overestimated, due to a variety of shortcomings in clinical trials (e.g., publication bias, weak control conditions such as waiting list). We performed an umbrella review of recent meta-analyses of randomized controlled trials (RCTs) of psychotherapies and pharmacotherapies for the main mental disorders in adults. We selected meta-analyses that formally assessed risk of bias or quality of studies, excluded weak comparators, and used effect sizes for target symptoms as primary outcome. We searched PubMed and PsycINFO and individual records of the Cochrane Library for meta-analyses published between January 2014 and March 2021 comparing psychotherapies or pharmacotherapies with placebo or treatment-as-usual (TAU), or psychotherapies vs. pharmacotherapies head-to-head, or the combination of psychotherapy with pharmacotherapy to either monotherapy. One hundred and two meta-analyses, encompassing 3,782 RCTs and 650,514 patients, were included, covering depressive disorders, anxiety disorders, post-traumatic stress disorder, obsessive-compulsive disorder, somatoform disorders, eating disorders, attention-deficit/hyperactivity disorder, substance use disorders, insomnia, schizophrenia spectrum disorders, and bipolar disorder. Across disorders and treatments, the majority of effect sizes for target symptoms were small. A random effect meta-analytic evaluation of the effect sizes reported by the largest meta-analyses per disorder yielded a standardized mean difference (SMD) of 0.34 (95% CI: 0.26-0.42) for psychotherapies and 0.36 (95% CI: 0.32-0.41) for pharmacotherapies compared with placebo or TAU. The SMD for head-to-head comparisons of psychotherapies vs. pharmacotherapies was 0.11 (95% CI: –0.05 to 0.26). The SMD for the combined treatment compared with either monotherapy was 0.31 (95% CI: 0.19-0.44). Risk of bias was often high. After more than half a century of research, thousands of RCTs and millions of invested funds, the effect sizes of psychotherapies and pharmacotherapies for mental disorders are limited, suggesting a ceiling effect for treatment research as presently conducted. A paradigm shift in research seems to be required to achieve further progress.
... That being said, citation count has been suggested as a major criterion to assess the influence of research, which eventually guides the allocation of resources by funding agencies. 29 Although it remains a matter of debate, citation frequency remains a crucial factor in allocating funds. 30 ...
This paper aims to examine the worldwide research development and trends in forensic anthropology by using bibliometric analysis. Scopus database was used to identify all published papers on forensic anthropology from establishment until the year 2020. A total of 4,499 records were analyzed for yearly publication productivity, authorship and citation pattern, types of documents, most productive journals, organizations, authors, and countries, frequently cited papers, most used keywords, countries of collaboration, and three-field plot analysis in the domain of forensic anthropology. The results indicated that the highest grand total researches were between the years 2016 and 2019, while the highest multi-authored publications were in the year 2018. The most productive journal, author, organization, and country were the ‘Journal of Forensic Sciences’ with grand total publications of 934, ‘Cattaneo, C.’ with 97 publications, ‘CNRS Centre National de la Recherche Scientifique’ with 130 publications, and the United States with 1020 total cited papers, respectively. The document with the highest citation rate was ‘Buckberry JL, 2002, Am J Phys Anthropol’ with a total citation of 387. Three-field plot analysis regarding the most outstanding keyword-source- country was “Forensic anthropology”- “Journal of Forensic Sciences” and “Forensic Science International”- “USA”, “China”, “France”, and “UK”. Moreover, the predomination of certain countries over others in the field of forensic anthropology limits its prosperity as ethnic variety is of important regard. Research collaborations were mainly observed between the United States and European countries, which highlights the need for strengthening collaborations between developed and developing nations.
... The NIH is the largest funder of US biomedical research and receiving NIH funding is associated with higher publica-tion productivity. 22,23 A study by Svider et al concluded that female ophthalmologists received lower NIH awards than their male ophthalmologists. 12 Contrary to this finding, our study found that oculoplastic surgeons do not follow the same trends as the entire field of ophthalmology. ...
Purpose The aim of the study is to investigate sex differences in academic rank, publication productivity, and National Institute of Health (NIH) funding among oculoplastic surgeons and whether there is an association between American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) membership and scholarly output.
Methods Sex, residency graduation year, and academic rank were obtained from institutional websites of 113 U.S. ophthalmology programs. H-indices and m-quotients were obtained from the Scopus database. NIH funding information was obtained from the NIH Research Portfolio Online Reporting Tool.
Results Of the 272 surgeons, 74 (30.2%) were females. When adjusted for career duration, differences in female to male proportions were only significant at the rank of assistant professor (assistant: 74.3 vs. 48.5%, p = 0.047; associate: 18.9 vs. 24.6%, p = 0.243; full professor: 13.0 vs. 37.2%, p = 0.114). Women had a shorter career duration than men [10.0 (interquartile range or IQR 12.0) vs. 21.0 (IQR 20.0) years; p < 0.001] and a lower h-index [4.0 (IQR 5.0) vs. 7.0 (IQR 10.0); p < 0.001], but similar m-quotients [0.4 (IQR 0.4) vs. 0.4 (IQR 0.4); p = 0.9890]. Among ASOPRS members, females had a lower h-index than males [5.0 (IQR 6.0) vs. 9.0 (IQR 10.0); p < 0.001] due to career length differences. No difference in productivity between sexes was found among non-ASOPRS members. ASOPRS members from both sexes had higher scholarly output than their non-ASOPRS counterparts. Just 2.7% (2/74) of females compared with 5.3% (9/171) of males received NIH funding (p = 0.681).
Conclusion Sex differences in academic ranks and h-indices are likely due to the smaller proportion of females with long career durations. ASOPRS membership may confer opportunities for increased scholarly output.
... Other scholars put the onerous on peer review that takes place in funding agencies, arguing that it focuses excessively on feasibility and things that can go wrong at the expense of supporting research that has the potential to lead to major breakthroughs (Azoulay and Li 2020;Criscuolo et al. 2017;Franzoni, Stephan, and Veugelers 2021;Heinze 2008;Nicholson and Ioannidis 2012;OECD 2018;Petsko 2011;Wagner and Alexander 2013). The result is a selection of projects that, despite ensuring good average quality (Li 2017;Park, Lee, and Kim 2015), disfavors novel approaches (Boudreau et al. 2014;Lane et al. 2021), with the potential of high value (Azoulay, Graff Zivin, and Manso 2012). ...
... Although this move reflected greater overall alignment with Kevin's personal goals for teaching, research, and service (Ibarra, 1999), there were key parts of those expectations, primarily related to grantspersonship, with which Kevin did not initially agree. He feared that he would be tempted to adjust the focus of his research and move away from studying teachers' lives and careers to begin asking questions that were more aligned with funders' goals (Nicholson & Ioannidis, 2012). This helps to illustrate the complexity of socialization and transitionary experiences as faculty members need to learn contextual norms in new environments (Zambroski & Freeman, 2004). ...
Purpose: Scholars, including those in physical education, have investigated the socialization of higher education faculty members. Informed by self-study of teacher education practices and occupational socialization theory, we aimed to understand Kevin’s experiences during a transition from one institution to another with the help of his critical friend, Jim. Methodology/Methods: Data were collected through prolonged journaling and critical friend discussions. Resulting text files were analyzed thematically with a focus on identifying turning points. Findings: Themes developed through qualitative analysis included: (a) readjusting scholarly targets and embracing grantspersonship, (b) giving up control and facilitating research, and (c) balancing being a team player with self-advocacy. Discussion/Conclusions: Kevin’s transition was supported by recognizing shifting norms of his new faculty role and influence of self-study of teacher education practices for ongoing, career professional development. Prior and current socialization influences framed this development.
... However, with its long-term impact and limited short-term recognition , novel research could struggle to ensure the necessary funds for its success. As suggested by Nicholson and Ioannidis (2012), it seems that the rule in science funding is closer to "Conform and be funded" than to push for the boldest proposals. Similarly, Stephan et al. (2017) express their concerns on the existence of a bias against novelty in science. ...
Public funding agencies aim to fund novel breakthrough research to promote the radical scientific discoveries of tomorrow. Identifying the profiles of scientists being financed to pursue their research is therefore crucial. This paper shows that the funding process is not always awarding the most novel scientists. Exploiting rich data on all applications to a leading Swiss research funding program, we find that novel scientists have a higher probability of applying for funds than non-novel scientists, but they get on average lower ratings by grant evaluators and have fewer chances of being funded. We discuss the implications for the allocation of scientific research spending.
... More recently, controversy regarding the NIH peer review process has centered on the possibility of racial (Ginther et al., 2011) and gender (Bornmann, Mutz, & Daniel, 2007) biases, as well as influences based on interpersonal relationships (Wessely, 2007). The peer-review process has also been criticized for encouraging conformity and discouraging innovation (Nicholson & Ioannidis, 2012). These criticisms regarding the fairness of the grant review process have led to calls for alternative systems for grant allocation, such as a modified lottery system (Fang & Casadevall, 2016) or a shift to funding investigators rather than projects (Kaiser, 2014;Germain, 2015). ...
Abstract
Biomedical research over the last century in the United States has resulted in new drugs, medical procedures, and medical devices that have dramatically increased the health and lifespan of individuals and global populations. However, after decades of growth, the federal appropriation to fund grants for biomedical research has decreased since 2002. Thus, the pressure on early-career faculty to obtain federally funded grants to perform biomedical research has increased due to competition for limited grant resources. Therefore, it was important to understand their perceptions about research funding, and the effect on achieving their career goals. First, a review of the literature was provided. The literature review included studies involving analysis of factors influencing the productivity of early-career faculty and provided a theoretical framework for interpreting the data. Next, an overview of the research methods was provided. The challenges facing early-career faculty were studied through a qualitative examination of their perspectives and experiences. This study used a qualitative case study research design at a single institution. Tenure-track and non-tenure-track faculty members at the rank of Assistant Professor were selected by purposeful sampling for participation in the study. They were interviewed using a semi-structured interview protocol. The interview data were presented as well as the major themes that emerged from the analysis of the data. Finally, a discussion of the data was provided in the context of the literature review and theoretical frameworks. The findings were aligned with three theoretical frameworks that were used to understand the results, namely Self-Determination Theory, Academic Capitalism, and Bourdieu’s Theory of Practice. The findings identified the importance of mentoring in learning how to write grants effectively. The findings also revealed that the competition for limited grant dollars has contributed to heightened stress and anxiety among the participants. In conclusion, the findings of this study could provide useful information helpful to both faculty and academic administrators. Because of this study, higher education leaders have useful data that support the importance of an environment that is conducive to successful grant-seeking strategies.
... These results open the doorway to the possibility that principles, intuitions, assumptions, and/or axioms on which analytical analysis in philosophy are based might be influenced, unwanted and unexpected, by the predispositions of the philosopher. Our tendency to conformity (e.g., Ioannidis & Nicholson, 2012) might be the reason this has escaped our attention until it was flagged by an external arbiter (e.g., empirical tests). Just like regular human beings, scientists and philosophers suffer from psychological biases (e.g., Alger, 2019;Pohl, 2004). ...
The content of this dissertation spans four years of work, which was carried out in the Netherlands (Tilburg University and University of Amsterdam) and Italy (University of Turin). It is part of the ERC project “Making Scientific Inference More Objective” led by professor Jan Sprenger, for which philosophy of science and empirical research were combined. The dissertation can be summarized as a small set of modest attempts to contribute to improving scientific practice. Each of these attempts was geared towards either increasing understanding of a particular problem or making a contribution to how science can be practiced. The general focus was on philosophical nuance while remaining methodologically practicable. The five papers contained in this dissertation are both methodologically and philosophically diverse. The first three (Chapters 2 through 4) are more empirical in nature and are focused on understanding and evaluating how science is practiced: a meta-analysis of semantic intuitions research in experimental philosophy; a systematic review on essay literature on the null hypothesis significance test; and an experiment on how teams of statisticians analyze the same data. The last two (Chapters 5 and 6) are focused on the improvement of scientific practice by providing tools for the improvement of empirical research with a strong philosophical foundation: a practicable and testable definition of scientific objectivity and a Bayesian operationalization of Popper’s concept of a severe test.
... As Adam Eyre-Walker and Nina Stoletzki have shown, scientists are poor at evaluating whether a paper will go on to have significant effects (2013). Several studies have also shown that work that went on to achieve high recognition and status could equally well have been rejected at earlier or later parts of significance-based peer-review processes and fared poorly as a benchmarking measure of career success (Azoulay, Zivin, and Manso 2011;Calcagno et al. 2012;Campanario 2009Campanario , 1993Campanario , 1996Campanario and Acedo 2007;Ceci and Peters 1982;Costello 2010;Fang, Bowen, and Casadevall 2016;Gans and Shepherd 1994;Lindner and Nakamura 2015;Meng 2016;Nicholson and Ioannidis 2012;Pagano 2006;Siler, Lee, and Bero 2015;Weller 2001). While there is some evidence that reviewers find greater consensus in distinguishing non-acceptable from acceptable work (that is, reviewers find agreement in work that should not be published) (Cicchetti 1991;Weller 2001), debates around the roles of novelty and significance in gatekeeping processes continue to rage and, as noted, appear thoroughly embedded within academic expectations and norms. ...
This Element describes for the first time the database of peer review reports at PLOS ONE, the largest scientific journal in the world, to which the authors had unique access. Specifically, this Element presents the background contexts and histories of peer review, the data-handling sensitivities of this type of research, the typical properties of reports in the journal to which the authors had access, a taxonomy of the reports, and their sentiment arcs. This unique work thereby yields a compelling and unprecedented set of insights into the evolving state of peer review in the twenty-first century, at a crucial political moment for the transformation of science. It also, though, presents a study in radicalism and the ways in which PLOS's vision for science can be said to have effected change in the ultra-conservative contemporary university. This title is also available as Open Access on Cambridge Core.
... Yaqub found that it is possible to classify serendipity into four basic types [65] and that there may be important factors a®ecting its occurrence. His conclusions seem to agree with ideas developed in earlier works [66][67][68][69][70][71] which argue that the commonly adopted À À À apparently meritocratic À À À strategies, which pursuit excellence and drive out variety, seem destined to be loosing and ine±cient. The reason is that they cut out a priori researches that initially appear less promising but that, thanks also to serendipity, could be extremely innovative a posteriori. ...
This paper further investigates the Talent versus Luck (TvL) model described by [Pluchino et al. Talent versus luck: The role of randomness in success and failure, Adv. Complex Syst. 21 (2018) 1850014] which models the relationship between ‘talent’ and ‘luck’ on the impact of an individuals career. It is shown that the model is very sensitive to both random sampling and the choice of value for the input parameters. Running the model repeatedly with the same set of input parameters gives a range of output values of over 50% of the mean value. The sensitivity of the inputs of the model is analyzed using a variance-based approach based upon generating Sobol sequences of quasi-random numbers. When using the model to look at the talent associated with an individual who has the maximum capital over a model run it has been shown that the choice for the standard deviation of the talent distribution contributes to 67% of the model variability. When investigating the maximum amount of capital returned by the model the probability of a lucky event at any given epoch has the largest impact on the model, almost three times more than any other individual parameter. Consequently, during the analysis of the model results one must keep in mind the impact that only small changes in the input parameters can have on the model output.
... Indeed, data suggest that some of the most widely scientific papers were initially rejected by journals (Campanario, 1996). This principle of ideological conservatism holds in at least equal force for grant submissions that are too unconventional (Nicholson & Ioannidis, 2012). Furthermore, scholars who routinely buck trends in the field by pursuing offbeat topics or methodological approaches may find it challenging to be hired, tenured, and promoted (Sternberg, 2005). ...
Constructive criticism is essential to scientific progress, and one means of encouraging such criticism is to embrace controversial and otherwise heterodox ideas. I examine several social psychological (e.g., groupthink, derogation of outsiders) and institutional (e.g., citation bias, white hat bias) processes that may promote ideological conformity in psychology, delineate 5 reasons that psychology should more actively embrace unpopular ideas, and I argue that psychology at large has not typically done enough to promote heterodoxy. This Special Section of Archives of Scientific Psychology features 16 articles, each of which challenges 1 or more mainstream psychological assumptions. I hope that it advances constructive discussion and debate regarding a host of scientific questions that have long been regarded as settled.
... Some critics of grant systems argue that there is a systemic bias against innovative projects (Nicholson & Ioannidis, 2012). Based simply on probabilities, a radical or unorthodox proposal is likely to be read by assessors who are closer to the mainstream than the converse. ...
Grants are essential for some scientists to do their research, and receiving them can be a mark of status. Grants are supposed to be awarded on merit, but there are many deviations from this ideal. There are a few publicized cases in which grants to dissident scientists have been blocked. Far more common, though difficult to prove, is routine bias in grant committees toward favored applicants and dominant views and against dissidents and competitors. This sort of bias can reflect altruism toward those with personal connections or ideological affinity with grant givers. Most grant systems serve to orient researchers to the agendas of government and industry. This is a systemic process independent of biases against individuals or topics.
... Grant applications are commonly centred on the assessment of a significant amount of preliminary data, which obviously favours established investigators and bigger laboratories (Alberts et al. 2014;McDowell et al. 2014;Daniels 2015). Moreover, there is evidence of bias in favour of the 'insiders and the familiar' over the 'unknown' (Daniels 2015;Fang and Casadevall 2009;Kirwan Institute 2014;Nicholson and Ioannidis 2012) and against unconventional and unorthodox ideas, which are more likely to be proposed by newly established scientists. The latter are also less likely to be involved in large research programmes, which can be favoured over small ones by 'inertia' and 'financial dependency' biases (Alberts et al. 2014). ...
... Second, it has been shown that when investigators had similar measures of productivity, their chance of success increased if the funding panel contained a member of the same institution [7]. Third, applications that are similar to the interests of the reviewers are often favored [8] and yet reviewed more critically [5], so they may be viewed as more important and also given more useful feedback due to the extent of expertise in the area. Put simply, reliance on scientists alone often results in 'echo-chamber' science, sacrificing novelty and innovation. ...
Abstract People with lived experience are individuals who have first-hand experience of the medical condition(s) being considered. The value of including the viewpoints of people with lived experience in health policy, health care, and health care and systems research has been recognized at many levels, including by funding agencies. However, there is little guidance or established best practices on how to include non-academic reviewers in the grant review process. Here we describe our approach to the inclusion of people with lived experience in every stage of the grant review process. After a budget was created for a specific call, a steering committee was created. This group included researchers, people with lived experience, and health systems administrators. This group developed and issued the call. After receiving proposals, stage one was scientific review by researchers. Grants were ranked by this score and a short list then reviewed by people with lived experience as stage two. Finally, for stage three, the Steering Committee convened and achieved consensus based on information drawn from stages one and two. Our approach to engage people with lived experience in the grant review process was positively reviewed by everyone involved, as it allowed for patient perspectives to be truly integrated. However, it does lengthen the review process. The proposed model offers further practical insight into including people with lived experience in the review process.
... However, unlike personal photography, travel, retail sales, and many more industries driven by consumer preference, ongoing funding of clinical research depends almost solely on the decision of trial funders, whether grant reviewers or medical industry leaders, who historically tend to support the status quo rather than drive innovation. 13 So, what is needed to overcome existing challenges and drive innovation using digital technologies? As noted earlier, establishing standards and protocols supporting transparency and privacy of participant's data is critical. ...
... Studies addressing the renewed research agenda and the issues listed above need to be properly supported by funding organizations. Decisions on funding from existing public agencies and other funders are often biased toward specific types of inbred research with limited returns, providing just more of the same, for example, funding primarily one form of treatment (Nicholson and Ioannidis, 2012;Lorsch, 2015;MQ, 2015). As advances often spring from unexpected sources, supporting a variety of different (treatment) approaches increases the chance for important discoveries. ...
Toward a paradigm shift in treatment and research of mental disorders - Volume 49 Issue 13 - Falk Leichsenring, Christiane Steinert, John P.A. Ioannidis
Purpose:
To examine the characteristics of pediatric ophthalmology fellowship program directors (FPDs) through cross-sectional analysis.
Methods:
All pediatric ophthalmology FPDs whose programs participated in the San Francisco Match in January 2020 were included. Information was collected through publicly available sources. Scholarly activity was measured by peer-reviewed articles and the Hirsch index.
Results:
Of the 43 FPDs, 22 (51%) were male, and 21 (49%) were female. The mean age of current FPDs was 53.5 ± 8.8 years old. There was a significant difference between the current age of male and female FPDs (57.8 ± 8 vs 49 ± 7.3, resp. [P <0.0001]). The mean term length of female and male FPDs also differed (11.5 ± 4.5 vs 16.1 ± 8.9 [P = 0.042]). Thirty-eight FPDs (88%) attended medical school in the United States. Forty-two FPDs had an MD (98%). Thirty-nine (91%) FPDs completed ophthalmology residency in the United States. Ten (23%) of the FPDs were dual fellowship trained. There was a significantly higher Hirsch-index among male than female FPDs (23.9 ± 15.7 vs 10.3 ± 10.1 [P = 0.0017]). There was a higher number of publications for male (91 ± 89) than female FPDs (31.5 ± 48.6 [P = 0.0099]).
Conclusions:
Pediatric ophthalmology fellowship programs have a uniquely equal representation of male and female FPDs as women continue to be underrepresented in ophthalmology. Female FPDs were younger and had spent less time in the position, suggesting a shift toward more female FPD over time.
Importance
Both citation and funding metrics converge in shaping current perceptions of academic success.
Objective
To evaluate what proportion of the most-cited US-based scientists are funded by biomedical federal agencies and whether funded scientists are more cited than nonfunded ones.
Design, Setting, and Participants
This survey study used linkage of a Scopus-based database on top-cited US researchers (according to a composite citation metric) and the National Institutes of Health RePORTER database of federal funding (33 biomedical federal agencies). Matching was based on name and institution. US-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. Data were downloaded on June 11, 2022.
Main Outcomes and Measures
Proportion of US-based top-cited biomedical scientists who had any (1996-2022), recent (2015-2022), and current (2021-2022) funding. Comparisons of funded and nonfunded scientists assessed total citations and a composite citation index.
Results
There were 204 603 records in RePORTER (1996-2022) and 75 316 US-based top-cited scientists in the career-long citation database; 40 887 scientists were included in the main analysis. The proportion of US-based top-cited biomedical scientists (according to career-long citation impact) who had received any federal funding from biomedical research agencies was 62.7% (25 650 of 40 887) for any funding (1996-2022), 23.1% (9427 of 40 887) for recent funding (2015-2022), and 14.1% (5778 of 40 887) for current funding (2021-2022). Respective proportions were 64.8%, 31.4%, and 20.9%, for top-cited scientists according to recent single-year citation impact. There was large variability across scientific subfields (eg, current funding: 31% of career-long impact top-cited scientists in geriatrics, 30% in bioinformatics and 29% in developmental biology, but 0% in legal and forensic medicine, general psychology and cognitive sciences, and gender studies). Funded top-cited researchers were overall more cited than nonfunded top-cited scientists (median [IQR], 9594 [5650-1703] vs 5352 [3057-9890] citations; P < .001) and substantial difference remained after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields.
Conclusions and Relevance
In this survey study, biomedical federal funding had offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists had current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.
A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.
A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.
Background:
Therapeutic research into Alzheimer's disease (AD) has been dominated by the amyloid cascade hypothesis (ACH) since the 1990s. However, targeting amyloid in AD patients has not yet resulted in highly significant disease-modifying effects. Furthermore, other promising theories of AD etiology exist.
Objective:
We sought to directly investigate whether the ACH still dominates the opinions of researchers working on AD and explore the implications of this question for future directions of research.
Methods:
During 2019, we undertook an international survey promoted with the help of the Alzheimer's Association with questions on theories and treatments of AD. Further efforts to promote a similar study in 2021 did not recruit a significant number of participants.
Results:
173 researchers took part in the 2019 survey, 22% of which held "pro-ACH" opinions, tended to have more publications, were more likely to be male, and over 60. Thus, pro-ACH may now be a minority opinion in the field but is nevertheless the hypothesis on which the most clinical trials are based, suggestive of a representation bias. Popular vote of all 173 participants suggested that lifestyle treatments and anti-tau drugs were a source of more therapeutic optimism than anti-amyloid treatments.
Conclusion:
We propose a more democratic research structure which increases the likelihood that promising theories are published and funded fairly, promotes a broader scientific view of AD, and reduces the larger community's dependence on a fragile economic model.
Competition for research funds has, in the recent decade, become hypercompetitive. Commonly, to determine which proposals receive funding, a system of peer review is used, which is broadly accepted, easily understood, and broadly trusted among researchers. It is often considered the best system in use, but it suffers from important shortcomings and adaptations to overcome these shortcomings have small and often short-lived effects. Hence, the preference for peer review does not mean it necessarily outperforms all other systems. In fact, it is time for an open discussion about alternative allocation mechanisms. Random allocation of research funding may be a viable alternative to the current peer review system. In particular the “organized randomness” of a modified lottery is interesting, combining the benefits of randomization with some of the most valuable aspects of peer review. Still, many questions remain and this is certainly not a plea to allocate all research funds using lotteries without further research. But we need to be prepared to consider alternatives, even though they are not perfect, and modified lotteries should be part of the solution.
This study examines the relationship between team's gender composition and outputs of funded projects using a large data set of National Institutes of Health (NIH) R01 grants and their associated publications between 1990 and 2017. This study finds that while the women investigators' presence in NIH grants is generally low, higher women investigator presence is on average related to slightly lower number of publications. This study finds empirically that women investigators elect to work in fields in which fewer publications per million‐dollar funding is the norm. For fields where women investigators are relatively well represented, they are as productive as men. The overall lower productivity of women investigators may be attributed to the low representation of women in high productivity fields dominated by men investigators. The findings shed light on possible reasons for gender disparity in grant productivity.
Blue thermally activated delayed fluorescence (TADF) organic light-emitting diodes (OLEDs) with enhanced efficiency and suppressed efficiency roll-offs were fabricated using a mixed host consisting of 3,3-di(9H-carbazol-9-yl)-biphenyl (mCBP) and bis[2-(diphenyl-phosphino)phenyl]ether oxide (DPEPO). Notably, mCBP and DPEPO, which are generally used as hosts in blue TADF OLEDs, exhibit material stability and efficiency as their trade-off characteristics. The mixed-host structure, when applied to 10,10′-(4,4′-sulfonylbis(4,1-phenylene))bis(9,9-dimethyl-9,10-dihydroacridine), which is a blue TADF dopant, facilitated a maximum efficiency of 18.1%. Furthermore, the critical current density (J0), at which the external quantum efficiency decreases by half of its maximum value, was observed to be 3.9 times that of the DPEPO device.
A previous model for a novel system is reinterpreted for the traditional systems of funding allocation. Empirical data can be well described. Both research funding and education funding are analyzed. The effect of merit-based cumulative advantage is more significant in research funding, where a slight difference is noticed between basic sciences and applied sciences. In contrast, the counter effect of cumulative advantage can be observed in education funding. Simple parameters are useful to distinguish between different distributions. The theoretical model presents three distinct regimes: equal sharing, cumulative advantage effect, and counter effect. The regime of equal sharing presents as a valley. Both cumulative advantage effect and counter effect result in the concentration of funding, which present as two plateaus of different heights.
Grants play a major role in higher education, including kinesiology. However, critical commentaries on the role of external funds appear nonexistent in kinesiology. Therefore, the purpose of this paper is to outline the most common criticisms of grants to stimulate a conversation in kinesiology. First, I discuss benefits of grants. Second, I examine the role of grants in higher education. Third, I discuss how external funds are not required to contribute meaningful research. Fourth, I examine how a major reason for grants, to produce research publications, often goes unfullfilled. Fifth, I show how the development of grant applications (especially unsuccessful applications) is an inefficient expenditure of resources. Sixth, I discuss how pursuing grants can be detrimental to other important academy goals. Seventh, I examine how grants may negatively influence faculty and administrator morale and quality of life. Eighth, I report on some common criticisms of the grant review process and discuss some alternative reviewing systems. Finally, I end with a brief summary and some recommendations.
Purpose
To investigate gender disparitie among pediatric ophthalmologists in academic rank, publication productivity, and National Institutes of Health (NIH) funding.
Methods
In this cross-sectional analysis of pediatric ophthalmologists at 113 US academic programs, data on gender, residency graduation year, and academic rank were obtained from institutional websites between January 2019 and March 2019. The Scopus database was used to calculate h-indices and m-quotients. The NIH Research Portfolio Online Reporting Tool was used to determine NIH funding.
Results
We identified 389 pediatric ophthalmologists: 194 women (49.9%) and 195 men (50.1%). A binomial logistic regression model, which included career length as an independent variable, showed proportions of women to men were similar across all academic ranks (assistant professor, 64.4% vs 46.2% [P = 0.738]; associate professor, 21.7% vs 19.0% [P = 0.357]; full professor, 13.9 vs 34.9% [P = 0.119]). Women had a lower median h-index (5.0 vs 8.0 [P = 0.008]) and a shorter median career duration (12.5 vs 25.0 years [P < 0.001]), but a similar median m-quotient (0.5 vs. 0.5; P = 0.525). Among pediatric ophthalmologists who received NIH funding (20 women vs. 27 men; P = 0.826), the overall median grant-funding total for women was $804K (interquartile range (IQR) 5.0M, mean $3.8M) compared to men, $2.2M (IQR, 4.0M; mean, $3.7M; P = 0.328).
Conclusions
The shorter career duration for women likely contributes to the difference in overall h-indices between genders, as m-quotients were similar. The m-quotient should be used over the h-index when comparing academic productivity across genders when disparities in career length exist.
External peer review can be a useful part of a quality improvement program. However, it can also be used for political and punitive purposes. The distinction between potentially useful and damaging review is largely an issue of the ethics of how it is done. The ethics of external peer review lie not only in the process itself, but also in the role of the pathologists performing the review. While there are many ethical issues involved in external peer review, the most important may be a dedication to due process, allowing the pathologist under review to respond to allegations, and an insistence on complete information prior to drawing conclusions. Well established criteria for external peer review may provide protection both for the pathologist under review against allegations of negligence or incompetence, but also for the reviewers against accusations of bad faith.
Purpose:
This study analyzed sex differences among cornea specialists with regards to academic rank, scholarly productivity, National Institute of Health (NIH) funding, and industry partnerships.
Design:
Cross-sectional study.
Methods:
This was a study of faculty at 113 US academic programs. Sex, residency graduation year, and academic rank were collected from institutional websites between January-March 2019. H-indices and m-quotients were collected from the Scopus database. The NIH Research Portfolio Online Reporting Tool and Centers for Medicare and Medicaid Services databases were queried for data on NIH funding and industry partnerships.
Results:
Of the 440 cornea specialists identified, 131 (29.8%) were females. The proportions of females and males at each academic rank (assistant: 69.5% vs. 41.8%; associate: 17.6% vs 21.0%; full professor: 13.0% vs. 37.2%) were not significant after adjusting for career duration (p=0.083, 0.459, 0.113, respectively). Females had significantly lower median h-indices [4.0, (Interquartile range (IQR) 7.0) vs. 11.0, (IQR 17.0); p<0.001] and shorter median career duration [12.0 (IQR 11.0) vs. 25.0 (IQR 20.0) years; p <0.001] than males, but similar median m-quotients (0.5 (IQR 0.8) vs. 0.5 (IQR 0.8); p=1.00). Sex differences in h-indices were not seen at each academic rank or career duration interval. Among NIH-funded investigators, median grant funding of was $1.6M (IQR 2.2M) for females and $1.2M (IQR 4.6M)(p=0.853) for males. Overall, 25.5% of females and 58.6% of males (p=0.600) had industry partnerships.
Conclusion:
Sex differences within academic ranks and h-indices are likely due to a smaller proportion of females with advanced career duration.
Academic training is the initial step for junior scientists to learn to develop into independent scientists. This study investigates how supervisors decide to employ different approaches of early-career research training, and how these approaches influence the degree of trainees' independence in their later careers. Drawing on survey and bibliometric data of life scientists in Japanese universities, this study presents the following findings. First, if scientists are allowed higher autonomy in upstream research functions in early-career training, they later tend to attain greater organizational independence with higher organizational ranks. Second, if scientists are encouraged to deviate from conventional research topics during early-career training, they later tend to achieve greater cognitive independence by producing original research output. Third, the differences in the training approaches chosen by individual supervisors are influenced by the training that they had received in their early-career training. Overall, the study suggests that training approaches and independence of scientists are socialized in the local training context and passed down from one generation to the next.
Cases of scientific fraud and research misconduct in general have escalated in Western higher education over the last 20 years. These practices include forgery, distortion of facts and plagiarism, the outright faking of research results and thriving black markets for positive peer reviews and ghost-written papers. More recently, the same abuses have found their way into Asian higher education with some high profile and widely covered cases in India, South Korea, China and Japan. Reports of misconduct are now reaching alarming proportions in Asia, and the negative consequences for individuals, institutions, governments and society at large are incalculable. The incentives for academic scientists in Asia are approaching and even surpassing those ordinarily seen in the West. Cash payments for publishing articles in high impact journals can double or even triple yearly salaries in some cases. Combining this environment with the simultaneous pressure to obtain oftentimes scarce funding for research has produced a culture of unethical behaviour worldwide. This article assesses three important issues regarding scientific fraud and research misconduct: distorted incentives for research and overreliance upon metrics, damage to the integrity of higher education and public trust and improving research environments so as to deter unethical behaviour. This is especially crucial for emerging Asian countries, in particular Association for Southeast Asian Nations (ASEAN), whose scientific infrastructure is less developed, but nonetheless has the potential to become a major player in the development of psychology as well as Science, Technology, Engineering and Mathematics (STEM) research and training.
Background
In this study, we aimed to assess health research funding allocation in South Korea by analysing the relationship between government funding and disease burden in South Korea, specifically focusing on cancers.
Methods
The relationship between research funding and the cancer burden, measured in disability-adjusted life-years (DALYs), was analysed using a linear regression method over a 10-year interval. Funding information on 25 types of cancer was obtained from the National Science and Technology Information Service portal in South Korea. Measures of cancer burden were obtained from Global Burden of Disease studies. The funding predictions were derived from regression analysis and compared with actual funding allocations. In addition, we evaluated how the funding distribution reflected long-term changes in the burden and the burden specific to South Korea compared with global values.
Results
Korean funding in four periods, 2005–2007, 2008–2010, 2011–2013 and 2015–2017, were associated with the cancer burden in 2003, 2006, 2009 and 2013, respectively. For DALYs, the correlation coefficients were 0.79 and 0.82 in 2003 and 2013, respectively, which were higher than the values from other countries. However, the changes in DALYs (1990–2006) were not associated with the funding changes (from 2005 to 2007 to 2015–2017). In addition, the value differences between Korean and global DALYs were not associated with Korean government research funding.
Conclusions
Although research funding was associated with the cancer burden in South Korea during the last decade, the distribution of research funds did not appropriately reflect the changes in burden nor the differences between the South Korean and global burden levels. The policy-makers involved in health research budgeting should consider not only the absolute burden values for singular years but also the long-term changes in burden and the country-specific burden when they prioritise public research projects.
Conflicting interests among group members are common when making collective decisions, yet failure to achieve consensus can
be costly. Under these circumstances individuals may be susceptible to manipulation by a strongly opinionated, or extremist,
minority. It has previously been argued, for humans and animals, that social groups containing individuals who are uninformed,
or exhibit weak preferences, are particularly vulnerable to such manipulative agents. Here, we use theory and experiment to
demonstrate that, for a wide range of conditions, a strongly opinionated minority can dictate group choice, but the presence
of uninformed individuals spontaneously inhibits this process, returning control to the numerical majority. Our results emphasize
the role of uninformed individuals in achieving democratic consensus amid internal group conflict and informational constraints.
Summary The current system of publication in biomedical research provides a distorted view of the reality of scientific data that are generated in the laboratory and clinic. This system can be studied by applying principles from the field of economics. The "winner's curse," a more general statement of publication bias, suggests that the small proportion of results chosen for publication are unrepresentative of scientists' repeated samplings of the real world. The self-correcting mechanism in science is retarded by the extreme imbalance between the abundance of supply (the output of basic science laboratories and clinical investigations) and the increasingly limited venues for publication (journals with sufficiently high impact). This system would be expected intrinsically to lead to the misallocation of resources. The scarcity of available outlets is artificial, based on the costs of printing in an electronic age and a belief that selectivity is equivalent to quality. Science is subject to great uncertainty: we cannot be confident now which efforts will ultimately yield worthwhile achievements. However, the current system abdicates to a small number of intermediates an authoritative prescience to anticipate a highly unpredictable future. In considering society's expectations and our own goals as scientists, we believe that there is a moral imperative to reconsider how scientific data are judged and disseminated.
Growing demand to quantify the research output from public funding has tempted funding agencies, promotion committees and employers to treat numerical indices of research output more seriously. So many assessment exercises are now conducted worldwide that traditional peer assessment is threatened. Here, we describe a new citation-based index (Hirsh's h index) and examine several factors that might influence it for ecologists and evolutionary biologists, such as gender, country of residence, subdiscipline and total publication output. We suggest that h is not obviously superior to other indices that rely on citations and publication counts to assess research performance.
Funding agencies (and journals) seem to be discriminating against ideas that are contrary to the mainstream, leading to leading to the preferential funding of predictable and safe research over radically new ideas. To remedy this problem a restructuring of the scientific funding system is needed, e.g. by utilizing laymen--together with scientists--to evaluate grant proposals.
John P. A. Ioannidis proposes ways to save scientists from spending all their time writing grants.
Funding is important for scientists' work and may contribute to exceptional research outcomes. We analyzed the funding sources reported in the landmark scientific papers of Nobel Prize winners. Between 2000 and 2008, 70 Nobel laureates won recognition in medicine, physics, and chemistry. Sixty five (70%) of the 93 selected papers related to the Nobel-awarded work reported some funding source including U.S. government sources in 53 (82%), non-U.S. government sources in 19 (29%), and nongovernment sources in 33 (51%). A substantial portion of this exceptional work was unfunded. We contacted Nobel laureates whose landmark papers reported no funding. Thirteen Nobel laureates responded and offered their insights about the funding process and difficulties inherent in funding. Overall, very diverse sources amounting to a total of 64 different listed sponsors supported Nobel-related work. A few public institutions, in particular the U.S. National Institutes of Health (with n=26 funded papers) and the National Science Foundation (with n=17 papers), stood out for their successful record for funding exceptional research. However, Nobel-level work arose even from completely unfunded research, especially when institutions offered a protected environment for dedicated scientists.