Standardizing the Evaluation of Scientific and Academic Performance in Neurosurgery-Critical Review of the "h" Index and its Variants

Department of Neurological Surgery, Feinberg School of Medicine and McGaw Medical Center, Northwestern University, Chicago, Illinois, USA.
World Neurosurgery (Impact Factor: 2.88). 01/2012; 80(5). DOI: 10.1016/j.wneu.2012.01.052
Source: PubMed


Assessing the academic impact and output of scientists and physicians is essential to the academic promotion process and has largely depended on peer review. The inherent subjectivity of peer review, however, has led to an interest to incorporate objective measures into more established methods of academic assessment and promotion. Journal impact factor has been used to add objectivity to the process but this index alone does not capture all aspects of academic impact and achievement. The "h" index and its variants have been designed to compensate for these shortcomings, and have been successfully used in the fields of physics, mathematics, and biology, and more recently in medicine. Leaders in academic neurosurgery should be aware of the advantages offered by each of these indices, as well as of their individual shortcomings, to be able to efficiently use them to refine the peer-review process. This review critically analyzes indices that are currently available to evaluate the academic impact of scientists and physicians. These indices include the total citation count, the total number of papers, the impact factor, as well as the "h" index with eight of its most common variants. The analysis focuses on their use in the field of academic neurosurgery, and discusses means to implement them in current review processes.

14 Reads
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Research-oriented cancer hospitals in the United States treat and study patients with a range of diseases. Measures of disease specific research productivity, and comparison to overall productivity, are currently lacking. Different institutions are specialized in research of particular diseases. To report disease specific productivity of American cancer hospitals, and propose a summary measure. We conducted a retrospective observational survey of the 50 highest ranked cancer hospitals in the 2013 US News and World Report rankings. We performed an automated search of PubMed and for published reports and registrations of clinical trials (respectively) addressing specific cancers between 2008 and 2013. We calculated the summed impact factor for the publications. We generated a summary measure of productivity based on the number of Phase II clinical trials registered and the impact factor of Phase II clinical trials published for each institution and disease pair. We generated rankings based on this summary measure. We identified 6076 registered trials and 6516 published trials with a combined impact factor of 44280.4, involving 32 different diseases over the 50 institutions. Using a summary measure based on registered and published clinical trails, we ranked institutions in specific diseases. As expected, different institutions were highly ranked in disease-specific productivity for different diseases. 43 institutions appeared in the top 10 ranks for at least 1 disease (vs 10 in the overall list), while 6 different institutions were ranked number 1 in at least 1 disease (vs 1 in the overall list). Research productivity varies considerably among the sample. Overall cancer productivity conceals great variation between diseases. Disease specific rankings identify sites of high academic productivity, which may be of interest to physicians, patients and researchers.
    PLoS ONE 03/2015; 10(3):e0121233. DOI:10.1371/journal.pone.0121233 · 3.23 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective assessment of academic productivity is useful for residency programs. This study aims to analyze the number of publications and Hirsch index (h index) among radiation oncology residents. Names of residents during the 2010 academic year (n = 607) were collected from the Association of Residents in Radiation Oncology 2010 Directory. Number of publications and h index from Jan. 1996 to Feb. 2012 were collected from a bibliographic database (SCOPUS, Elsevier, BV, Amsterdam, NL). Analysis of h index included stratification by gender, residency size, and postresidency private practice or academic employment. Six hundred seven residents, 67 % men and 33 % women, had an overall mean h index of 2.5 ± 3.2. Graduates in academia exhibited a higher mean h index (3.9 ± 0.30) compared to private practice (2.0 ± 0.25; p < 0.01). Gender, residency size, and post-graduate position remained correlates of h index (all p ≤ 0.01). Women had lower mean h index and number of publications than men (2.1 ± 2.3 vs 2.7 ± 3.5, 4.5 ± 5.3 vs 6.2 ± 8.0, respectively; both p < 0.05). However, when stratified by current position (resident, private practice, or academic), there were no significant differences in h index by gender. The mean ± SD h indices for institutions comprising the top 10 % ranged 4.17 ± 3.2-5.25 ± 5.4 while the bottom 10 % ranged 0.0 ± 0.0-0.75 ± 1.4. The h index is a useful metric to assess residents' early dedication to scholarly endeavors. Female radiation oncology residents had fewer total publications and slightly lower h indices, warranting accessible research avenues and environments for future female physician-scientists. The application of the h index provides a reference for medical students, residents, residency program directors, and many others to gauge academic performance and establish appropriate benchmarks.
    Journal of Cancer Education 07/2013; 28(3). DOI:10.1007/s13187-013-0500-2 · 1.23 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The relationship between metrics, such as the h-index, and the ability of researchers to generate funding has not been previously investigated in neurosurgery. This study was performed to determine if a correlation exists between bibliometrics and NIH funding data among academic neurosurgeons. The h-index, m-quotient, g-index, and contemporary h-index (hc) were determined for 1,225 academic neurosurgeons from 99 (of 101) departments. Two databases were used to create the citation profiles; Google Scholar (GS) and Scopus. The NIH RePORTER tool was accessed to obtain career grant funding amount, grant number, year of first grant award, and calendar year of grant funding. Of the 1,225 academic neurosurgeons, 182 (15%) had at least one grant with a fully reported NIH award profile. Bibliometric indices were all significantly higher for those with NIH funding compared to those without NIH funding (p<0.001). The contemporary h-index was found to be significantly predictive of NIH funding (p<0.001). All bibliometric indices were significantly associated with the total number of grants, total award amount, year of first grant, and duration of grants in calendar years (bivariate correlation, p<0.001) except for the association of m-quotient with year of first grant (p=0.184). Bibliometric indices are higher for those with NIH funding compared to those without, but only the contemporary h-index was shown to be predictive of NIH funding. Among neurosurgeons with NIH funding, higher bibliometric scores were associated with greater total amount of funding, number of grants, duration of grants and earlier acquisition of their first grant.
    World Neurosurgery 11/2013; 81(3-4). DOI:10.1016/j.wneu.2013.11.013 · 2.88 Impact Factor
Show more