Standardizing the Evaluation of Scientific and Academic Performance in Neurosurgery-Critical Review of the "h" Index and its Variants

Department of Neurological Surgery, Feinberg School of Medicine and McGaw Medical Center, Northwestern University, Chicago, Illinois, USA.
World Neurosurgery (Impact Factor: 2.88). 01/2012; 80(5). DOI: 10.1016/j.wneu.2012.01.052
Source: PubMed


Assessing the academic impact and output of scientists and physicians is essential to the academic promotion process and has largely depended on peer review. The inherent subjectivity of peer review, however, has led to an interest to incorporate objective measures into more established methods of academic assessment and promotion. Journal impact factor has been used to add objectivity to the process but this index alone does not capture all aspects of academic impact and achievement. The "h" index and its variants have been designed to compensate for these shortcomings, and have been successfully used in the fields of physics, mathematics, and biology, and more recently in medicine. Leaders in academic neurosurgery should be aware of the advantages offered by each of these indices, as well as of their individual shortcomings, to be able to efficiently use them to refine the peer-review process. This review critically analyzes indices that are currently available to evaluate the academic impact of scientists and physicians. These indices include the total citation count, the total number of papers, the impact factor, as well as the "h" index with eight of its most common variants. The analysis focuses on their use in the field of academic neurosurgery, and discusses means to implement them in current review processes.

16 Reads
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Research-oriented cancer hospitals in the United States treat and study patients with a range of diseases. Measures of disease specific research productivity, and comparison to overall productivity, are currently lacking. Different institutions are specialized in research of particular diseases. To report disease specific productivity of American cancer hospitals, and propose a summary measure. We conducted a retrospective observational survey of the 50 highest ranked cancer hospitals in the 2013 US News and World Report rankings. We performed an automated search of PubMed and for published reports and registrations of clinical trials (respectively) addressing specific cancers between 2008 and 2013. We calculated the summed impact factor for the publications. We generated a summary measure of productivity based on the number of Phase II clinical trials registered and the impact factor of Phase II clinical trials published for each institution and disease pair. We generated rankings based on this summary measure. We identified 6076 registered trials and 6516 published trials with a combined impact factor of 44280.4, involving 32 different diseases over the 50 institutions. Using a summary measure based on registered and published clinical trails, we ranked institutions in specific diseases. As expected, different institutions were highly ranked in disease-specific productivity for different diseases. 43 institutions appeared in the top 10 ranks for at least 1 disease (vs 10 in the overall list), while 6 different institutions were ranked number 1 in at least 1 disease (vs 1 in the overall list). Research productivity varies considerably among the sample. Overall cancer productivity conceals great variation between diseases. Disease specific rankings identify sites of high academic productivity, which may be of interest to physicians, patients and researchers.
    PLoS ONE 03/2015; 10(3):e0121233. DOI:10.1371/journal.pone.0121233 · 3.23 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objective assessment of academic productivity is useful for residency programs. This study aims to analyze the number of publications and Hirsch index (h index) among radiation oncology residents. Names of residents during the 2010 academic year (n = 607) were collected from the Association of Residents in Radiation Oncology 2010 Directory. Number of publications and h index from Jan. 1996 to Feb. 2012 were collected from a bibliographic database (SCOPUS, Elsevier, BV, Amsterdam, NL). Analysis of h index included stratification by gender, residency size, and postresidency private practice or academic employment. Six hundred seven residents, 67 % men and 33 % women, had an overall mean h index of 2.5 ± 3.2. Graduates in academia exhibited a higher mean h index (3.9 ± 0.30) compared to private practice (2.0 ± 0.25; p < 0.01). Gender, residency size, and post-graduate position remained correlates of h index (all p ≤ 0.01). Women had lower mean h index and number of publications than men (2.1 ± 2.3 vs 2.7 ± 3.5, 4.5 ± 5.3 vs 6.2 ± 8.0, respectively; both p < 0.05). However, when stratified by current position (resident, private practice, or academic), there were no significant differences in h index by gender. The mean ± SD h indices for institutions comprising the top 10 % ranged 4.17 ± 3.2-5.25 ± 5.4 while the bottom 10 % ranged 0.0 ± 0.0-0.75 ± 1.4. The h index is a useful metric to assess residents' early dedication to scholarly endeavors. Female radiation oncology residents had fewer total publications and slightly lower h indices, warranting accessible research avenues and environments for future female physician-scientists. The application of the h index provides a reference for medical students, residents, residency program directors, and many others to gauge academic performance and establish appropriate benchmarks.
    Journal of Cancer Education 07/2013; 28(3). DOI:10.1007/s13187-013-0500-2 · 1.23 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The h-index was introduced as a means of quantifying the contribution a researcher makes to the scientific literature. We evaluated the h-index for academic neurosurgeons to assess the various methods of calculation and to determine whether the h-index can be used to differentiate groups of individuals by various classifications. The h-index was calculated for all neurosurgeons from 10 institutions ranked highly by 2012 US News & World Report plus the authors' institution using Scopus. The h-index was also manually calculated to evaluate accuracy. The average h-index was calculated for groups based on sex, academic rank, years in practice, institution, and subspecialty. Cumulative and mean h-indices were calculated for each department. The median h-index for the 188 neurosurgeons was 16 (mean=19.71; range=0-61). There was a positive association between the h-index, academic rank, and years post-training. There was a significant difference between the "manually calculated" and automated h-indices, particularly for more senior physicians. The difference in h-index between men and women was not statistically significant. Among subspecialties, vascular surgeons had the highest average h-index and general neurosurgeons had the lowest. There were significant shifts in departmental rankings when the cumulative or mean departmental indices were compared with the US News & World Report rankings. Application of the h-index as a bibliometric in neurosurgery can distinguish academic productivity based on academic rank, years post-training, and neurosurgical subspecialties. Application of the h-index to compare departments is problematic and, at this time, not reliable.
    World Neurosurgery 07/2013; 80(6). DOI:10.1016/j.wneu.2013.07.010 · 2.88 Impact Factor
Show more

Similar Publications