Measuring the Impact of Medical Research: Moving From Outputs to Outcomes

Department of Psychiatry, MGH-East, Charlestown, MA 02129, USA.
American Journal of Psychiatry (Impact Factor: 12.3). 03/2007; 164(2):206-14. DOI: 10.1176/appi.ajp.164.2.206
Source: PubMed


Billions of dollars are spent every year to support medical research, with a substantial percentage coming from charitable foundations. To justify these expenditures, some measure of the return on investment would be useful, particularly one aligned with the intended ultimate outcome of this scientific effort: the amelioration of disease. The current mode of reporting on the success of medical research is output based, with an emphasis on measurable productivity. This approach falls short in many respects and may be contributing to the well-described efficacy-effectiveness gap in clinical care. The author argues for an outcomes-based approach and describes the steps involved, using an adaptation of the logic model. A shift in focus to the outcomes of our work would provide our founders with clearer mission-central return-on-investment feedback, would make explicit the benefits of science to an increasingly skeptical public, and would serve as a compass to guide the scientific community in playing a more prominent role in reducing the efficacy-effectiveness gap. While acknowledging the enormous complexity involved with the implementation of this approach on a large scale, the author hopes that this essay will encourage some initial steps toward this aim and stimulate further discussion of this concept.

Full-text preview

Available from:
  • Source
    • "), particularmente en sus hospitales, considerados como centros con una triple función: asistencial, docente e investigadora (Puerta et al, 2011). El programa nace en 1998 con los objetivos de a) incorporar al SNS, a través de contratos temporales, a investigadores con excelente formación para potenciarLa investigación biomédica y en salud (IBS) tiene una evidente relevancia e impacto social (Hanney et al, 2000;Buxton et al, 2004;Weiss, 2007;Patel et al, 2011). En el ámbito concreto de las instituciones que llevan a cabo investigación, o son susceptibles de hacerlo – como es el caso de los centros asistenciales del SNS, a raíz de su citada triple misión –, la investigación puede aportarles beneficios económicos directos y evidentes. "
    [Show abstract] [Hide abstract]
    Full-text · Article · Mar 2015 · Medicina Clínica
  • Source
    • "Academic metrics assess scholarly output rather than research impact. There is further confusion between research impact and research quality, and differentiating between terms such as outcomes, outputs and impact is challenging (Weiss, 2007). Outputs are the direct result of the project and by those directly involved in it, and include both core research activities as well as dissemination activities (Economic and Social Research Council nd). "
    [Show abstract] [Hide abstract]
    ABSTRACT: The real-world impact of research is gaining much attention across the international Higher Education sector. Funding agencies, government organisations and community groups are seeking evidence that research initiatives are delivering impact beyond contributions to academia. Researchers, practitioners, educators, learning designers and developers require a good understanding of research impact, and associated terminology, to articulate the real-world benefits of technology-enabled initiatives. There are three good reasons to understand research impact in a Higher Education context. Firstly, comprehending the language of research impact facilitates meaningful discussion with research stakeholders. Secondly, recognising and communicating the real-world impact of an initiative affirms the 'so what' factor of a research project. And thirdly, demonstrating research impact, rather than reporting research outputs, is becoming more important in funding applications and project documentation. This paper concludes with a brief review of assessment frameworks developed to evaluate the real-world impact of Higher Education research.
    Full-text · Conference Paper · Nov 2014
  • Source
    • "Kuruvilla and colleagues (2006) used four main categories including research-related impacts, policy impacts, service impacts and societal impacts. Weiss (2007) criticised medical research evaluation systems as being output-based and measuring only productivity. He therefore suggested an outcome-oriented model for measuring medical research instead. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Faculty of 1000 (F1000) is a post-publishing peer review web site where experts evaluate and rate biomedical publications. F1000 reviewers also assign labels to each paper from a standard list or article types. This research examines the relationship between article types, citation counts and F1000 article factors (FFa). For this purpose, a random sample of F1000 medical articles from the years 2007 and 2008 were studied. In seven out of the nine cases, there were no significant differences between the article types in terms of citation counts and FFa scores. Nevertheless, citation counts and FFa scores were significantly different for two article types: “New finding” and “Changes clinical practice”: FFa scores value the appropriateness of medical research for clinical practice and “New finding” articles are more highly cited. It seems that highlighting key features of medical articles alongside ratings by Faculty members of F1000 could help to reveal the hidden value of some medical papers.
    Full-text · Article · Nov 2013 · Scientometrics
Show more