Non-Financial Conflicts of Interest in Academic Grant Evaluation: A Qualitative Study of Multiple Stakeholders in France

AP-HP, Hôpital Robert Debré, Unité d'Épidémiologie Clinique, Paris, France.
PLoS ONE (Impact Factor: 3.23). 04/2012; 7(4):e35247. DOI: 10.1371/journal.pone.0035247
Source: PubMed


Peer review is the most widely used method for evaluating grant applications in clinical research. Criticisms of peer review include lack of equity, suspicion of biases, and conflicts of interest (CoI). CoIs raise questions of fairness, transparency, and trust in grant allocation. Few observational studies have assessed these issues. We report the results of a qualitative study on reviewers' and applicants' perceptions and experiences of CoIs in reviews of French academic grant applications.
We designed a qualitative study using semi-structured interviews and direct observation. We asked members of assessment panels, external reviewers, and applicants to participate in semi-structured interviews. Two independent researchers conducted in-depth reviews and line-by-line coding of all transcribed interviews, which were also subjected to Tropes® software text analysis, to detect and qualify themes associated with CoIs. Most participants (73/98) spontaneously reported that non-financial CoIs predominated over financial CoIs. Non-financial CoIs mainly involved rivalry among disciplines, cronyism, and geographic and academic biases. However, none of the participants challenged the validity of peer review. Reviewers who felt they might be affected by CoIs said they reacted in a variety of ways: routine refusal to review, routine attempt to conduct an impartial review, or decision on a case-by-case basis. Multiple means of managing non-financial CoIs were suggested, including increased transparency throughout the review process, with public disclosure of non-financial CoIs, and careful selection of independent reviewers, including foreign experts and methodologists.
Our study underscores the importance of considering non-financial CoIs when reviewing research grant applications, in addition to financial CoIs. Specific measures are needed to prevent a negative impact of non-financial CoIs on the fairness of resource allocation. Whether and how public disclosure of non-financial CoIs should be accomplished remains debatable.

Download full-text


Available from: Florence Tubach
  • Source
    • "In a previous study [17], we investigated biases in the grant application review process used by the French Academic Hospital Research Grant Agencies (Programmes Hospitaliers de Recherche Clinique, PHRCs). The results showed that conflicts of interests affected the review process. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Peer review of grant applications has been criticized as lacking reliability. Studies showing poor agreement among reviewers supported this possibility but usually focused on reviewers' scores and failed to investigate reasons for disagreement. Here, our goal was to determine how reviewers rate applications, by investigating reviewer practices and grant assessment criteria. We first collected and analyzed a convenience sample of French and international calls for proposals and assessment guidelines, from which we created an overall typology of assessment criteria comprising nine domains relevance to the call for proposals, usefulness, originality, innovativeness, methodology, feasibility, funding, ethical aspects, and writing of the grant application. We then performed a qualitative study of reviewer practices, particularly regarding the use of assessment criteria, among reviewers of the French Academic Hospital Research Grant Agencies (Programmes Hospitaliers de Recherche Clinique, PHRCs). Semi-structured interviews and observation sessions were conducted. Both the time spent assessing each grant application and the assessment methods varied across reviewers. The assessment criteria recommended by the PHRCs were listed by all reviewers as frequently evaluated and useful. However, use of the PHRC criteria was subjective and varied across reviewers. Some reviewers gave the same weight to each assessment criterion, whereas others considered originality to be the most important criterion (12/34), followed by methodology (10/34) and feasibility (4/34). Conceivably, this variability might adversely affect the reliability of the review process, and studies evaluating this hypothesis would be of interest. Variability across reviewers may result in mistrust among grant applicants about the review process. Consequently, ensuring transparency is of the utmost importance. Consistency in the review process could also be improved by providing common definitions for each assessment criterion and uniform requirements for grant application submissions. Further research is needed to assess the feasibility and acceptability of these measures.
    Full-text · Article · Sep 2012 · PLoS ONE
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: As medical editors, we are faced with a host of ethical dilemmas on a daily basis. Most are recognised and dealt with expediently, but the few that remain must be challenged as a cohesive body of editorial opinion.
    Full-text · Article · Sep 2012 · Head & Neck Oncology
  • [Show abstract] [Hide abstract]
    ABSTRACT: Objectives: The American College of Chest Physicians Antithrombotic Guidelines ninth iteration placed restrictions on panelists with recommendations on which they disclosed a primary conflict of interest (COI). We aimed to describe panelists' financial and intellectual COI and evaluate to what extent, beyond assessing financial COI, assessing intellectual COI affected COI management. Study design and setting: We classified financial and intellectual COI into primary (causes voting restriction) and secondary (no restrictions). We analyzed disclosures respectively with panelists and recommendations as units of analysis. Results: One hundred two panelists made 4,030 disclosures for 431 recommendations. The median number (and range) of panelists per recommendation who disclosed the various categories of COI was 0 (0-5) for primary financial COI, 0 (0-4) for secondary financial COI, 0 (0-7) for primary intellectual COI, and 1 (0-6) for secondary intellectual COI. Of the 431 recommendations, 63 (15%) had at least one panelist with a primary intellectual COI but no primary financial COI. Conclusion: COI had a relatively low prevalence and a skewed distribution, many panelists with none and some with many disclosures. A substantial number of disclosures should have resulted in restrictions based on intellectual COI in the absence of financial COI.
    No preview · Article · Jun 2014 · Journal of Clinical Epidemiology
Show more