On March 1, NIH Director Francis Collins announced NIH’s broad-based initiative, UNITE, to end structural racism and racial inequities in biomedical science. This is a recognition of the need for urgent, sustained effort on many fronts across the research enterprise, including in all parts of the NIH’s extramural processes, to change culture. While the NIH Institutes and Centers will examine their programmatic priorities and discretionary funding practices, here at CSR, we are committed to pushing ahead with efforts to protect the peer review process from the systemic biases that exist in all areas of the scientific community.
In the June 2020 Review Matters blog, I wrote about some of the steps that CSR is taking to address individual and systemic biases in peer review. Following that, in July 2020, we held three community listening sessions, in which we heard the rightful anger and the call for urgent and specific action around the persistent funding disparity for Black investigators. I shared the report and recommendations from those forums with NIH leadership, with the UNITE E group that is focused on extramural changes, as well as with our own CSR Advisory Council. Since then, I have held a number of individual and small group conversations with investigators, who shared their personal experiences of bias as an applicant or reviewer, which has helped us further refine the strategies we were already pursuing, as well as develop some new approaches. Below are a few of the actions we are taking:
- Reporting: Many of you asked for a way to report concerns regarding bias in the peer review process directly to CSR management. Our Associate Director for Diversity & Workforce Development, Dr. Gabriel Fosu, will serve as a reporting avenue for any concerns around fairness in review. Dr. Fosu reports directly to me and I will see all reports. Beginning on March 15, all CSR scientific review officers (SROs) and staff will provide this information in their email signature lines. To report concerns around fairness in review: G.Fosu_AssocDir@csr.nih.gov.
- “Bias awareness in review” training for SROs, Reviewers, Chairs: Despite a brief interruption due to an executive order that has since been rescinded, we are forging ahead with the development of an interactive training module on bias. It will include a range of nuanced case studies to raise awareness of potential biases and mitigation strategies and tools for bystanders. We plan to launch the training for all CSR reviewers, chairs and SROs in August 2021.
- Diversifying and broadening the pool of reviewers: Through repeated multi-pronged communication efforts, CSR leadership has made our expectation and commitment to improving demographic diversity in special emphasis and chartered panels clear to staff. We have posted data on reviewer demographics and those data will be updated regularly. CSR is working hard to expand the pool of well-qualified reviewers and to build tools to help our SROs identify new scientists to bring to NIH review.
- Decoupling the science from the investigator/environment: The systemic advantages of reputation, network and pedigree are deeply entrenched in the culture of the biomedical research enterprise, and rarely benefit minorities and women, who are less likely to be part of these networks. The peer review system is certainly not immune from this systemic bias. In an effort to address the effect of these positive biases on the evaluation of scientific merit, CSR is investigating the use of blinded review processes. In collaboration with the NIH Common Fund, CSR is now conducting a pilot of a multi-stage, partial double-blinded review tR01 applications (April 2021 review). In addition, a Working Group of the CSR Advisory Council is spearheading efforts to simplify peer review criteria, recommending a decoupling of the science from the investigator/environment criteria, opening the door for a first-stage review of the science without knowledge of the investigator or institution.
Culture change is never easy, and I expect that there will be resistance from some who have benefited from the status quo. Together with all of CSR’s senior management, I stand with the NIH Director and leadership as we commit to making the changes that are long overdue.
It would be beneficial to all if NIH would include scientists/engineers/clinicians with disabilities and/or chronic conditions within NIH diversity initiatives including peer-review. As with other under-represented populations they have important insights that should be represented, and their careers need to be supported to advance science and technology.
We agree. In fact, NOT-OD-20-031 Notice of NIH’s Interest in Diversity (https://grants.nih.gov/grants/guide/notice-files/NOT-OD-20-031.html) explicitly discusses diversity beyond racial, ethnic, and gender demographics and includes other groups under-represented in science such as people with disabilities. Not surprisingly, review using the Zoomgov platform makes it easier for some groups to participate. Some reviewers have told us that they normally decline review invitations due to disabilities that make travel difficult but that they are happy to review via a remote format.
Decoupling the science from the investigator/environment: I think is a really good idea. It might be difficult to achieve as the reviewers will know where the content of the grant is coming from and the authors style. It is going to need some thought – we need a way of obfuscating the source of the science and this has to be done fairly.
I think science should be based on its on merits, not demographics. Double blinded reviews would be a welcome change.
Systemic and structural racism is only one form of bias in NIH. It is not clear how the Policy aims to correct other forms of systemic and structural biases? For example, some study sections have prejudices against specific sub-disciplines. Is Policy seeks to broaden the pool of reviewers from a diversity of sub-disciplines? How would this aspect be implemented?
Thank you for the update. I appreciate all the effort and look forward to the guidance as we all strive to improve.
Although Black scientists are disproportionately affected by policies and procedures conducive to funding, CSR needs to also pay attention to Hispanic Scientists that are also affected by systemic biases.
I has the same thought. Often our names make it clear to reviewers that we are Latino. When your name reveals your background it makes you vulnerable to bias, including implicit bias. Making the review of the scientific merit blind would help to address this. Although I think funding by reputation is one of the biggest problems. Having served as a reviewer, it is blatant and I have even heard it verbalized during discussion by reviewers who similarly enjoy such advantages. Thereby pressuring other reviewers on the panel to score low and overlook flaws because of the reputation and prior work of the applicant – rather than the scientific merits of the application under review. At the same time, new or less prolific investigator applications are strongly affected by even minor flaws.
We agree completely. We expect our efforts to address biases – diversifying panels, bias awareness training, multi-stage anonymized review, and the like – will have broad effects.
The “environment” criterion in grant review creates bias toward well-resourced institutions and contributes to downgrading the opportunities for those at other institutions, and particularly harms minorities. It should be abolished.
The ‘investigator’ also biases toward already well-funded investigators, but it is hard to see how one could fairly review a grant without knowing something about the investigator.