How will study sections meet in the future? NIH peer review depends on robust meetings where groups of scientists, through vigorous discussion, identify the applications of highest merit. For the last 75 years, until last March, nearly all chartered review committee meetings were held in-person. Today, in response to the pandemic, 90% of all CSR review meetings are run as video (“Zoom”) meetings. CSR is taking steps now so that when all options are back on the table, we can make informed choices about how best to convene review meetings. Last round we obtained survey responses from 3,000 NIH reviewers, ratings by scientific review officers (SRO) of 230 review meetings, compiled quantitative data comparing in-person versus Zoom instances of over 275 meetings, analyzed rosters from those meetings, and also surveyed our support staff. The data give no indication that the forced switch to Zoom has introduced major problems. Quality of
Continue reading →

The murder of George Floyd in Minneapolis is just one of the latest disgusting examples of the systemic racial bias that has plagued this country for centuries. While our Black/African-American colleagues have to deal with this in their everyday lives, the recent incidents have led the rest of us to do some soul-searching about the part that we play in perpetuating this bias, either by our actions or by our failure to act. Here at the Center for Scientific Review, our mission includes words such as “fair”, “independent” and “free from inappropriate influences”. And bias, in all its many insidious forms, is the very antithesis of fairness. As indicated by several published studies over the last decade, and NIH’s own analyses, there remains a serious and disturbing disparity in NIH R01 award rates between White and Black applicants. Isolating the effect of race in the peer review process is a
Continue reading →

CSR has launched an online portal through which scientific societies may recommend scientists to serve as NIH reviewers. This comes in response to requests from professional societies for a way to recommend potential reviewers and is part of CSR’s ongoing efforts to refresh and expand the pool of well-qualified reviewers in every area of science. This new online tool is easy to use and, by gathering key review-relevant information, makes it much more likely that scientific review officers will be able to find and invite the scientists who are recommended. We ask that scientific societies vet reviewer qualifications before entering the recommendations. They should be scientists who are generally interested in serving as reviewers. Scientific expertise, extramural funding, and productivity are examples of qualifications. In addition, we strongly encourage societies to recommend productive scientists from diverse backgrounds and career stages – e.g. assistant, associate, and full professors. Early career scientists
Continue reading →

CSR will conduct all summer peer review meetings using one of three platforms – 1) video; 2) telephone; 3) web-based discussion. A majority will take place using the Zoom video platform. We want to provide information about how we are maintaining the security and confidentiality of our review meetings. The Zoom video platform that we are using is not the same as that used by schools or by you at home. Instead, we are using a FedRAMP-certified version of Zoom within the zoomgov.com domain. It meets requirements for other agencies that handle very sensitive information, including the Department of Homeland Security. FedRAMP certification means, for reviewers, the platform can be used without risking installation of malware and, for applicants, meetings remain confidential. Key features include:   – All video traffic is highly encrypted and continuously monitored via stringent security controls in place   – Strong configuration management is in place
Continue reading →

Over the past several years we have heard consistent concerns about the complexity of review criteria and administrative load of peer review. CSR shares the concern that the current set of standards has the unintended consequence of dividing reviewer attention among too many questions, thus reducing focus on scientific merit and increasing reviewer burden. Each element was intended make review better, but we worry that the cumulative whole may in fact distract from the main goal of review — to get input from experts on the scientific and technical merit of the proposed work. To address these concerns, CSR has convened a working group of our advisory council, charged with recommending changes to research project grant review criteria that will improve review outcomes and reduce reviewer burden. The group is co-chaired by Tonya Palermo and me, and includes some of our council members, other members of the scientific community, and
Continue reading →

The scientific peer review process benefits greatly when the study section reviewers bring not only strong scientific qualifications and expertise, but also a broad range of backgrounds and varying scientific perspectives. Bringing new viewpoints into the process replenishes and refreshes the study section, enhancing the quality of its output. In this context, CSR recently removed the requirement to have at least 50% full professors on committees. This had sometimes led to a misguided attempt to “do better than the metric” by aiming for a committee of all full professors. We are now encouraging scientific review officers (SROs) to focus on scientific contributions (demonstrable in a range of ways, e.g. recent publications, R01 or equivalent extramural funding from other sources, etc.), expertise, and breadth instead of trying to meet a career-stage metric. Our goal is to achieve a balance of perspectives by including a mix of qualified senior, mid-career, and junior
Continue reading →

In June I wrote that we intended to re-evaluate our Early Career Reviewer (ECR) Program. Thanks to the work and ideas of many people, including some of you, I can provide you with an update. In July, I convened a CSR Advisory Council Working Group, comprising two members of the CSR Advisory Council (Drs. Mark Peifer and Elizabeth Villa), two CSR SROs (Drs. Kristin Kramer and Antonello Pileggi), and four scientists who recently served as ECRs (Dr. Vinay Aakalu, University of Illinois at Chicago; Dr. Stephanie Cook, New York University; Dr. Lisa Jones, University of Maryland; and Dr. Manuel Llano, University of Texas at El Paso). I am very grateful to all the members of this working group for their thoughtful recommendations, which were informed by the members who had direct experience as ECRs, as well as by ECR survey results and input from the broader scientific community of early
Continue reading →

Diversity and inclusion in all areas of research, including at scientific meetings and in conference representation, helps bolster innovations in science. On behalf of the NIH Center for Scientific Review (CSR), I stand in full support of NIH Director Francis Collins’ commitment toward greater diversity and inclusivity in the biomedical research community. As Dr. Collins pledged, “Too often, women and members of other groups underrepresented in science are conspicuously missing in the marquee speaking slots at scientific meetings and other high-level conferences. Starting now, when I consider speaking invitations, I will expect a level playing field, where scientists of all backgrounds are evaluated fairly for speaking opportunities. If that attention to inclusiveness is not evident in the agenda, I will decline to take part.” I share Dr. Collins’ concerns and I will decline speaking invitations when attention to inclusiveness is not evident in the agenda. Scientific innovation proceeds at a
Continue reading →

The Center for Scientific Review established the Early Career Reviewer (ECR) program in late 2011 with two major goals – 1) to expose early-career scientists to the peer review process, with the ultimate goal of helping them to become more competitive as applicants, and 2) to enrich and diversify NIH’s pool of trained peer reviewers. We are delighted that the program has generated a lot of interest over the years. However, the strong response has also resulted in a significant backlog of potential ECRs, one that has existed since the program’s inception almost eight years ago. At this time, we have over 2,500 qualified ECRs in our system – a number that is increasing as we continue to publicize the program. In the past few years, we have made attempts to clear the backlog, with mixed results. We instituted a requirement to include an ECR on every recurring R01 study
Continue reading →

It is critical for the NIH and for CSR to ensure the integrity and impartiality of the peer review process. Service on peer review is neither a right nor a requirement. As an agency, we can exercise discretion on who we invite to serve, or continue to serve, on a peer review committee. We are not arbitrary in our actions, but there are many reasons we may choose not to include specific individuals on our peer review committees. For example, some individuals may have too many conflicts of interest, or too much review service, resulting in undue influence over an area of science. We may remove or not invite back a reviewer who has a pattern of submitting reviews late, requiring applications to be rereviewed. And we are particularly concerned when evidence arises that a peer reviewer may have breached confidentiality of the review process. These are just a few
Continue reading →