As outlined in “Update on Improving Fellowship Review: A Request for Information (RFI),” NIH issued an RFI—from April 17 through June 23, 2023—seeking input on the proposed changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications by restructuring the review criteria and modifying some sections of the PHS Fellowship Supplemental Form that are specific to NRSAs.
The RFI was published as a Guide Notice, in the Federal Register, and on our CSR Review Matters and NIH Open Mike blogs. It was pushed out through social media channels across NIH and we directly emailed leadership at almost 500 institutions across the United States.
NIH received 164 unique responses to the RFI: 147 from individuals, 10 from scientific societies, and 7 from academic institutions. Regarding review criteria, many respondents were supportive of efforts to simplify the review of NRSA fellowships to emphasize the potential of applicant, strength of science, and the quality of the training plan. They also believed that the proposed changes would make reviews more equitable and less subjective for applicants from disadvantaged backgrounds, without penalizing other applicants.
Many respondents were in favor of revising the PHS Fellowship Supplemental Form and believed it would streamline the application, reduce burden, and improve clarity and accessibility of the application process. While many were in favor of removing grades from this section, a smaller subset of respondents believed that grades should still be evaluated, as they believe past performance is often a predictor of future performance.
There were many comments that clarification on some sections of the revised review criteria and supplemental section is needed to avoid duplicative evaluation and review burden. Along with this clarification, they commented that proper training would be instrumental to guide not only reviewers, but also applicants, sponsors/co-sponsors, scientific review officers, and chairpersons for an equitable review process for all fellowship applicants. Therefore, a trans-NIH committee has been established for implementation of the changes for improving fellowship review. This committee is developing a timeline as well as designing the rollout and trainings.
We thank all who took the time to work with us in this effort to improve NRSA fellowship review and provide feedback through the RFI. We also thank those involved in other initiatives of improving peer review at NIH, such as the Simplifying Review Framework initiative for research project grants. We believe these two efforts will strengthen NIH peer review and improve the ability to identify the science with the greatest potential impact and allow peer review to consistently identify the most promising fellowship applicants.
In my experience (reviewer at many NRSA study sextions), past performance is an excellent predictor of future performance, with few exceptions. The priority of the Reviewers should be the Approach (Training Plan), the training potential score is overrated and can be easily manipulated.
I have served as a reviewer on several occasions and have been a sponsor.
I observed that high-achieving well-funded PIs (who most times are members of Centers and other organizations in highly ranked Institutions) have their PhD students and trainees apply as soon as they join their labs; the amount of preliminary data and the depth of the thought process in those F applications are largely due to the PI’s funded NIH proposals. In contrast, if a PI who runs a small lab has their student apply, the proposal, although clearly driven and put together by the student applicant (an applicant who has tremendous potential), may not be perceived as favorably by the reviewers. This leads to the situation where those who have funding for their students, now also have their students get the F award. I am not sure how to solve this problem, but in essence: The rich are getting richer and the poor are getting poorer. And I am saying this with all the respect for my colleagues in every Institution, as well as for the NIH officials.