Cross-posted on Open Mike. We are pleased to announce the details of changes to the peer review process and application forms for fellowship applications, to be implemented for applications submitted for due dates on or after January 25th, 2025. (See NOT-OD-24-107.) As we have discussed on this blog before (see here, here, here, and here), these changes are meant to facilitate the identification of the most promising candidates and the individualized training opportunities that will assist these researchers along their paths to careers in biomedical research. The changes to peer review and the fellowship application result from years of analysis and discussion. In response to continued concerns voiced by the extramural community that the current fellowship review process potentially disadvantages some highly qualified candidates, the NIH Center for Scientific Review (CSR) formed a CSR Advisory Council working group in Fall 2021, charged with evaluating the peer review process for NIH
Continue reading →

Cross-posted on Open Mike. Last October, we announced that NIH was implementing a simplified review framework for most research project grants (RPGs). As a reminder, in the simplified review framework NIH aims to better facilitate the mission of scientific peer review – identification of the strongest, highest-impact research. The changes are intended to: Enable peer reviewers to better focus on answering the key questions necessary to assess the scientific and technical merit of proposed research projects: Can and should the proposed research project be conducted? Mitigate the effect of reputational bias by refocusing the evaluation of investigator/environment to within the context of the proposed research. Reduce reviewer burden by shifting policy compliance activities to NIH staff. Today, NIH released a Guide Notice (NOT-OD-24-085) to provide an update on our implementation plans for the simplified review framework. The Notice provides guidance to applicants on navigating new and updated funding opportunities expected
Continue reading →

Cross-posted on Open Mike. As we have discussed in previous blogs, NIH has heard concerns from the extramural community about the complexity of the peer review process for research project grants (RPGs) and the increasing responsibilities of peer reviewers in policy compliance. NIH has also heard concerns about the potential for reputational bias to affect peer review outcomes. After careful input gathering, development, and discussion, NIH is pleased to announce that a Simplified Review Framework will be implemented for grant receipt deadlines of January 25, 2025 and beyond. The simplified framework is expected to better focus peer reviewers on the key questions needed to assess the scientific and technical merit of proposed research projects: “Can and should the proposed research project be conducted?” To achieve this, the five current review criteria (defined as Significance, Innovation, Approach, Investigator, and Environment; derived from NIH peer review regulations 42 C.F.R. Part 52h.8) are
Continue reading →

As outlined in “Update on Improving Fellowship Review: A Request for Information (RFI),” NIH issued an RFI—from April 17 through June 23, 2023—seeking input on the proposed changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications by restructuring the review criteria and modifying some sections of the PHS Fellowship Supplemental Form that are specific to NRSAs. The RFI was published as a Guide Notice, in the Federal Register, and on our CSR Review Matters and NIH Open Mike blogs. It was pushed out through social media channels across NIH and we directly emailed leadership at almost 500 institutions across the United States. NIH received 164 unique responses to the RFI: 147 from individuals, 10 from scientific societies, and 7 from academic institutions. Regarding review criteria, many respondents were supportive of efforts to simplify the review of NRSA fellowships to emphasize the potential of
Continue reading →

“As the scientific community continues to evolve, it is essential to leverage the latest technologies to improve and streamline the peer-review process. One such technology that shows great promise is artificial intelligence (AI). AI-based peer review has the potential to make the process more efficient, accurate, and impartial, ultimately leading to better quality research.” I suspect many of you were not fooled into thinking that was me who wrote that statement. A well-known AI tool wrote those words after I prompted it to discuss using AI in the peer review process. More and more, we are hearing stories about how researchers may use these tools when reviewing others’ applications, and even writing their own applications. Even if AI tools may have “great promise,” do we allow their use? Reviewers are trusted and required to maintain confidentiality throughout the application review process. Thus, using AI to assist in peer review would
Continue reading →

Cross-posted on Open Mike. As discussed in the blog, “Update on Simplifying Review Criteria: A Request for Information (RFI),” NIH issued an RFI—from December 8, 2022, through March 10, 2023—seeking feedback on its proposed plan to revise and simplify the framework for the first level of the peer review of research project grant (RPG) applications. NIH received more than 800 responses to the RFI: 780 from individuals, 30 from scientific societies, and 30 from academic institutions. The vast majority were supportive of the proposed changes, although a minority were in favor of Factor 3 (Investigator, Environment) being scored, and a smaller minority advocated for a blinded or partially blinded review process. Most of the respondents highlighted the need for strong training resources for reviewers, study sections chairs, and scientific review officers. One question that often arises is how investigator and institution will be weighted in arriving at the Overall Impact
Continue reading →

NIH is recommending changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications by restructuring the review criteria and modifying some sections of the PHS Fellowship Supplemental Form that are specific to NRSAs. The goal of this effort is to facilitate the mission of NRSA fellowship peer review – to identify the most promising trainees and the excellent, individualized training programs that will help them become the outstanding scientists of the next generation. The proposed changes will 1) allow peer reviewers to better evaluate the applicant’s potential and the quality of the scientific training plan without undue influence of the sponsor’s or institution’s reputation; and 2) ensure that the information provided in the application is aligned with the restructured criteria and targeted to the fellowship candidate’s specific training needs. The RFI requests public input on this proposal. To comment, go to the RFI, which
Continue reading →

NIH has a vested interest in receiving proposals from a wide range of investigators, from a wide range of applicant institutions. However, data on funding of research project grants (RPGs) and a recent analysis done by CSR on applicant institutions for National Research Service Award (NRSA) fellowship applications indicate some disparities among institutions in terms of funding and application rates. In FY 1985-2020, the top 10% of organizations received approximately 70% of RPG funding (Open Mike, Jan 18, 2022;Lauer and Roychowdhury, 2021). An analysis of NRSA fellowship applications in 2021 found skewed application rates; 15 institutions submitted more than 100 applications, while 106 institutions submitted one or two applications (Final report of the CSR Advisory Council Working Group on Peer Review of NRSA Fellowship Applications). To assist investigators/institutions that are submitting fewer applications, we’ve developed this resource to help clarify the process, highlighting institution- and investigator-specific information, including key actions
Continue reading →

Cross-posted on Open Mike. NIH has issued a request for information (RFI) seeking feedback on revising and simplifying the peer review framework for research project grant applications. The goal of this effort is to facilitate the mission of scientific peer review – identification of the strongest, highest-impact research. The proposed changes will allow peer reviewers to focus on scientific merit by evaluating 1) the scientific impact, research rigor, and feasibility of the proposed research without the distraction of administrative questions and 2) whether or not appropriate expertise and resources are available to conduct the research, thus mitigating the undue influence of the reputation of the institution or investigator. Currently, applications for research project grants (RPGs, such as R01s, R03s, R15s, R21s, R34s) are evaluated based on five scored criteria: Significance, Investigators, Innovation, Approach, and Environment (derived from NIH peer review regulations 42 C.F.R. Part 52h.8; see Definitions of Criteria and
Continue reading →

Guest post by Dr. Christine Hunter, Acting Director, NIH Office of Behavioral and Social Sciences Research; Dr. Valerie Durrant, Director of the Division of AIDS, behavioral and Population Sciences at the Center for Scientific Review; and Ms. Elan E. Ey, Deputy Director, Client Services Division, NIH Office of Human Resources. Originally released on July 7, 2022, on the OBSSR Director’s Voice Blog. Our hope with this blog is to highlight some exciting career paths at NIH for behavioral and social scientists that offer the opportunity to be highly engaged in shaping the future of the scientific enterprise. Science administrator jobs at NIH, called Program Officers (PO) and Scientific Review Officers (SRO), primarily involve planning, directing, and managing the evaluation of the science to ensure the best health-relevant research now and in the future. The PO and SRO roles are quite different, but each require a high level of scientific expertise
Continue reading →