“As the scientific community continues to evolve, it is essential to leverage the latest technologies to improve and streamline the peer-review process. One such technology that shows great promise is artificial intelligence (AI). AI-based peer review has the potential to make the process more efficient, accurate, and impartial, ultimately leading to better quality research.” I suspect many of you were not fooled into thinking that was me who wrote that statement. A well-known AI tool wrote those words after I prompted it to discuss using AI in the peer review process. More and more, we are hearing stories about how researchers may use these tools when reviewing others’ applications, and even writing their own applications. Even if AI tools may have “great promise,” do we allow their use? Reviewers are trusted and required to maintain confidentiality throughout the application review process. Thus, using AI to assist in peer review would
Continue reading →

Cross-posted on Open Mike. As discussed in the blog, “Update on Simplifying Review Criteria: A Request for Information (RFI),” NIH issued an RFI—from December 8, 2022, through March 10, 2023—seeking feedback on its proposed plan to revise and simplify the framework for the first level of the peer review of research project grant (RPG) applications. NIH received more than 800 responses to the RFI: 780 from individuals, 30 from scientific societies, and 30 from academic institutions. The vast majority were supportive of the proposed changes, although a minority were in favor of Factor 3 (Investigator, Environment) being scored, and a smaller minority advocated for a blinded or partially blinded review process. Most of the respondents highlighted the need for strong training resources for reviewers, study sections chairs, and scientific review officers. One question that often arises is how investigator and institution will be weighted in arriving at the Overall Impact
Continue reading →

NIH is recommending changes to the peer review of Ruth L. Kirschstein National Research Service Award (NRSA) fellowship applications by restructuring the review criteria and modifying some sections of the PHS Fellowship Supplemental Form that are specific to NRSAs. The goal of this effort is to facilitate the mission of NRSA fellowship peer review – to identify the most promising trainees and the excellent, individualized training programs that will help them become the outstanding scientists of the next generation. The proposed changes will 1) allow peer reviewers to better evaluate the applicant’s potential and the quality of the scientific training plan without undue influence of the sponsor’s or institution’s reputation; and 2) ensure that the information provided in the application is aligned with the restructured criteria and targeted to the fellowship candidate’s specific training needs. The RFI requests public input on this proposal. To comment, go to the RFI, which
Continue reading →

NIH has a vested interest in receiving proposals from a wide range of investigators, from a wide range of applicant institutions. However, data on funding of research project grants (RPGs) and a recent analysis done by CSR on applicant institutions for National Research Service Award (NRSA) fellowship applications indicate some disparities among institutions in terms of funding and application rates. In FY 1985-2020, the top 10% of organizations received approximately 70% of RPG funding (Open Mike, Jan 18, 2022;Lauer and Roychowdhury, 2021). An analysis of NRSA fellowship applications in 2021 found skewed application rates; 15 institutions submitted more than 100 applications, while 106 institutions submitted one or two applications (Final report of the CSR Advisory Council Working Group on Peer Review of NRSA Fellowship Applications). To assist investigators/institutions that are submitting fewer applications, we’ve developed this resource to help clarify the process, highlighting institution- and investigator-specific information, including key actions
Continue reading →

Cross-posted on Open Mike. NIH has issued a request for information (RFI) seeking feedback on revising and simplifying the peer review framework for research project grant applications. The goal of this effort is to facilitate the mission of scientific peer review – identification of the strongest, highest-impact research. The proposed changes will allow peer reviewers to focus on scientific merit by evaluating 1) the scientific impact, research rigor, and feasibility of the proposed research without the distraction of administrative questions and 2) whether or not appropriate expertise and resources are available to conduct the research, thus mitigating the undue influence of the reputation of the institution or investigator. Currently, applications for research project grants (RPGs, such as R01s, R03s, R15s, R21s, R34s) are evaluated based on five scored criteria: Significance, Investigators, Innovation, Approach, and Environment (derived from NIH peer review regulations 42 C.F.R. Part 52h.8; see Definitions of Criteria and
Continue reading →

Guest post by Dr. Christine Hunter, Acting Director, NIH Office of Behavioral and Social Sciences Research; Dr. Valerie Durrant, Director of the Division of AIDS, behavioral and Population Sciences at the Center for Scientific Review; and Ms. Elan E. Ey, Deputy Director, Client Services Division, NIH Office of Human Resources. Originally released on July 7, 2022, on the OBSSR Director’s Voice Blog. Our hope with this blog is to highlight some exciting career paths at NIH for behavioral and social scientists that offer the opportunity to be highly engaged in shaping the future of the scientific enterprise. Science administrator jobs at NIH, called Program Officers (PO) and Scientific Review Officers (SRO), primarily involve planning, directing, and managing the evaluation of the science to ensure the best health-relevant research now and in the future. The PO and SRO roles are quite different, but each require a high level of scientific expertise
Continue reading →

I am pleased to announce the release of the CSR 2022-2027 Strategic Plan. CSR is entrusted with most of the peer review that enables NIH to support a broad range of biomedical research. Our primary goal, to ensure that peer review identifies the strongest, most promising science, depends upon an evaluation process that is fair, independent, expert, timely and free from inappropriate influences. This plan delineates a forward-looking framework comprising five overarching goals that organize CSR’s current and future initiatives in support of our important mission.   Goal 1: Maintain scientific review groups that provide appropriate scientific coverage and review settings for all of NIH science.   Goal 2: Further develop a large cadre of diverse, well-trained, and scientifically qualified experts to serve as reviewers.   Goal 3: Further develop an outstanding, engaged, and diverse staff.   Goal 4: Implement changes to the peer review process to make it more
Continue reading →

I am pleased to announce that CSR’s draft strategic plan is now open for public comment. This 5-year plan (for 2022–2027) will serve as our roadmap as CSR advances its mission of seeing that NIH grant applications receive fair, independent, expert, and timely scientific reviews—free from inappropriate influences—so NIH can fund the most promising research. Input from CSR’s stakeholders—the external scientific community, the CSR Advisory Council, NIH institutes and centers, and our own CSR staff—helped to shape the goals of the plan, all of which center on strengthening peer review. Input included critical discussions about topics that have received increased and necessary attention recently, including structural racism and the COVID-19 pandemic. The goals are:   • Goal 1: Maintain scientific review groups that provide appropriate scientific coverage and review settings for all of NIH science.   • Goal 2: Further develop a large cadre of diverse, well-trained, and scientifically qualified
Continue reading →

Have you applied for, sponsored, or reviewed NIH fellowship applications? We would like to hear your thoughts on what works, what doesn’t, and how the process could be improved. National Research Service Award (NRSA) Fellowship (F) awards are intended to support training that will enhance pre- and post-doctoral trainees’ potential to develop into productive, independent research scientists. In 2021, CSR handled the review of more than 5500 of the approximately 6800 NRSA F applications received by NIH. We recently convened a CSR Advisory Council working group, charged with evaluating the fellowship review process and making recommendations to make it as effective and fair as possible for all. The working group has noted multiple concerns, many of which center around the challenges of discerning the potential of the applicant and the value of the training planned, as opposed to the general reputation of the school and sponsor. There are concerns that
Continue reading →

An anniversary is a time for reflection on our history, the goals we’ve accomplished, the challenges we’ve surmounted, and the lessons we’ve learned along the way. Our video, “Catalyst of Hope and Health,” reflects on CSR’s work over the past 75 years to ensure that grant applications sent to NIH receive fair, independent, expert, and timely scientific reviews that are free from inappropriate influences, so NIH can fund the most promising research. Since its establishment, CSR has also sought to continually improve. I invite you to watch the video to learn about this ongoing commitment to a high-quality, fair review process that serves to advance NIH’s mission. It features former NIH director Dr. Francis Collins and additional NIH leaders such as Drs. Marie Bernard, Anthony Fauci, and Michael Lauer, as well as NIH historian Dr. Richard Mandel, present and former CSR advisory council members, and reviewers and scientific review officers.
Continue reading →