The Center for Scientific Review established the Early Career Reviewer (ECR) program in late 2011 with two major goals – 1) to expose early-career scientists to the peer review process, with the ultimate goal of helping them to become more competitive as applicants, and 2) to enrich and diversify NIH’s pool of trained peer reviewers.
We are delighted that the program has generated a lot of interest over the years. However, the strong response has also resulted in a significant backlog of potential ECRs, one that has existed since the program’s inception almost eight years ago. At this time, we have over 2,500 qualified ECRs in our system – a number that is increasing as we continue to publicize the program.
In the past few years, we have made attempts to clear the backlog, with mixed results. We instituted a requirement to include an ECR on every recurring R01 study section every review cycle. However, that resulted in only ~540 ECRs being used per year. Another attempt to reduce the backlog involved restricting each ECR to serving just once instead of twice, as was originally planned, to allow more participants to gain review experience. However, feedback from participants indicated a need for increased frequency of service to solidify the experience. We have discussed the possibility of allowing ECRs to serve on study sections that review other mechanisms, such as fellowships or SBIRs or institute review committees that handle multicomponent applications, etc. However, that seems to not be aligned with the original goal of the program – and with what most ECRs are looking for – a chance to serve on an R01 committee that is related to their area of science, where they might send their own R01 for review. With ECR service now restricted to one time, it seems best if ECRs do not use that one opportunity to serve in a review meeting without R01 applications. There are no easy solutions to the problem.
To help us rethink and restructure this important program so that it can benefit more early career scientists, I will be convening a working group of the CSR Advisory Council this summer. This working group will include some members of our council, as well as several early career scientists and our own CSR staff. We will be discussing the issues raised above, as well as others. Stay tuned.
We also welcome your ideas at feedback@csr.nih.gov. We look forward to engaging more with the community about these issues and will share our findings.
Thank you for asking for input on improving the ECR program. The challenge for the ECR program is to provide the very best review experience for all participants. That has been defined as being provided with the opportunity to serve as a reviewer on R01 applications at two separate review meetings. While ideal, there are obvious space limitations. Based on my experience with the ECR program, I would suggest the following:
Add submission of at least one R01 or comparable grant application to the inclusion criteria for participation on a study section. This will improve the study section experience of the ECR who will have had direct practice in writing a grant and therefore more aware of the complexities. IT would also decrease the pool of ECR applicants.
Allow ECRs to participate in 2 study sections when possible.
When a second R01 review experience is not available, offer the ECR an opportunity to review another award type.
Allow 2 ECRs per study section, one as a 4th reviewer/discussant and one as a reviewer 3. Keep track of reviewer type for each ECR so that at least one R01 review was completed. This may require a modification to the ECR database.
I think that new investigators would welcome a less than perfect experience instead of sitting on a waiting list. Allow them to participate on study sections in the I/Cs if they have not been placed on a study section within two years of entering the program. Allow this to count as one of the two ECR review experiences.
Send out annual queries to the ECRs in the database who have not yet participated to check on their continued interest in being included on a review panel. In the many summaries, we often found that a substantial number had moved to a different institution, had not written NIH grant applications, or were not longer in research positions.
Create a template for mock study sections. Engage reviewers to participate in mock study sections at regional NIH meetings. Invite several ECRs to participate, including those who have had at least one real review experience. Again, something less than perfect is better than nothing.
In addition to ECRs, invite a broader pool of reviewers to participate as ad hoc reviewers, including researchers who have had smaller NIH grants. Make it a point to recruit a larger number of women and members of other under-represented groups to serve as ad hocs. This might include people more senior than ECRs, but less senior than our usual pool of ad hoc reviewers.
Finally, accept the fact that the CSR will not be able to provide ideal review experiences for all potential ECRs. Work with OER to find other ways to provide training for early career researchers.
Thank you for your comments and suggestions. Many of these – increasing the number of ECRs at a meeting, removing people from the database who are no longer in research positions, and alternative forms of training – are already being considered by a CSR Advisory Council working group. I’ll share your post with that group as well for discussion. We look forward to sharing their recommendations and changes to the ECR program soon.
Dr. Byrnes:
Thanks for the very informative post. When I was early career scientist I participated in research proposals mostly on R-21 and rarely for R-01. Now I rarely get any proposal to review from NIH. I feel that scientist as all stages of their career should rotate and perhaps each study section have a mixed group of reviewer’s who are at various stage of their career to serve. I also feel that life time of each study section should be no more than two years to allow a large majority of reviewer to participate. This is because if there are more reviewers waiting on line for the opportunity to serve, then perhaps the review time should be shortened and so is the member’s serving in the study section.
I hope that a large scientific community could be engaged on a more rotation basis in many study section of NIH.
Sincerely
Thank you for sharing your thoughts. We agree that study sections should have a mixed group of reviewers. Not only are we considering how we might involve more ECRs, we are also moving to include more Assistant and Associate Professors on study sections, so that there is better representation on our panels across career stage. We agree that membership on study sections should be dynamic and it is. While members of standing study sections have either a 4-year term (serving at 3 meetings/year) or a 6-year term (serving at 2 meetings/year), membership on panels changes every year as some members rotate off and new members begin a term. The composition of study sections also changes every review cycle as ad-hoc reviewers are recruited for each meeting and these ad-hoc reviewers typically change every meeting, in part because they are recruited to accommodate changing topics and the different expertise needed from meeting to meeting. We want to broaden our pool of reviewers and engage a larger portion of the scientific community and are working to do so.