As we begin this new millennium, the Center for Scientific Review (CSR) continues to prepare to meet the challenges of the 21st Century. The way in which CSR must conduct its reviews has changed significantly in recent years, and will most likely continue to require new approaches in the coming years. For example, an increasing number of applications are being received that address complex biological problems with broad, multi-disciplinary programs. They are often collaborative, multi-investigator, multi-institutional, or multi-center. Because of the breadth of expertise required and the many conflicts-of-interest that arise, these applications present new challenges for the review process. CSR is actively exploring the issues involved in the review of such complex applications, and experimenting with new ways to meet these challenges. To attract the most appropriate reviewers to the rigors of study section membership service, we are also exploring the logistics of study section membership service and the possible creation of new types of service that will provide the best reviewers.
In addition to the increased complexity of applications, the recent increases in funding allocations for extramural research in all of NIH's Institutes and Centers have resulted in increased workloads in an understaffed environment, and CSR has been working hard to alleviate the burdensome workloads on our staff. To this end, we added 25 new professional staff in 1999 and continue to recruit new professional staff. In the current year, we hope to hit our goal of 140 SRAs, approximately one for each study section. Regretfully, Ms. Linda Engel, our former Associate Director for Planning and Outreach, has recently accepted a position with the Center for Alternative Medicine. Ms. Engel was extremely active in coordinating activities of the Panel on Scientific Boundaries for Review and other major CSR initiatives, and her enthusiastic involvement in these activities will be missed. However, we are currently recruiting for a Deputy Director for CSR, as well as additional staff. The work of the Panel on Scientific Boundaries for Review, which is a working subcommittee of the CSR Advisory Committee, is a landmark activity designed to bring peer review into the 21st century. The Panel has undertaken a comprehensive examination of the organization and function of the review process carried out in CSR to ensure that the review system is aligned with the current scientific landscape. The effort is being carried out in two phases, with extensive involvement of the extramural community in both phases. A draft of the Phase 1 report was posted on our website last July for comment. It proposed a new set of Integrated Review Groups (IRGs), recommended some cultural norms to govern the operation of the CSR review process, and outlined principles and procedures for Phase 2. Based on the responses from more than 800 individuals and societies, the Panel expanded and refined the proposed organization of CSR Integrated Review Groups (IRGs) and clarified some of the recommended operating principles. The final Phase 1 report was accepted by the CSR Advisory Committee on January 10, 2000, and is now posted on our website (http://www.csr.nih.gov). Phase 2, which will recommend specific study sections that will populate the newly formed IRGs, will be initiated shortly and will proceed over the next two to three years. During the Phase 2 process, the Panel will again rely on guidance from the extramural scientific community.
Progress also continues on our parallel efforts to evaluate the structure and function of our current IRGs. Working Groups for each IRG are being established to assess, at five-year intervals, the organizing principles and operating procedures for the study sections within each IRG. The process has recently been completed for the Biophysical and Chemical Sciences (BPC) IRG. The recommendations of the BPC Working Group will be considered in Phase 2 of the Panel on Scientific Boundaries; overall, the Working Group concluded that "BPC is functioning well and should not be compromised." Working Groups for the Oncological Sciences IRG and the Cardiovascular Sciences IRG are now in the process of evaluating these IRGs, and their efforts should be completed within the next few months.
Concurrent with these activities, another study section evaluation activity has been initiated to obtain direct input from the actual customers and stakeholders in the review process. To this end, surveys are currently being designed and tested for use by study section reviewers, by applicants, and by Institute staff regarding their perceptions of the operations of CSR study sections. Results from these surveys will also be provided to the Panel on Scientific Boundaries for consideration in their Phase 2 efforts.
Still another effort underway in conjunction with the CSR Advisory Committee is an attempt to define best practices within study sections and to optimize consistency in operations across study sections. To this end, two documents have now been completed and accepted by the CSR Advisory Committee, namely the "Guidelines for Study Section Chairs" and "The Role of the Scientific Review Administrator at CSR." Both of these documents will be posted on the CSR website shortly. A third document, "Guidelines for Study Section Reviewers" is currently underway, and should be completed this summer. Along with the development of these three documents to promote best practices, we have also instituted numerous new training programs for our Scientific Review Administrators (SRAs), with wonderful contributions to the training agendas from many of our senior SRAs. These training exercises, needed because of our many new hires and because of the many recent changes in NIH initiatives, should help achieve optimal review quality.
And, finally, I would like to report on an analysis conducted jointly by members of our professional staff and members of our Advisory Committee regarding the review of study section member applications. There has long been a perception that study section members are disadvantaged in the review process because their applications are in conflict with the study section of which they are a member, and must instead be reviewed by an alternate study section or by a special emphasis panel. To address this issue, a comparison was made of success rates and priority scores of study section members versus non-study section members, and CSR staff members were surveyed for their perceptions as to the extent their members felt disadvantaged. The overall conclusion of the final report, now posted on our website (http://www.csr.nih.gov/reviewmems.htm), is that study section members do not appear to be disadvantaged by their service to CSR as indicated by their relatively high success in funding, the absence of a substantial difference in priority scores during and after study section service, and the absence of an effect of percentiling against CSR or study section scores.
I am gratified by the interest and involvement shown by so many members of the extramural research community as we proceed in these efforts to align peer review at CSR to science in the 21st century. I look forward to your further participation as we continue in the activities of the Panel for Scientific Boundaries, the IRG evaluation activities, and the development and implementation of best practices.
Ellie Ehrenfeld Director, CSRCSR continues to actively recruit new scientific staff. Since the beginning of this fiscal year (October 1, 1999), two new SRAs have joined our ranks.
Dr. Karen Sirocco, a research psychologist with a Ph.D. from George Mason University in Applied Developmental Psychology, has recently joined the Risk, Prevention, and Health Behaviors IRG. She comes to us after 10 years with the Laboratory of Clinical Studies at the National Institute of Alcohol Abuse and Alcoholism where she was involved in basic and clinical biobehavioral research.
Dr. Ann Jerkins, who received her Ph.D. in Nutritional Sciences and Physiology from the University of Wisconsin, has recently joined the Nutritional and Metabolic Sciences IRG. Dr. Jerkins was previously Assistant Professor of Nutritional Sciences at the University of Arizona at Tucson, where her research efforts focused on cysteine catabolism.
As more and more researchers participate in consortiums, share electronic data and databases, and prepare complex, multi-disciplinary, multi-center, and multi-investigator grant applications, the review process becomes increasingly complicated because frequently the most appropriate experts for reviewing a particular application are in conflict with that application. Two new policy announcements recently released from the Office of Extramural Research (OER), NIH, should help alleviate this situation.
OER Policy Announcement 1999-02 (http://odoerdb2.od.nih.gov/oer/policies/oer_announce_1999_02.htm) provides a blanket waiver of conflict of interest for peer review consultants of multi-component institutions. Components of multi-component academic institutions are now considered independent, and an employee of one component may review an application from another component without conflict of interest, so long as no apparent or real conflict of any other sort exists. For example, this means that the separate campuses of the California State system are now considered separate components in the same way that the separate campuses of the University of California system are considered separate campuses. Similarly, the separate campuses of the Harvard system are now considered separate components. In another example, the Johns Hopkins Schools of Arts and Sciences and of Engineering, Homewood Campus, are separate components; however, for purposes of this blanket waiver, the Department of Biology and the Department of Chemistry within the School of Arts and Sciences are not separate components.
The second announcement concerns conflict of interest for peer review consultants named in applications when the named individual is not from the applicant institution. According to OER Policy Announcement 1999-03 (http://odoerdb2.od.nih.gov/oer/policies/oer_announce_1999_03.htm), when a Principal Investigator submits an application naming a participating investigator who does not have a relationship with the applicant institution, the named investigator may still review other applications from the applicant institution. Also, other individuals from the named investigator's institution may review the Principal Investigator's application, as long as other apparent and real conflicts do not exist. Formerly, when reviewers were named as consultants in applications, the entire study section was considered in conflict with that application. Now, however, the conflict is only with the reviewer (consultant) who must be out of the room for the review of that application. On the other hand, if an individual named as a participating investigator simply supplies a resource or service that is freely available to anyone in the scientific community, then the individual supplying the resource is not in conflict and can review the application.
Policy Announcement 1999-03 further clarifies conflict of interest for certain situations involving fellowship and K awards, conference grant applications, and multi-center networks. For fellowship and K award applications, reviewers who have written reference letters are in conflict and must leave the room for review of the applicant's application. However, the study section is not in conflict and can review such applications. On the other hand, if an applicant's sponsor is a member of the study section, the study section cannot review the application. For conference grant applications, planning group members and proposed speakers are in conflict, but their institutions are not, and the study section can review the application with the reviewers in conflict out of the room. For applications involving multi-center networks and advisory board members, conflicts do not extend to other institutions in the network or to other applications from the institutions directly involved. Rules and regulations are unlikely to cover all situations, and SRAs must often make decisions about individual reviewer conflicts with input from the reviewers. For example, what about collaborations in which a study section member is a participating investigator with an intellectual or financial stake in the application? The new policies do not directly address these situations. However, if the participation were substantial, then a conflict would exist for the reviewer and for the study section. At present, what constitutes "substantial" participation is a subjective decision of the SRA. If the Principal Investigator and study section member intend to publish together, the collaboration would most likely be considered substantial. Even if review participation may be technically legal, a study section member may feel uncomfortable and indicate that the situation does in fact create a conflict.
It pays to be informed, especially regarding something as important as your grant application to the NIH!
The Division of Receipt and Referral in CSR receives thousands of applications each review round. Some receipt dates require as much as three weeks to log in all of the applications, and then another three weeks to assign the applications to study sections for review and to Institutes or Centers for possible funding.
After an application is assigned to a study section, a letter is sent to the Principal Investigator/Program Director using the address given on the face page of the application. The letter contains a wealth of information, including the application assignment number, the study section assignment, the name and telephone number of the Scientific Review Administrator (SRA) who will oversee the review of your application, and a telephone number for contacting staff of the Institute or Center that will consider your application for possible funding. Many of these letters fail to reach the applicant because of problems in the address or with the delivery system, but the lack of a letter could also mean that your application was never received. Therefore, if you do not receive such a letter within six weeks after submitting a regular application (or three weeks after submitting an AIDS, fellowship, or small business application), you should contact the Division of Receipt and Referral at (301) 435-0715.
Your application will undergo peer review approximately four to six months after the receipt date, and the SRA of the assigned study section is your point of contact from the time your application is assigned there until the time it is reviewed. For information on the overall review process for unsolicited applications, see the Grant Application Kit. For solicited initiatives, the corresponding Program Announcement (PA) or Request For Applications (RFA) may provide additional useful information. Meeting dates for CSR study sections can be obtained from the CSR web site at http://www.csr.nih.gov/Committees/meetings/ssmeet1.asp, and SRAs can be contacted through this web address. If you send supplementary information to the SRA, be sure to allow plenty of time in order to permit distribution and evaluation prior to the review. About two weeks after the application has been reviewed, the priority score assigned by the study section to your application will be sent to you, and the summary statement will follow in four to six weeks after the review. Once your application has been reviewed, your point of contact becomes the Institute/Center program administrator assigned to your application. You should contact that individual with any questions or problems, or if you do not receive your summary statement within eight weeks following the review of your application.
NIH strives to keep you as informed as possible, through its website and through its staff, who encourage your calls and emails. Be sure to contact the appropriate individual with any concerns you may have.
Modular grant applications have now been submitted for two review cycles. Under the modular grant application and award procedures, applicants request total direct costs in $25,000 (one module) increments up to $250,000 in any year of a project. Detailed budgets are no longer required and information requested on Other Support has also been modified and is now requested as part of the Biographical Sketch.
To date, almost 14,500 applications (R01, R03, R15, R21) have been submitted in modular format, of which 11,600 were R01s. The 14,500 represent 93% of the total number of applications that should have been submitted in the modular format. In addition, 70% of all modular applications requested the same number of modules in all requested years. The most frequent numbers of modules requested per year were 7 ($175,000) and 8 ($200,000). Each accounted for 20% of all R01s submitted. Applicants requested 6 ($150,000), 9 ($225,000), and 10 ($250,000) modules equally, with each representing 15% of the total R01s submitted. A more detailed analysis of requested amounts for FY2000 is currently underway.
The first review cycle has generated a wide variety of opinions and reactions from reviewers on modular grant applications. Comments ranged from support and encouragement for their continued use to serious concerns involving every aspect of the modular application process. NIH is preparing a brief summary of these comments and issues that will be distributed by the Scientific Review Administrators to all reviewers. Topics to be discussed include: 1) procedures and guidance for making changes in modular grant budgets; 2) complete data on number of modules requested; 3) historical data on average costs for R01 grants; 4) historical data on the distribution of R01 grant award costs by budget category; 5) why information on Other Support is no longer provided in the applications and 6) why reviewers are no longer asked to comment on Other Support as part of their evaluation of the scientific merit of applications.
Questions and comments should be sent to modulargrants@nih.gov.