Table of Contents
As I write this article in late September, the federal fiscal year (FY) 2000 is rapidly drawing to a close. We do not yet know what FY 2001 will bring in terms of appropriations for the National Institutes of Health (NIH), but many congressional leaders seem determined to continue on the course they have set - to double the NIH budget over five years. While the level of funding that NIH will receive starting in October 2000 is still unknown, even more uncertain is what lies beyond. There will be many changes in Congress following the November elections; we know already that NIH's strong advocate, Mr. Porter, has decided not to seek another term. And, of course, there will also be a new administration. We cannot know exactly what changes in policies and priorities will result from these changes, but it is heartening to read over the past few weeks that the Presidential contenders from both major parties have embraced the importance of biomedical research and voiced their strong support for continuing the increase in NIH's funding level. The change in administration will also mean the eventual appointment of a new NIH Director. It is not at all certain when this will actually happen, but at least at the beginning of the new President's term, Dr. Ruth Kirschstein, Principal Deputy Director, will continue to provide her strong leadership to the NIH enterprise. Meanwhile, the Center for Scientific Review (CSR) continues to conduct its business and to move forward on its various initiatives, some of which I have described previously and are updated below.
In the previous issue of Peer Review Notes, I reported that CSR had initiated the Phase 2 implementation of the recommendations of the Panel on Scientific Boundaries for Review (PSBR). These recommendations and the implementation plan may be found at (http://www.csr.nih.gov/events/specialreports.htm). The first step in the process was to determine how a typical round of applications distributes among the 24 proposed Integrated Review Groups (IRGs). CSR has now completed its "mock referral" of all applications received for the May 2000 Council meetings, and identified overlaps and ambiguities which CSR has tried to resolve. The next step in the process is to form Steering Committees for each IRG or group of IRGs. The purpose of the Steering Committees will be to create Study Section Boundaries Teams (SSB Teams) to determine the scientific boundaries for study sections to be established within each of the proposed IRGs. As each SSB Team completes its proposal for the study sections, the proposed descriptions will be posted on the Web for public comment. The process is now underway for the Hematology IRG (HEM) and it is anticipated that this effort will be completed by January 2001. Upon completion of this test case, SSB Teams will be created for three more IRGs. Although this may be optimistic, we hope to address three IRGs at a time on four-month cycles, such that nine IRGs will be initiated in a year. The order in which the IRGs will be addressed is currently being finalized, after which it will be posted on our Website (http://www.csr.nih.gov).
In a separate effort, we are establishing Working Groups to evaluate the structure and function of our current IRGs as part of our program for periodic (every 5 years) monitoring and review of study section function. To date, Working Groups have completed the evaluation of six IRGs, namely, the Biophysical and Chemical Sciences IRG (BPC), the Oncological Sciences IRG (ONC), the Cardiovascular Sciences (CVS), the Musculoskeletal and Dental Sciences IRG (MSD), the Immunological Sciences IRG (IMM), and the Molecular, Cellular and Developmental Neuroscience IRG (MDCN). In addition, the final reports for two more neuroscience IRGs are in draft form. We aim to complete evaluations of all IRGs by fall 2001, and so far are proceeding on schedule.
As the Working Groups' reports become available, some common themes emerge. One such theme is a difficulty in accepting the modular grant format and the process for streamlining grant applications. Other common themes are the need for effective training of reviewers and study section chairs, and an emphasis on the role of the study section Chair in the assignment of grant applications to reviewers by the Scientific Review Administrator (SRA). Lessons from the neuroscience IRGs, that have already undergone reorganization, are that implementation of changes in study section organization should not immediately follow design, but must be carefully planned out, and that attention should be paid to the size of the resulting study section.
Still another effort within CSR is the development of survey instruments to provide feedback from our partners and customers on how we are accomplishing our mission. A reviewer survey was distributed to all study section members during the June round of meetings; the results of this survey are now undergoing detailed analysis by our consultants. Preliminary results regarding reviewer workloads are already available. Based on the last two review rounds, written critiques per reviewer ranged from 3 to 10 per round, with 82% of the reviewers preparing 4 to 9 written critiques per meeting, most commonly 6 written critiques per reviewer per round. The average time spent on written critiques is 38 hours, or about 6 hours per application. In addition, on average, each reviewer is assigned 2 additional applications to read (range 0 - 6). The results are preliminary, and it is anticipated that a full report of reviewer survey results should be available for the January 2001 issue of Peer Review Notes.
Another noteworthy item is that the Office of Extramural Research in the Office of the Director, NIH, has determined that study section meeting rosters should be available to the scientific community in advance of the study section meeting. Accordingly, CSR is currently programming its Web to display meeting rosters for each study section approximately four weeks in advance of the meeting. Both permanent and temporary members who will be present are listed, thus providing the total scope of expertise that may be present at the meeting. Of course, these rosters are tentative, pending last minute changes before the meeting. Rosters are provided for information purposes only, and applicants should not communicate directly with study section members about their applications; all questions should be directed to the SRA in charge of the study section.
Finally, CSR continues to be active in bringing on outstanding new staff. We have recruited 28 new SRAs in calendar 1999, and 20 so far in 2000. These recruitments reflect replacement of retired individuals as well as expansion toward our goal of one SRA for each standing study section. In addition, we have added a few senior management staff to coordinate our many new initiatives. SRA workload analyses show that, although we are better off than last year, too many study sections still review more than 90 applications per meeting, and too many SRAs manage too many meetings or too many applications. We hope that the FY2001 budget will allow us to continue with our growth plan to address these inequities as well as to accommodate the many demands that increasingly challenge our staff.
CSR continues bring on new staff. The following professional staff members have joined CSR since the May 2000 issue of Peer Review Notes.
Dr. Brent Stanfield came on board in July 2000 as the new Deputy Director for CSR. Dr. Stanfield will interact with CSR Director Dr. Ehrenfeld in establishing a strategic plan for CSR, in developing policies and priorities, and in planning, directing, and coordinating CSR's activities. Dr. Stanfield was previously Director of the Office of Science Policy and Program Planning at the National Institute of Mental Health. His Ph.D. is in Neurobiology from Washington University in St. Louis.
Dr. Robert Eisinger, the new Associate Director for Planning, Analysis and Evaluation at CSR, will be involved in implementation and evaluation of the recommendations by the Panel on Scientific Boundaries for Review. He comes to us from the Office of AIDS Research, Office of the Director, where he was Chair of the Therapeutics Coordinating Committee. Dr. Eisinger's Ph.D. is in Microbiology from the University of North Texas.
Dr. Donald Schneider is now the permanent Director of the Division of Molecular and Cellular Mechanisms. He has been acting in that capacity since July 1999. His Ph.D. is in Biochemistry from Michigan State University, and he was previously Chief of the Biophysical and Chemical Sciences IRG.
Mr. John Czajkowski has recently become the Deputy Executive Officer at CSR. His previous position was Executive Officer and Chief Financial Officer at the Center for Information Technology, NIH. Mr. Czajkowski holds a B.A. in Economics from the University of Maryland and is currently working towards a Master of Policy Sciences.
Dr. Dat Tran has joined CSR as a Medical Science Writer and Editor in the Division of Molecular and Cellular Mechanisms. His Ph.D. is in Chemistry from the University of California at Santa Barbara. Before joining CSR, Dr. Tran was a Postdoctoral Research Fellow in the Department of Chemistry at Princeton University.
Dr. Karen Bowers is the new Medical Science Writer and Editor in the Division of Physiological Sciences. Until becoming a permanent CSR employee, Dr. Bowers was a contract science writer for CSR. Dr. Bowers' Ph.D. is in Biochemistry from the University of Illinois at Champaign Urbana.
Ms. Kelly Kim is the new Medical Science Writer and Editor in the Division of Clinical and Population Based Studies. She recently received a Master's Degree in Microbiology from the University of Indiana.
Dr. Yvette Davis is now SRA of the Social Sciences, Nursing, Epidemiology, and Methods Study Section 2 (SNEM-2) within the SNEM IRG. Dr. Davis comes to us from the Division of Tuberculosis Elimination at the Centers for Disease Control where she was a Medical Epidemiologist. Her doctorate is in Veterinary Medicine from the University of Pennsylvania, and she has a Master of Public Health in Epidemiology from the Johns Hopkins University.
Dr. Sam Edwards has just recently joined CSR as the new SRA of the Allergy and Immunology Study Section within the Immunological Sciences IRG. Dr. Edwards was formerly Assistant Professor, Department of Pharmacology and Therapeutics at the University of South Florida College of Medicine. His Ph.D. is in Zoology from the University of Maryland .
Dr. Victor Fung is now SRA of the Chemical Pathology Study Section within the Oncological Sciences IRG. Dr. Fung comes to us from the National Cancer Institute where he was Program Director for Biological and Chemical Prevention. His Ph.D. is in Chemistry from Stanford University.
Dr. Deborah Hamernik is the new SRA of the Biochemical Endocrinology Study Section in the Endocrinology and Reproductive Sciences IRG. She was previously a Program Director in the U.S. Department of Agriculture. Her Ph.D. is in Reproductive Endocrinology from Colorado State University.
Dr. Angela Ng has become the SRA of the Metabolic Pathology Study Section in the Oncological Sciences IRG. Dr. Ng's Ph.D. is in Pathology from Northwestern University. Immediately before joining CSR, she was Senior Scientist at UroCor, Inc. in Oklahoma City.
Dr. Tracey Orr has just joined CSR as an SRA within the Surgery, Radiology and Bioengineering IRG. Dr. Orr's Ph.D. is in Mechanical Engineering from Stanford University. She was previously Group Leader in Basic and Clinical Biomechanics at the University of Bern, Switzerland.
Dr. Randall Owens is the new SRA of the AIDS and Related Research-3 Study Section within the AIDS and Related Research IRG. Dr. Owens' Ph.D. is in Virology from the University of Alabama at Birmingham. His previous position was Head of Vaccine and Viral Pathogenesis Research at the Southern Research Institute in Frederick, Maryland.
Dr. Sharon Pulfer is now SRA in the Oncological Sciences IRG where she will be responsible for the review of Small Business applications. Dr. Pulfer holds a Ph.D. in Chemistry from the University of Akron (Ohio). She was previously a Postdoctoral Fellow in the Department of Pharmacology, Fox Chase Cancer Center, Philadelphia.
Dr. Charles Rafferty is SRA for the Safety and Occupational Health (SOH) Study Section within the SNEM IRG. This study section is chartered by the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention, but is operated within CSR through an intra-agency agreement. Dr. Rafferty's Ph.D. is in Biophysics from Ohio State University. Immediately prior to joining CSR, he was Senior Manager of the Electromagnetic Force Health Effects Program for the Electric Power Research Institute, Palo Alto, California.
Dr. Ellen Schwartz is the new SRA of the SNEM-1 Study Section within the SNEM IRG. Dr. Schwartz received her Ed.D. in Educational Administration from the University of Rochester. She comes to us from the U.S. General Accounting Office where she was Senior Evaluator/Education Systems Evaluation Specialist.
Dr. Rass Shayiq recently became SRA of the Alcohol and Toxicology Study Sections 1 and 4 within the Pathophysiological Sciences IRG. Dr. Shayiq's Ph.D. is in Biochemistry from Aligarh Muslim University, India. He was previously Director of the Laboratory of Investigative Medicine, Division of Clinical Pharmacology, Thomas Jefferson University, Philadelphia.
Human pluripotent stem cells were first isolated and successfully established as cultured cell lines in 1998. This development sparked considerable excitement for basic research opportunities and for new therapies, because these cells are capable of giving rise to virtually any type of differentiated cell in the body, offering a possible renewable source of cells and tissues to treat a variety of degenerative diseases or injuries.
The achievement has also generated much discussion about ethical concerns. Federal law prohibits NIH from funding work that harms or destroys a human embryo. However, in January of 1999, the Department of Health and Human Services (DHHS) issued a legal opinion that, although DHHS funds cannot be used to derive stem cells from human embryos, the Congressional restriction does not prohibit DHHS from funding research utilizing human pluripotent stem cells because such cells are not embryos. Before allowing such research to utilize federal funds, NIH entered into a long process to develop guidelines, with ample opportunity for public input, to ensure scientific and ethical oversight of this important area of research.
On August 25, 2000, the Federal Register published the NIH Guidelines for Research Using Human Pluripotent Stem Cells (http://www.nih.gov/news/stemcell/stemcellguidelines.htm). These Guidelines describe the documentation and assurances that must be submitted by institutions on behalf of investigators who propose using human pluripotent stem cells in NIH-funded research. To help ensure compliance with the Guidelines, NIH has convened a Human Pluripotent Stem Cell Review Group (HPSCRG) which must review documentation about the derivation of human pluripotent stem cell lines before NIH-funded research can proceed.
Because human pluripotent stem cell research will likely be proposed in areas that fall under the purview of many different Institutes, NIH has decided that the HPSCRG will be a working group of the CSR Advisory Committee (CSRAC). HPSCRG will hold public meetings when a request proposes use of a human pluripotent stem cell line whose derivation has not previously been reviewed and approved. The recommendations of HPSCRG members will then be reviewed by the CSRAC, and if appropriate, approved in its public meetings. NIH will not fund research or allow existing funds to be used for research using human pluripotent stem cells derived from human embryos or human fetal tissue until the derivation protocol has received this approval.
The documentation required by the Guidelines is not part of a grant application. The HPSCRG process will run in parallel to the peer review process for any grant application that proposes using human pluripotent stem cells. Reviewers on CSR study sections may see such applications, and will review the scientific merit of the proposed studies, but they will not be responsible for reviewing compliance with the Guidelines.
Information about HPSCRG is on the NIH web site: (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-00-050.html). An additional notice outlining further details of the process will be published shortly.
Since the original announcement by the National Institutes of Health in December 1998 of Modular Grant Applications, about 25,000 applications have been submitted in the new modular format. Applications have been received for four of the standard investigator-initiated rounds, as well as for many receipt dates for Request for Applications (RFAs) and Program Announcements (PAs). Information on modular grants may be found on the Web at: http://grants.nih.gov/grants/funding/modular/modular.htm.
Many questions, comments, concerns, and inquiries about modular grants have been sent to the e-mail address:modulargrants@nih.gov. NIH has now completed analyzing comments received during the first two review cycles and prepared a response entitled: Modular Grant Applications Update: Peer Review (http://grants.nih.gov/grants/funding/modular/modular_peer_review_update.pdf). This document includes information on the review of modular applications with respect to budget adjustments, overlap issues, preliminary data on the number of modules and direct costs requested, and most importantly, historical data on the pattern of direct cost awards for R01s by budget category. Information on this website will be updated in response to additional questions or as additional information is available.
In response to the many questions, comments, and suggestions received, NIH has also recently modified and clarified some of the original instructions and guidance to applicants and applicant institutions. These modifications can be found at (http://www.grants.nih.gov/grants/guide/notice-files/NOT-OD-00-046.html) and are summarized in the following paragraphs:
An application will be considered non-compliant if: (1) The requested direct cost budget is not in modules of $25,000 for all years of support for requests up to $250,000 per year; (2) A detailed itemized categorical budget is provided; (3) The Budget Narrative Justification page fails to comply with the instructions (Failure to comply includes an itemized justification for one or more of the following: equipment, supplies, travel, other expenses, etc. when the number of modules requested in each year is the same, or the information is not intended to explain the request for a different number of modules in one or more years); (4) Other Support are supplied, in addition to or in the absence of the section in the Biographical Sketch identifying "Research Projects Ongoing or Completed During the Last Three Years"; (5) The Biographical Sketch lists "Current and Pending Support" instead of, or in addition to, the required information.
Editor's Note: NIH will continue to monitor all aspects of the modular grant process and to collect additional information about the process. Following the first two years of implementation, there will be an assessment of the Modular Grant format. Comments or suggestions may be submitted to modulargrants@nih.gov.
A question CSR staff members are frequently asked is: How are the reviewers of my application picked? This brief article attempts to clarify the "how" and also some of the "why" of the reviewer assignment process.
The Public Health Service (PHS) Act authorizes the NIH to fund research grants, but only after a two-stage scientific review process has approved them. The PHS Act also authorizes the Director of NIH, or federal designees, to establish scientific review groups and appoint their members. As the designated federal official, the SRA is responsible for recruitment of members to a study section and assignment of applications to them. Scientific review groups must have a minimum of three members. Scientific review groups at CSR have a minimum of five members for special emphasis panels, but CSR standing study sections have many more. Each application within a scientific review group is reviewed by a minimum of three reviewers.
The PHS Act also directs the designated federal official to prepare a written report of the scientific review for the appropriate Advisory Board or Council. To facilitate the preparation of the written report, and to ensure effective communication of differing views, CSR asks at least two of the assigned reviewers to prepare written critiques prior to the study section meeting and at least one additional reviewer to read and be prepared to discuss the application in depth. For multidisciplinary applications, and often for amended applications, SRAs are encouraged to assign additional reviewers. In practice, inclusion of a third written critique, albeit abbreviated, usually ensures adequate coverage of the science.
The assignment of applications to reviewers is one of the most important responsibilities of the SRA. Assignments should bring appropriate expertise to the review of each application. Assignments should also attempt to ensure that differing scientific perspectives are heard. In addition, assignments must take into account other pragmatic considerations, such as reviewer availability and workload, and avoidance of conflict of interest. Finally, SRAs are encouraged to ensure an appropriate level of interaction among study section members by avoiding assigning too many applications to the same set of reviewers. The SRA may not delegate the assignment of applications, nor can the SRA allow reviewers to self-assign. However, the SRA may seek advice about assignments from the IRG Chief and from the study section Chair, with whom the SRA partners in managing the study section. In fact, the SRA is expected to share the complete assignment list with the study section Chair. After discussions with the Chair, the SRA may, on occasion, consult one or more senior members of the study section to ensure that the assignments are appropriate and that there is adequate expertise. The SRA may also ask individual reviewers whether they are comfortable reviewing a particular application on grounds of scientific expertise or conflict of interest. While different review situations and mechanisms will dictate a variable number of applications assigned to any regular study section member for a regular R01 meeting, a full load tends to be about eight written reviews and four reader assignments.
SRAs are encouraged to complete all of their assignments and mailings of applications at least five weeks before the study section meeting. Within the first few days after reviewers receive their assigned applications, some review adjustments may be made because of latent conflicts of interests or reviewer discomfort with the subject area. Assignment issues can sometimes be resolved by soliciting a mail review, particularly if the scope of the problem is narrow, such as a special technique for a small part of the application. Assignment concerns can usually be addressed smoothly if dealt with immediately after the mailing; however, as the time of the study section meeting approaches, assignment concerns become increasingly difficult to resolve. As a reviewer, you can help guarantee expert and fair review by checking your assignments promptly and letting the SRA know shortly after the mailing if you have a problematic assignment.
Peer review at NIH is carried out both at CSR and at review units within the Institutes and Centers (ICs). CSR review groups and the IC review units follow the same standard NIH review policies, and all use the CSR Division of Receipt and Referral for receipt, initial processing, and assignment of grant applications. Both CSR and ICs also have Scientific Review Groups with standing membership, and both utilize Special Emphasis Panels whose membership is convened for a single meeting.
While there are many similarities, a number of features distinguish CSR from IC review, namely, the types of funding mechanisms usually reviewed, the range of primary IC assignments reviewed in a single study section meeting, and the degree of interaction between SRAs and IC program staff.
CSR reviews primarily those funding mechanisms whose features are standardized across NIH. These features include eligibility criteria, receipt dates, application instructions, budget constraints, and review criteria. CSR reviews most of the investigator-initiated R01 grant applications, as well as the majority of R01s submitted in response to Program Announcements, most SBIR and STTR applications, and most pre- and postdoctoral fellowships.
On the other hand, IC review units (often called Scientific Review Branches), tend to review applications for funding mechanisms with Institute-specific features such as programmatic focus, eligibility criteria, receipt dates, application instructions, budget constraints, and review criteria. These mechanisms include program project grants, training grants, career development awards, and large, multicenter clinical trials. IC review units also generally review applications submitted in response to Requests for Applications (RFAs). RFAs are Institute-specific solicitations published in the NIH Guide for Grants and Contracts (http://grants.nih.gov/grants/guide/index.html) and on IC Web pages located through the NIH Web site (http://www.nih.gov/icd/). For each of these mechanisms, the features may be highly variable from one IC to another.
There are a number of exceptions to the generalization that CSR reviews the trans-NIH applications and ICs review Institute-specific applications and RFAs. Upon request from a particular Institute, CSR will review mechanisms that would typically be reviewed by an IC committee. For example, CSR reviews some program projects for the National Institute of General Medical Sciences and career awards for the National Institute of Mental Health. On a case-by-case basis, if requested, CSR will review an RFA for an IC; RFAs reviewed by CSR tend to be those in which a large number of sponsoring ICs participate. On the other hand, some Institutes review applications that are typically reviewed by CSR. For example, the National Institute of Dental and Craniofacial Research (NIDCR) reviews postdoctoral fellowships and Phase 2 Small Business Innovation Research applications assigned to NIDCR.
IC review units are also solely responsible for the technical merit review of research-and-development contract proposals. The review of contract proposals is quite different from that of grant applications. Contract proposals are received by the Contracts Management Branch of each IC in response to a published Request for Proposals (RFP), which may be highly specific with respect to the nature of the work to be performed and the results to be delivered. In contrast to grant applications, which are given an overall priority score ranging from 100 to 500 (best to worst), contract proposals are scored according to points assigned to each of several evaluation criteria, as described in the RFP, with a maximum of 100 points for all criteria taken together.
With regard to the range of primary Institute assignments within a given study section meeting, each CSR study section generally reviews applications for several of the NIH Institutes. This is because assignment of an application to a CSR study section is based on the type of science proposed, rather than on the Institute to which the application has been assigned. This situation creates a "level playing field" for applications within a given subject matter, regardless of the Institute to which the application is assigned. Because CSR reviews applications for all the ICs, CSR has a much larger number of standing study sections than even the largest IC review unit; CSR's large size gives it economies of scale and a strong infrastructure.
IC study sections, on the other hand concentrate on reviewing applications assigned primarily to that IC. This enables the study section to tailor its service to the needs of that IC. While CSR study sections would have a great deal of difficulty reviewing those grant mechanisms that are used in highly variable ways by different ICs, IC review units, because they work closely with only one IC, are able to deal with this effectively. While a CSR SRA may interact with program staff from many different ICs, IC review staff work primarily with their own IC program staff. Because of the specialized nature of many IC reviews, IC review and program staff collaborate in crafting RFAs and PAs to meet the goals of the Institute, particularly with respect to the instructions to applicants and reviewers. The geographic proximity of IC program and review staffs facilitates this close collaboration.
In summary, CSR and IC review units function somewhat differently; so that between them they provide the range of services needed for peer review at NIH. CSR has the advantages associated with being a large organization with standard operating procedures, while the IC review units are better suited to providing more specialized services. In more important ways, however, CSR and IC review units are fundamentally alike - both operate within the same NIH-wide review policies, and both strive to provide the unbiased and expert reviews that are the cornerstone of the NIH extramural program.