NIH Peer Review Notes

February 1997

Contents

• From the Director's Desk
• DRG Advisory Committee November 1996 Meeting
• Peer Review Oversight Group November 1996 Meeting
• Update on Rating of Grant Applications (RGA)
• Review of Just-in-time (JIT) Applications
• May Applications Designated as Unscored or Not Recommended for Further Consideration Be Funded?
• The Roles of Program Administrators During the Review Process
• Review in Cyberspace
• Presidential Early Career Awards for Scientists and Engineers
• DRG Home Page
• Personnel Update
• Grant Applications Reviewed
• NIH Peer Review Notes Notice
• DRG Office of the Director and Peer Review Notes Editorial Board

This year's turning of the calendar year marked my permanent arrival at DRG, after several months of intermittent coast-to-coast "work-visits." I am very excited to be here and to begin to lead DRG through the continuing processes of evaluation and change needed to accomplish its mission. Our task is to identify the best science that will advance the biomedical research enterprise most effectively. This requires incisive technical and scientific evaluation of applications, as well as recognition of a sufficient diversity of ideas and people to ensure that new directions and areas continue to emerge and develop for the future. DRG's goal is to conduct these activities as efficiently as possible.

The spirit of reinvention has already permeated DRG, and we will continue to analyze our procedures, test new approaches, evaluate and redesign when necessary, and finally implement new practices. Many of these changes will improve the speed and ease of application receipt, referral, and review; some will focus on improving the quality of review. The greater challenge, however, requires consideration of broad scientific issues to ensure that the review process tracks the rapidly moving directions of the science being practiced. We need a careful look at the organization and boundaries of our study sections, and we need enthusiastic participation of top quality reviewers. DRG cannot and must not address these issues in isolation.

The most exciting part of this challenge is the opportunity to form a powerful partnership with DRG, NIH program staff, and the extramural research community. Thus, one of my most important initial efforts will be to engage both the NIH Institutes, Centers, and Divisions and the external scientific community, via professional societies, commissioned panels, and broad individual input, into the discussion and decision-making process. Some useful structures are already in place to facilitate and stimulate communication, such as the DRG Advisory Committee, currently chaired by Keith Yamamoto from the University of California at San Francisco, and the Peer Review Oversight Group (PROG), established last year by NIH Director Harold Varmus. I will also develop additional mechanisms to ensure that the entire spectrum of research activities-- from behavioral science to molecular biology and clinical research, and from microbial pathogenesis to human brain function--is well served.

Maintaining and advancing the existing strength of U.S. leadership in the world's research enterprise requires that we train our young people and provide them with opportunities to contribute, evaluate and identify the best quality and most important programs, and adjust our funding priorities to accommodate a balanced and carefully thought-out mission. Each of these steps is essential. The peer review process is the cornerstone of our entire system, and I am proud and excited to embark upon my new directorship.

- Ellie Ehrenfeld

Return to Top

DRG ADVISORY COMMITTEE NOVEMBER 1996 MEETING

On November 18, 1996, Dr. Keith Yamamoto, the new chairperson of the DRG Advisory Committee (DRGAC), called to order the 15th meeting of the DRGAC. Dr. Donald Luecke, former Acting Director of DRG, officially welcomed the new DRG Director, Dr. Ellie Ehrenfeld, who was introduced by Dr. Ruth Kirschstein, NIH Deputy Director.

As before, DRGAC invited ad hoc advisors to enrich and broaden the focus of the discussions. The discussions during the two day meeting were wide ranging and intense, covering many topics. These topics included, among others: coordination of responsibilities with the new Peer Review Oversight Group ( PROG); pilot study of criterion rating by reviewers; new NIH policy limiting the number of times applications may be amended and resubmitted; recalibration (open window) of the percentile rankings of DRG study sections; review of patient-oriented research grant applications; review of fellowship applications; and integration of review activities in the neurosciences area between DRG and the Institutes. The last topic included presentations by and discussions with the Directors of the National Institute of Neurological Disorders and Stroke (Dr. Zach Hall) and the National Institute on Aging (Dr. Richard Hodes).

The minutes of this meeting will be available soon on the DRG home page --- http://www.drg.nih.gov --- under News & Events. If you also wish to receive paper copies of the minutes, contact the Committee's Executive Secretary, Dr. Samuel Joseloff, at: (301) 435-0691 (phone); (301) 480-3693 (fax); or sj29@nih.gov (e-mail).

Return to Top

PEER REVIEW OVERSIGHT GROUP NOVEMBER 1996 MEETING

The NIH Peer Review Oversight Group (PROG) met for the second time on November 20-21, 1996, in Bethesda, MD. Dr. Wendy Baldwin, NIH Deputy Director for Extramural Research and Chair of the PROG, gave updates on several topics: the Panel on Clinical Research, which met November 5, 1996; the Small Business Innovation Research program, including notice of a meeting scheduled for January 22, 1997, to discuss ways to strengthen that program; and strategies for interim support of research, which were addressed at a meeting on December 4, 1996.

The major topic addressed by the PROG at the November meeting was the Rating of Grant Applications, specifically the use of explicit grant review criteria in the rating of grant applications. For further information, see the article "Update on Rating of Grant Applications (RGA)" in this issue and "Peer Review Oversight Group (PROG) on the Rating of Grant Applications (RGA): Deliberations and Decisions" on the NIH Office of Extramural Research (OER) home page at http://www.nih.gov/grants/oer.htm under Peer Review Issues.

The PROG also heard presentations on the numbers of specific types of grant applications reviewed in DRG and in the Institutes and Centers and on the current effort to reorganize the review of neuroscience applications. For a summary of the presentations and discussion at the PROG meeting, see the meeting minutes on the NIH OER home page under Peer Review Issues. The next meeting of the PROG will take place in Bethesda on February 13-14, 1997.

Return to Top

UPDATE ON RATING OF GRANT APPLICATIONS (RGA)

Both the Division of Research Grants (DRG) and the review branch of the National Institute of Allergy and Infectious Diseases (NIAID) have tested the use of explicit criteria in review, and preliminary results were presented at the PROG meeting in November. The pilots studies examined the use of explicit criteria to structure the written critiques of reviewers and the discussion of the applications at the review meetings. The studies also compared the process of rating the application for each criterion to the customary practice of rating an application for all of the criteria. A summary of these pilot studies and the deliberations of PROG about the results are in the PROG November 20, 1996 meeting minutes on the NIH OER home page (address indicated above). There appear to be reservations about scoring by individual criterion, but the use of explicit criteria to structure review activities is being viewed quite favorably. Decisions regarding the use of criteria, what they will be, and specific definitions will be made early in 1997.

Return to Top

REVIEW OF JUST-IN-TIME (JIT) APPLICATIONS

JIT budget request should include narrative information about the personnel needed to conduct the proposed project and any unusual budget expenses. The applicant is instructed to identify all personnel (paid and unpaid) and justify their roles in and level of effort on the project. All proposed consultants need to be identified by name, organizational affiliation, and services to be performed. Although specific costs are not required to be shown, any major budget item, e.g., equipment (especially those that would be considered unusual for the scope of the work) must be justified. Furthermore, a budget justification is required for contractual/consortium costs. Finally, the budget request should be appropriate for the scope of the proposed research.

Using narrative not numbers, reviewers are expected to recommend a reasonable budget. They should assess the personnel, their roles in the project, and the appropriateness of their efforts. If consultants, major budget items, contracts/consortiums, and/or the amount of the total budget are not justified, reviewers should explain the basis of their opinion and recommend adjustments where appropriate.

Reviewers are expected to assess the qualifications of the investigators to conduct the proposed work. The JIT application should contain a modified biographical sketch that includes information directly relevant to the project on educational background, current and previous positions, selected peer-reviewed publications with full citations, and information about current and recent research projects (last 5 years). The inclusion of descriptions of other projects in the biographical sketch is new and very important because the Other Support pages are not submitted at the time of initial application.

Because the Other Support section is not completed in a JIT application, some reviewers may experience difficulty assessing overlap with other projects. Overlap is an administrative issue that can be handled by reviewers raising the possibility of overlap in an administrative note, a helpful flag to NIH program staff. However, overlap can be useful information for reviewers in the assessment of innovation and impact. The information needed to assess intellectual overlap should be available in the biographical sketches. If reviewers need more information to assess the application, they should notify the Scientific Review Administrator of their review group, who is the contact for all matters relating to initial review.

Return to Top

MAY APPLICATIONS DESIGNATED AS UNSCORED OR NOT RECOMMENDED FOR FURTHER CONSIDERATION BE FUNDED?

Grant applications assigned priority scores by the scientific peer review groups (SRGs) are considered to be "recommended by an initial review group" and can be funded with appropriate council concurrence. An application rated Not Recommended for Further Consideration (NRFC) may not be considered for funding because, by definition, it has been assessed as lacking significant and substantial merit and not recommended by the initial review group.

Applications that were "unscored" because they fell into the lower half of those being reviewed may be considered to have been "recommended" in initial review unless the summary statement explicitly states otherwise. If an Institute wants to fund an unscored application, the action must be considered by the Advisory Council for special action. A compelling reason for making an award must be presented. In practice, this action is a very rare event. In fiscal year 1996 no new or competing unscored R01 or R29 application was funded by this process . Nevertheless if a reviewer considers an application as part of the streamlined review process to be without significant and substantial merit, the reviewer's written critique should explicitly state this and indicate why. Such a declaration in a summary statement would effectively prohibit funding.

Return to Top

THE ROLES OF PROGRAM ADMINISTRATORS DURING THE REVIEW PROCESS

To appreciate the role of program administrators at review meetings, it is important to understand that NIH requires separation of extramural staff functions to ensure fairness and objectivity in the review process. NIH staff consisting of the Scientific Review Administrator (SRA), the Grants Technical Assistant (GTA), the Program Administrator, and the Grants Management Specialist have important and complementary roles and responsibilities in managing the grants process and in ensuring the proper stewardship of Federal funds. Each member of the NIH team is responsible for work that is essential to NIH making well reasoned funding decisions. This article will address the role of the program administrator, especially during the review process.

The program administrators' role before the review meetings is more obvious, since they may be actively advising potential applicants about funding opportunities, developing program initiatives, and providing the SRA with background information relevant to the review.

Program staff observe the review of applications for which they or their Institute are responsible to better understand and interpret the resultant summary statements. This information will benefit subsequent National Advisory Council discussions, funding decisions, grants management, and discussions with principal investigators regarding resubmissions or NIH policy. At the review meeting, program representatives may provide descriptive background information on the funding history of an application, but should coordinate this with the SRA. If program representatives perceive the need to provide input, they are to approach the SRA before addressing the reviewers directly. The SRA (or Chair, with the approval of the SRA) may call upon the program representative to provide non-evaluative information or clarification as needed. No staff (program or SRA) may make evaluative statements about the application or the prior productivity of the project or investigators at the review meeting.

After the review meeting, the program administrator becomes the primary contact for the applicant, serves as a scientific resource to the Institute Advisory Council, and is directly involved with the funding process.

Throughout the application review process and the life of the grant, NIH staff interaction is defined by the roles and responsibilities established for those serving in these positions. These roles reflect a balance of cooperation and independent responsibilities, and a balance that changes at various stages of the grant process. During the review process, this balance is intended to ensure fair and objective review, resulting in quality funding decisions.

Return to Top

REVIEW IN CYBERSPACE

Geoff Grant of the NIH Office of Extramural Research and Donna Dean of the Division of Research Grants were kickoff speakers, highlighting the coming innovations in information dissemination and considerations of the human factor in peer review. Carolyn Miller and Gloria McCabe described the National Science Foundation's new FastLane on-line application and review process, both the required internal logistical support and the applicant and reviewer acceptance of the new process. NIH's first experience in electronic review of grant applications is being carried out by the National Institute of Allergy and Infectious Diseases (NIAID) review units. NIAID's Madelon Halula and Paula Strickland reported on their pilot experiments, while Alex Rosental described the WEB accessibility and hardware standards needed to support this electronic review approach. The final presentation of the afternoon was an overview of emerging technologies by Charlie Havekost of the NIH Division of Computer Research and Technology.

Peer review currently involves a number of people engaged in here-and-now, face-to-face debate over the merits of an application, followed by a vote. The forum participants tackled such issues as: To what extent can electronic information handling augment or even substitute for the current review process? How soon might this occur? Would the likely cost savings come at an unacceptable price - the quality of review? How should NIH Institutes and their staff prepare for possible technological and philosophical changes? Charlie Havekost's concluding comment formed a highly appropriate take-home message for forum participants, as they contemplated the issues and concerns discussed: "Try to understand the changes and develop a personal infrastructure for the unanticipated."

Return to Top

PRESIDENTIAL EARLY CAREER AWARDS FOR SCIENTISTS AND ENGINEERS

The Presidential Award embodies the high priority of the Government on maintaining the leadership position of the United States in science by producing outstanding scientists and engineers and nurturing their continued development. The Award recognizes some of the finest scientists and engineers, who early in their research careers, show exceptional potential for leadership at the frontiers of scientific knowledge during the twenty-first century.

The NIH selected its 1996 Presidential Awardees from the most meritorious investigators funded by NIH FIRST awards (R29) and traditional research project grants (R01). Of the R01s, only first-time recipients who also met the eligibility criteria for the FIRST award were considered. The 1996 Presidential Awardees selected by the NIH, their affiliations, and the funding Institutes or Centers are the following:

Ali Hemmati-Brivanlou, Ph.D.
Rockefeller University
National Institute of Child Health
and Human Development

Allison Jane Doupe, M.D., Ph.D.
University of California, San Francisco
National Institute of Mental Health

Paul Khavari, M.D., Ph.D.
Stanford University
National Institute of Arthritis
and Musculoskeletal and Skin Diseases

Aron Lukacher, M.D., Ph.D.
Emory University
National Cancer Institute

Deirdre Meldrum, Ph.D.
University of Washington
National Human Genome Research Institute

Lee Niswander, Ph.D.
Sloan-Kettering Institute for Cancer Research
National Institute of Child Health
and Human Development

David Self, Ph.D.
Yale University
National Institute on Drug Abuse


Morgan Sheng, Ph.D.
Massachusetts General Hospital
National Institute of Neurological
Disorders and Stroke

Mark Walter, Ph.D.
University of Alabama at Birmingham
National Institute of Allergy
and Infectious Diseases

Keith Woerpel, Ph.D.
University of California, Irvine
National Institute of General Medical Sciences

Approximately 60 Presidential Awards were presented at a ceremony in Washington, D. C., on December 16, 1996.

*National Science Foundation; National Aeronautics and Space Administration; Environmental Protection Agency; Department of Veterans Affairs; Department of Energy: Energy Research Programs, Defense Programs; Department of Defense: U.S. Air Force, U.S. Army, U.S. Navy; Department of Agriculture: National Research Initiative, Agricultural Research Service, Forest Service; and Department of Commerce: National Oceanic and Atmospheric Administration, National Institute of Standards and Technology.

Return to Top

DRG HOME PAGE

Return to Top

PERSONNEL UPDATE

Appointments:
Mr. Geoffrey Grant, Director, Office of Policy for Extramural Research Administration, Office of Extramural Research, Office of the Director

Dr. Charles Hollingsworth, Deputy Director, Office of Review, National Center for Research Resources

Dr. Thomas Puglisi, Director, Division of Human Subjects Protections, Office for Protection from Research Risks, Office of the Director

Retirements:
Ms. Eleanor Friedenberg, Director, Office of Extramural Program Review, National Institute on Drug Abuse

Dr. Marilyn Semmes, Acting Chief, Scientific Review Branch, National Institute on Deafness and Other Communication Disorders

DRG

Appointments:
Dr. John Bowers, Scientific Review Administrator (SRA), Biophysical and Chemical Sciences Initial Review Group (IRG), Metallobiochemistry Study Section, Referral and Review Branch (RRB)

Retirements:
Ms. Patricia Bailey, Chief, Administrative Services Office, Office of the Director (OD)

Ms. Jean Gunton, Administrative Officer, Administrative Services Office, OD

Dr. Lynwood Jones, Jr., SRA, Immunological Sciences IRG, Immunology, Virology, and Pathology Study Section, RRB

Dr. Paul Parakkal, SRA, Surgery, Radiology, and Bioengineering IRG, Surgery and Bioengineering Study Section, RRB

Mr. Nick Suszynski, Associate Director for Advanced Computer Technology, OD

Departures:
Dr. Lillian Pubols, SRA, Neurological Sciences IRG, Neurology B-1 Study Section, was recently appointed Chief, Scientific Review Branch, National Institute of Neurological Disorders and Stroke

Dr. Jerry Roberts, SRA, Genetic Sciences IRG, Mammalian Genetics Study Section, recently joined the new Center for Inherited Disease Research, National Human Genome Research Institute

Dr. Jules Selden, SRA, Health Promotion and Disease Prevention IRG, Alcohol and Toxicology 2&4 Study Sections, recently transferred to the Division of Basic Research, National Institute on Alcohol Abuse and Alcoholism

Return to Top

GRANT APPLICATIONS REVIEWED

From January 1993 to January 1997, the total number of grant applications reviewed by NIH for those council cycles increased 5 percent, from 11,779 to 12,346. The total direct costs requested in applications for research grants increased 16 percent, from $1,647 million in January 1993 to $1,913 million in January 1997.

                                        January     January
                                           1997        1993



Applications reviewed....................12,346      11,779 
  DRG.....................................9,021       8,425
  Institutes/Centers......................3,325       3,354

Research grant applications..............11,054      10,477
  Research projects.......................8,267       8,007
  Small business/Technology transfer..... 1,158         950
  Research centers..........................129          91
  Other research..........................1,500       1,429

Training applications.....................1,258       1,256
  Fellowships...............................875         904
  Training grants...........................383         352 

Other applications...........................34          46

Applications amended......................3,855       3,180   
  Percent of total number reviewed...........31          27

Applications responding to RFAs.............561         683

Return to Top

NIH PEER REVIEW NOTES NOTICE

http://www.drg.nih.gov/prnotes/prnotes.htm

For those individuals who do not have access to the Web but have access to E-mail, an ASCII or WordPerfect 6.1 copy can be E-mailed as an attachment. To request a copy, or if you have questions concerning NIH Peer Review Notes, please send an E-mail message to drginfo@drg.nih.gov or call (301) 435-0688.

Return to Top

DRG OFFICE OF THE DIRECTOR AND PEER REVIEW NOTES EDITORIAL BOARD

Dr. Ellie Ehrenfeld
Director

Dr. Anthony E. Demsey
Acting Deputy Director

Dr. Donna J. Dean
Acting Chief, Referral and Review Branch

Dr. Samuel H. Joseloff
Assistant Director for Special Projects

Mr. John Jones
Acting Executive Officer

Peer Review Notes
Editorial Board

Chairperson
Dr. Samuel C. Rawlings
Coordinator, Sensory Sciences Initial Review Group
Referral and Review Branch
Division of Research Grants

Dr. Claudia Blair
Director, Extramural Outreach and Information Resources
Office of Extramural Programs
Office of Extramural Research
Office of the Director, NIH

Dr. Mark R. Green
Chief, Extramural Projects Review Branch
National Institute on Alcohol Abuse and Alcoholism

Mr. Robert Moore
Project Leader, Office of Reports and Analysis
Office of Extramural Research
Office of the Director, NIH

Ms. Jo Pelham
Scientific Review Administrator,
Geriatrics and Rehabilitation Medicine Study Section
Referral and Review Branch
Division of Research Grants

Dr. Michael E. Rogers
Director, Pharmacology, Physiology, and Biological Chemistry Division
National Institute of General Medical Sciences

Editor

Mr. Nicholas Moriarty
Program Analyst, Information and Analysis Office
Office of Administrative Management
Division of Research Grants

Last Revised: May 9, 1997

Return to Top