RECOMMENDATIONS FOR CHANGE AT THE NIH’S
CENTER FOR SCIENTIFIC REVIEW

PHASE 1 REPORT
PANEL ON SCIENTIFIC BOUNDARIES FOR REVIEW
DRAFT

(Currently under revision)

EXECUTIVE SUMMARY

The Panel on Scientific Boundaries for Review is conducting a comprehensive examination of the organization and function of the review process carried out by the Center for Scientific Review (CSR) at the National Institutes of Health (NIH). The purpose of this evaluation is to position the CSR peer review system to foster the expanded research opportunities created by the stunning successes of the biomedical research enterprise, as well as to permit the review system to keep pace with the accelerating rate of change in the way biomedical research is now performed.

This examination is being carried out in two phases. In Phase 1, to be completed in November 1999, our Panel is developing a set of Integrated Review Groups (clusters of scientifically related study sections, referred to as IRGs) that have been designed to facilitate the review of contemporary scientific areas and opportunities, and thereby contribute to the translation of progress in the basic science laboratory into progress at the bedside, as well as to move progress from the bedside to the laboratory bench. In addition, the revised structure should help CSR anticipate emerging fields of research and accommodate the rapid pace of scientific change. In designing the proposed set of IRGs, we were guided by four principles: (i) There should be a home for the review of all science that is relevant to contemporary biomedical research. (ii) The research topics encompassed by each IRG should be sufficiently cohesive to allow the external advisory group of scientists for that IRG to judge its entire scope of science. (iii) The research related to a given system or disease, including fundamental studies, should be clustered for review within a single IRG or a related set of such IRGs. (iv) The organization should be flexible enough to adjust to the rapid changes in scientific opportunities expected in the years ahead.

In addition to designing a set of IRGs, we have also outlined some cultural norms that we believe should govern the CSR review process. In Phase 2, which will begin in 2000 and is likely to continue through the next two years, expert groups of extramural scientists and NIH staff will create the scientifically related study sections that will populate each IRG on the basis of principles outlined in this report. Recommendations will be implemented slowly over the following years, with extensive involvement of the extramural research community.

In Phases 1 and 2 we are also relying heavily on input from the broad scientific community, both within and outside the NIH, to develop final recommendations. Thus, before completing our Phase 1 report, the Panel is seeking comments on this draft, in which we present our view of the goals of an optimal review process for selection of the highest quality research grant proposals; a proposed new IRG organization; some cultural norms according to which study sections and IRGs should operate; and the procedures and principles planned for Phase 2.

INTRODUCTION

The NIH peer review system is designed to ensure that the resources provided by the public for the support of biomedical research are allocated as the result of a fair and rigorous competition among scientists. The long-range purpose of this research is to develop knowledge that will add, both directly and indirectly, to the improvement of human health. At the same time, research supported by the NIH plays a critical role in training the next generation of biomedical scientists.

NIH's peer review system is recognized as the cornerstone of the NIH extramural program, because it is the principal mechanism by which the Institutes and Centers identify high quality research that is worthy of funding. Established over 50 years ago, the system has been outstandingly successful, and, in fact, may be the most important single reason for the remarkable success of our federally funded biomedical research enterprise. During these 50 years, NIH peer-reviewed research has fueled the acquisition of basic knowledge about every aspect of biomedicine. Application of this knowledge to human health issues now advances at an accelerating rate, profoundly affecting medicine. These stunning successes have resulted in an outpouring of data and have given rise to promising new technologies, causing the way we do science to change and expanding the opportunities for the NIH to fully achieve its mission to improve human health.

The Center for Scientific Review (CSR) manages the peer review process for the majority of the grant applications submitted to the NIH.¹ (Note: to see endnotes, click on superscript numbers.) Since its establishment, the CSR peer review system has evolved continuously. Currently, a CSR Advisory Committee is playing a key role in advising the Director on all aspects of CSR function, and has initiated a number of activities to improve the peer review process at CSR (see Appendix II). In addition, the Peer Review Oversight Group (PROG) was established in 1996 to provide advice regarding the NIH-wide peer review process (http://www.nih.gov/grants/peer/peer.htm#prog). However, the rapid progress in biomedicine and its accelerating rate of change now challenge the CSR review system to keep pace.

Through extensive outreach to the extramural research community, as well as through her own assessment, CSR’s Director, Dr. Ellie Ehrenfeld, has identified a number of issues regarding study section organization and composition. While these are subjective impressions that are hard to document, they are worthy of concern and sufficiently common to ask whether there are better ways to organize the review process. For example, researchers perceive that there are no appropriate study sections for many newly emerging fields; that applications describing some of the most productive, highest impact work may be assigned to too few study sections, causing too much of the "best science" to compete with itself; that the scope of some study sections is restricted to research with relatively low impact, resulting in undeserved "entitlements;" and that the breadth of knowledge needed to assess the importance and potential impact of research proposals may often be sacrificed in composing narrowly focused review committees. In addition, certain segments of the research community, including clinical researchers, behavioral scientists, bioengineers, and developers of technology and instrumentation, believe that they are inadequately served by the existing system.

Many researchers fear that conservatism in the system and an undue requirement for preliminary data discourage innovation. They also note the need to define best practices and to institute procedures to enable them to be applied consistently by peer reviewers, study section chairs, and Scientific Review Administrators (SRAs).

Recently, to establish better opportunities for teamwork, a flexible distribution of applications, and a sharing of reviewer expertise believed to be most appropriate for the review of today’s science, the integrated review group (a cluster of scientifically-related study sections, referred to as IRG and previously called an initial review group) has replaced the individual study section as the functional unit of review.² For example, seven study sections review applications related to various aspects of molecular, cellular, and developmental neuroscience; all are organized into one IRG, much as multiple courses on related subjects are organized into a single academic department’s curriculum. External advisory groups are being established for each IRG to provide expert input for the continual improvement of its review processes. However, there has been no overall assessment as to whether the present IRGs and study sections are properly configured to respond to current and future research opportunities, so as to best promote the long-range health goals of society.

For this reason, our Panel on Scientific Boundaries for Review (see Appendix III for the roster of our Panel) was established in April 1998 as a working group of the CSR Advisory Committee to undertake a comprehensive examination of the organization and function of the CSR review process. Our Panel is conducting its evaluation in two parts. In Phase 1, to be completed in November 1999, the Panel is proposing a set of IRGs that are designed to optimize the CSR review process, as well as producing a list of cultural norms that we believe should govern the operation of the CSR review process. Here, we have benefited from a number of ongoing initiatives and recent changes that have established a foundation upon which we can build more comprehensive reforms (described in Appendix II). In Phase 2, which will begin in 2000 and is likely to continue through the next two years, expert groups of extramural scientists and NIH staff will create the scientifically based study sections that will populate each IRG. Recommendations will be implemented slowly over the following years, with extensive involvement of the extramural research community.

For both Phase 1 and 2, we will be relying heavily on input from the broad scientific community, both within and outside the NIH, to develop our final recommendations. At this time, we seek a critical review from the community as we develop our final recommendations for an organization of IRGs that provides a suitable home for the review of all areas of research.

This Phase 1 report begins with our view of the goals of an optimal review process for identification of the highest quality research grant proposals (Section I). The report then continues with a proposed new IRG organization (Section II and Appendix I); some cultural norms according to which study sections and IRGs should operate (Section III); and an outline of the procedures and principles planned for Phase 2 (Section IV).

I. GOALS OF THE RESEARCH GRANT REVIEW PROCESS

In addition to making efficient use of time and resources, CSR must continue to ensure that the following criteria for an optimal review process are met:

Set high standards: The review process should continue to set a high standard for scientific excellence, facilitating the selection of the best scientists and ideas on any given topic for NIH support.
Contribute to the advance of biomedical science: The peer review process should optimize the benefits that can be gained from the progress of science and its contributions to health.
Encourage innovation and risk taking: When knowledge is generated and combined in new and unexpected ways, discoveries are made that can advance biomedical research and dramatically improve human health.
Exercise fairness: Judgments made in distributing limited resources are difficult, both for those who make them and for those being judged. There is unanimous support for the principles of peer review, despite the inherent complexity in executing these principles. For scientists and the public to maintain confidence that the NIH review system is fair, there must be clarity about the means by which reviewers and staff are chosen, trained, and held accountable.
Be monitored continuously: To ensure that it is responsive to the rapid changes that energize and renew biomedical research, the system should be reassessed at regular intervals. Such examination should be conducted not only at the level of the IRG, via external advisory groups as planned, but globally by the CSR Advisory Committee based on input from applicants, reviewers, and NIH staff.
Be clearly explained: The process must be clearly explained so as to be understood not only by scientists but also by the general public, who need to comprehend both the substance of biomedical science and the selection processes that are used to fund it.

II. PROPOSED STRUCTURE OF INTEGRATED REVIEW GROUPS

A. Guiding Principles

In designing the proposed set of IRGs described below, we have been guided by the following general principles:

There should be a home for the review of all science that is relevant to contemporary biomedical research.
The research topics encompassed by each IRG should be sufficiently cohesive to allow the external advisory group of scientists for that IRG to judge its entire scope of science.
The research related to a given system or disease, including fundamental studies, should be clustered for review within a single or related IRGs.
The organization should be flexible enough to adjust to the rapid changes in scientific opportunities expected in the years ahead.

B. Proposed IRGs

A wealth of experience and documentation has made it clear that clinical advances often rest on the results of basic research. To ensure a vigorous foundation for future progress, NIH must continue to support a sizable proportion of research that has no immediate or specific application to human health. Much of this research must be reviewed in a fundamental context, without regard to a specific organ, biological system, or disease. Therefore, we recommend formation of multiple IRGs for review of such applications. However, the members of the Panel also believe that, whenever reasonable, basic research that more directly underlies clinical or applied studies on specific diseases, organs, physiological systems, or general health problems, should be reviewed within the broader biological and medical context to which it will ultimately be applied. Thus, we have attempted to place the review of as much fundamental research as possible in the IRG that is most relevant.

As a result, all types of research related to a given system or disease will be clustered in an IRG devoted to that system or disease. For example, the IRG for cardiovascular sciences would include basic studies of heart and vessel development and physiology, studies of pathophysiology of the heart and vasculature, and clinical studies pertaining to specific cardiovascular diseases and their treatment. However, within the IRG, individual study sections may be designed to cluster similar types of scientific approaches (e.g., molecular projects versus patient-oriented studies) or to integrate such studies as is deemed appropriate for the field (See Section IV). Thus, in the present system, an application proposing an investigation of the detailed mechanisms that cause specific genes to respond to hormonal ligands would likely be reviewed in the Cell Development and Function IRG. In our system, the application would likely be reviewed in a study section in the Endocrinology, Metabolism, and Reproductive Sciences IRG, albeit one with a molecular focus.

The rationale for this decision is as follows.

The overarching mission of NIH to improve human health will be best served by reviewing clinically relevant science in the context of the basic knowledge on which it is founded, and by reviewing basic science in the context of the human condition that it is designed to improve.

The new opportunities created by research require that the new knowledge from basic studies be applied to disease problems in a timely and appropriate way, so as to translate progress in the basic science laboratory into progress at the bedside. Likewise, a strong communication link is needed between those engaged in human disease research and basic scientists, in order to help direct new efforts aimed at understanding disease processes through fundamental research. This critical interaction can be fostered by ensuring a high level of communication between basic and disease-oriented researchers in the review processes at NIH.

Within an IRG, the exchange of ideas and perspectives among reviewers who work on the same biological system or group of diseases, but who have very different skills and points of view, may help to promote identification of more ambitious and interdisciplinary research proposals and may encourage the submission of broader and more innovative research proposals to the NIH.

The review of basic and applied science in the same IRG will provide an overview of the quality of the work evaluated across all of its study sections. This will enable NIH Institute staff to monitor the relative scientific merits of fundamental and clinical research continually to ensure that it responds to changes in the relative opportunities in a given field, as well as to the relative quality of the investigators available to conduct the work.

Finally, the new structure will ameliorate a difficulty with the current situation, in which many investigators who are using a particularly powerful methodology (e.g., using yeast as a model system) tend to compete with one another based on the methodology, rather than competing in different study sections and IRGs on the basis of the potential (and often very different) applications of the proposed science.

Our Panel recommends the following 21 IRGs, beginning with the 5 that are concerned with the fundamental biological processes or technologies that underlie approaches to the properties and treatments of all organ systems, disease processes, and general health issues. A fuller description, including representative topics to be covered within each IRG and the relationship between the current and proposed IRGs, is provided in Appendix I. As new areas emerge in the future, the IRG system must remain flexible so as to accommodate them.

CHEMICAL BIOLOGY AND BIOPHYSICS IRG - This IRG will consider research applications focused on the detailed structures, chemistry, and physics of macromolecules and interacting small molecules.
MOLECULAR APPROACHES TO GENE FUNCTION IRG - This IRG will consider research applications focused on the molecular mechanisms and regulation of the global processes of gene expression that are fundamental to all living cells.
MOLECULAR APPROACHES TO CELL FUNCTIONS AND INTERACTIONS IRG - This IRG will consider research applications at the level of the functions, interactions, and regulation of cells and cellular organelles, focusing on fundamental cell biological processes.
FUNDAMENTAL GENETICS AND POPULATION BIOLOGY IRG - This IRG will consider research applications focused on general issues in genetics, genomics and population dynamics.
FUNDAMENTAL BIOENGINEERING AND TECHNOLOGY DEVELOPMENT IRG - This IRG will consider research applications to develop fundamental, broadly applicable new methodologies and instrumentation.
HEALTH OF THE POPULATION IRG - This IRG will consider research applications focused on broad social, environmental, cultural, and other contextual influences on health behavior.
RISK, PREVENTION, AND HEALTH BEHAVIOR IRG - This IRG will consider research applications focused on risk, prevention and health in the individual; studies of risk and protective processes, including genetic risk factors; and studies of interventions aimed at reducing these risks.
BEHAVIORAL AND BIOBEHAVIORAL PROCESSES IRG - This IRG will consider research applications focused on cognitive, perceptual, intra- and interpersonal processes, studies of normal and disordered movement, and studies of developmental, psychopathological and substance use disorders.
IMMUNOLOGY IRG - This IRG will consider research applications focused on basic immunology and diseases that are principally immunological in their origin or manifestations.
INFECTIOUS DISEASES AND MICROBIOLOGY IRG - This IRG will consider research applications focused on studies of infectious diseases, pathogenesis, prevention, and treatment.
ONCOLOGICAL SCIENCES IRG - This IRG will consider research applications focused on cancer, spanning pathogenesis to clinical trials of therapeutics.
HEMATOLOGY IRG - This IRG will consider research applications focused on studies of blood cells and their diseases, and studies of the coagulation system and its pathology.
CARDIOVASCULAR SCIENCES IRG - This IRG will consider research applications focused on basic studies of heart and vessel development and physiology, studies of pathophysiology of the heart and vasculature, and clinical studies pertaining to specific cardiovascular diseases and their treatment.
ENDOCRINOLOGY, METABOLISM, AND REPRODUCTIVE SCIENCES IRG - This IRG will consider research applications focused on endocrine and reproductive systems and associated diseases, as well as basic intermediary metabolism and metabolic disorders.
BONE, MUSCLE, CONNECTIVE TISSUE, AND SKIN IRG - This IRG will consider research applications focused on studies of connective tissues, which include bone and dental tissues, skeletal muscle, skin, and the extracellular matrix, and their associated diseases.
DIGESTIVE SCIENCES IRG - This IRG will consider research applications focused on studies of the entire gastrointestinal tract and related organs.
PULMONARY SCIENCES IRG - This IRG will consider research applications focused on the basic studies of the lung and respiratory system development and physiology, studies of pathophysiology of the lung and respiratory system, and clinical studies pertaining to specific pulmonary diseases and their treatment.
MOLECULAR, CELLULAR, AND DEVELOPMENTAL NEUROSCIENCE IRG - This IRG will consider research applications focused on the basic mechanisms that determine the structure and function of neurons, glia, and other excitable cells, and aspects of development in both the central and peripheral nervous system.
INTEGRATIVE, FUNCTIONAL, AND COGNITIVE NEUROSCIENCE IRG - This IRG will consider research applications focused on how the nervous system is organized and functions at an integrative, systems level.
BRAIN DISORDERS AND CLINICAL NEUROSCIENCE IRG - This IRG will consider research applications focused on disease and injury to the nervous system, including issues of neural substrate, functional consequences, and rehabilitation.
SURGERY, APPLIED IMAGING, AND APPLIED BIOENGINEERING IRG - This IRG will consider research applications focused on surgical and related disciplines and on applied imaging and applied bioengineering.

III. CULTURAL CHANGES

In addition to optimally organizing IRGs and study sections, the Panel believes that the key to maximizing the health of peer review in CSR lies in instilling in the applicants, reviewers and administrators a shared perception of their roles and responsibilities: a broad agreement -- in principle and in practice -- concerning not only what reviewers should be doing, but also how they should be doing it. Below, our Panel recommends a set of cultural norms according to which we believe study sections and IRGs should operate. Adopting them need not await the proposed reorganization. Rather, the system could benefit from implementing them now. The Panel notes that to effect the proposed changes in culture, the NIH review and program staff, as well as the reviewers themselves will need to be involved.

Who is a peer? Peer review is based on the idea that active research scientists who are familiar with the kind of research being proposed are the best judges of the scientific merit of a research proposal (including the quality and originality of the ideas, the level of craftsmanship, the likelihood of success, and the eventual impact on medicine, science, health, and society). The Panel believes that an appropriate peer reviewer is an active researcher, who is fully aware of (or can easily learn about) both the research goals and the research means being proposed. Peer reviewers thus need not be scientific "competitors" of the applicant, or even be studying the same disease or organ system. But they should be experienced researchers who are reasonably diverse in seniority, outlook, geographical location, gender, and ethnicity and who have achieved recognition for their research accomplishments. Such recognition may include successful competition for research grants, but will vary depending on whether they work in academia, industry or government-sponsored intramural laboratories, or are, for example, biologists, engineers, or physical scientists.

What is the role of a peer reviewer? The Panel believes that a peer reviewer's only role is to judge the research proposed, providing the Institutes with honest and informed advice about the merits of the proposals under review. Advocacy or gate keeping for a field, discipline, or style of research is not the function of a peer reviewer. Thus the Panel urges that every effort be made to recruit the wisest and most knowledgeable peer reviewers, and to instruct them that their only function is to judge the research proposals on their merits. Any residual confusion about the role of reviewers in keeping fields alive, or in helping researchers succeed, should be eliminated. These functions may be important to the overall scientific enterprise, but they are inappropriate to the review process and should be pursued separately.

What are the roles and responsibilities of the Scientific Review Administrators (SRAs), Chairs, and study section members? The chair and the SRA are partners with complementary roles and responsibilities in the peer review system; both are essential if the system is to work well. The SRA (as the Designated Federal Official in charge) is responsible for managing the proceedings, ensuring that they are fair and thorough, and maintaining the integrity of the process. The composition of the study section, the assignments of grants for review, and the final summary statements are by statute the responsibility of the SRA. However, each of these activities should be conducted in close partnership with the chair of the study section, who has major responsibility for ensuring that the study section members determine scientific merit accurately and fairly, and for the substance of the judgments of the grant applications. Chairs must therefore be very carefully chosen for their leadership abilities and breadth of knowledge, and both SRAs and chairs must be clearly instructed with regard to their roles and responsibilities.

What is the appropriate relationship between study sections and disciplines? There are true disciplines (e.g., biochemistry, genetics, physiology, statistics.....) that have come to underlie and be deeply entwined in all kinds of research. But the Panel believes that fostering the health and propagation of disciplines is not the function of NIH, but rather of universities and other extramural research institutions. In its proposals for reorganization, the Panel recognizes the need for well-informed biochemists, geneticists, physiologists, and statisticians on many diverse peer review groups. Although there will be valid reasons for technology development in particular disciplines, the need for "disciplinary" study sections as such may well have passed. The review system must emphasize that the NIH supports research, not disciplines or fields. Making review groups suit the nature of an applicant's research, irrespective of his/her discipline or field, should be a continuing goal of CSR and the NIH peer review system.

What types of research have the potential to have an impact on the ability of the NIH to achieve its mission? If NIH is to accomplish its full mission, all research styles must be judged on their potential to advance our understanding of biological systems, to improve the prevention and treatment of disease, and to enhance the health and well being of people. Certainly, the development and application of new methods and technologies have been critical to the advance of biomedical science. However, peer review of NIH research grants has traditionally emphasized the testing of hypotheses. Although "hypothesis-driven" research could be interpreted broadly to include all the styles of research - including generating the knowledge needed to solve important problems and developing novel techniques or instrumentation to enable knowledge generation -- the practice of NIH study sections has been to interpret it narrowly as a formal exercise in the proposal and proof of a well circumscribed idea. Under these conditions, exploratory research using and developing new technology suffers, and opportunities for generating new knowledge and ideas are thereby lost. Such an exclusionary insistence on hypothesis-driven research can impede the ability of NIH to accomplish its broad charge.

What should a grant application propose? The American public expects the NIH to seek out ideas and knowledge that can, directly or indirectly, improve human health. Thus, the NIH must support a broad portfolio of grants ranging from discovery projects that offer important evolutionary advances through high risk/high reward efforts with the potential for revolutionary changes. However, the Panel is concerned that in practice the present system tends to discourage risk taking and to undervalue new ideas. We urge that reviewers endorse the importance of ideas that are original and have yet to be tried. Peer reviewers should eschew the common current tendency to find fault or to identify minor errors. Instead, they should strive to assess the potential impact of the proposed research and to encourage good ideas and novel concepts, even if they appear to be risky. Countering the conservatism of the peer review system is a critical issue that should become a primary long-term focus for the CSR.

What perspective should be used in review, and how should the results of the review be communicated? All members of study sections should judge grant applications on their fundamental merits (soundness and originality of concept, potential impact on the fields, technical and intellectual sophistication, quality of the investigator, and appropriateness of approach), without undue emphasis on minor technical details. Although it may be appropriate for peer reviewers to provide some helpful general advice on ways to improve the application, they should not be in the business of educating the applicants, let alone designing their next experiments or grant applications. If a proposal is poor because, for example, it is likely to have little or no impact, it should receive a poor score. The summary statement should convey the rationale for the score to the applicant, Institute staff and advisory councils. It should not be a detailed recounting of errors nor a compendium of redesigned experiments for a future amended application. Adherence to this recommendation would simplify the task of reviewers, who must provide enough justification to satisfy the other study section members.

What is the role of preliminary data? Although preliminary data can reassure reviewers that the applicant has the means and the understanding needed to carry out the proposed studies, the Panel cautions that an obsession with preliminary data discriminates against bold new ideas, against young scientists, and against risk taking. For new ideas, little or no preliminary data may be required.

IV. PROCEDURES AND PRINCIPLES TO BE FOLLOWED IN PHASE 2

Phase 2, in which study sections within the IRGs are created, will begin in 2000 and likely continue throughout the next two years. This Phase will also depend heavily on input from the broad scientific community. It will be carried out by sets of consultants; each composed of at least one member of the parent Boundaries Panel, experts in the specific scientific and medical fields under review, and NIH staff.

To ensure that the resulting structure evolves to accommodate the ongoing emergence of new scientific opportunities and practices, our Panel recommends that a formalized process be carried out at regular intervals to evaluate the overall structure and function of the CSR peer review process. Thus, the study section design produced in Phase 2 should be reviewed on approximately a five-year basis by the ad hoc external advisory groups that are being established for each IRG. In addition, a global assessment, based on input from applicants, reviewers, and NIH staff, should be made periodically by the CSR Advisory Committee.

Our Panel believes that study sections should be created according to the following principles:

Not too narrow: The range of science that is considered by each study section should be sufficiently broad as to allow a comparative evaluation of the merit of grant proposals in several related fields. Study sections should not be restricted either to a single field or to a single experimental approach.

Not too broad: On the other hand, the range of science considered by each study section should be sufficiently coherent to allow its members to reach independent evaluations of the merit of each proposal, following the panel discussion of each one.

Overlap is desirable: To allow flexibility in review, the range of scientific expertise of study sections within an IRG (and often between IRGs) should overlap, such that more than one study section could appropriately review any individual grant application.

Connected to specific diseases or organs: Whenever appropriate, basic research should be reviewed in the context of the biological question to which it relates; i.e., in an IRG focused on a particular organ, biological system and/or disease. However, within the IRG, individual study sections may either be designed to cover a single type of scientific approach (e.g., molecular studies versus patient-oriented programs), or to integrate such studies as is deemed appropriate for the field.

Connected to basic science: Because applied research must be rooted in a firm understanding of underlying biological and behavioral mechanisms, it is important for grant applications on applied subjects to receive a careful review of the basic science on which they are founded. Furthermore, while clinical research applications should be focused in clinical research panels to the extent possible, their review should be informed by the perspective of basic scientists to ensure that the highest quality of contemporary science is applied to clinical problems.

Density of expertise: When it is necessary or desirable to review very different types of research, for example clinical and basic, or hypothesis- and problem-driven, in the same study section, at least 30 percent of the applications reviewed should be of each type. In addition, reviewers with expertise in each broad category of research should comprise no less than 30 percent of the total membership of the study section.

Balance of breadth and depth of study section members: The study section should be populated to balance the sometimes conflicting goals of furnishing sufficient expertise for rigorous review and providing sufficient breadth and understanding of biomedicine (rather than methodology) to evaluate significance.

Flexibility of reviewers: To help achieve sufficient breadth and depth of expertise on each study section and create flexibility while maintaining consistency of review procedures and practices within (and even across) IRGs, study section members should function as "mobile" experts, moving from one study section to another as required.

Serving multiple Institutes: Where possible, study sections should serve more than one NIH Institute. This practice allows the Institutes to gauge the quality of their applications relative to those assigned to other Institutes for potential funding.

V. CONCLUSION

In this draft Phase 1 report, the Panel has attempted to provide our best advice on a revised organization and on cultural norms to govern the review of NIH grant applications by the Center for Scientific Review. Many details and adjustments remain to be made in Phase 2, and we recognize that judgment will be required to balance an obvious tension between worthy goals. This has been a very challenging task, and it is with some trepidation that we now call on the wider community to examine our efforts. We are confident that your comments will improve our product by providing us with a broader range of inputs than those which we have thus far managed to accommodate through our own limited consultations with outside experts.

While recognizing that perfection is unattainable, our goal is to optimize the CSR review system to provide a review process that encourages risk-taking and innovation and is flexible and responsive enough to keep up with the many new opportunities developed by the striking advances in biomedical science. We hope that the final result will be a dynamic system that appreciates new ideas and all research styles – one that facilitates acceleration of the pace of progress in biomedical research through an improved, merit-based competitive review of all applications.³

___________________________
¹ Approximately 40,000 applications are subjected to the peer review process each year. Approximately one-fourth of these applications are evaluated in scientific review groups (SRGs) managed by the Institute to which the application is assigned for potential funding. In general, these are applications for which optimal review requires greater programmatic context (for example, Center (P50) and institutional training (T32) grant applications). Three-fourths (primarily individual investigator-initiated (R01s), fellowship (F32s) and Small Business Innovation Research Program (R43, R44) applications are evaluated in the Center for Scientific Review (CSR), one of NIH’s 25 Institutes and Centers (ICs), whose mission is to provide this service for all NIH extramural programs.

2 The generic, functional term for any group engaged in scientific and technical peer review is Scientific Review Group (SRG). SRGs are commonly called study sections in CSR. They may either be individually chartered or be part of an Integrated Review Group (IRG), which is a cluster of Scientific Review Groups chartered as a single entity with a related integrated scientific focus. A Special Emphasis Panel (SEP) is a scientific review group whose membership is fluid, with members designated to serve for individual meetings rather than for fixed terms of service. SEPs are constituted by each IRG to evaluate applications that do not fit well in one of the component study sections. Within this report, the term study section, while not always technically correct, is invoked liberally to match its use by members of the extramural scientific community. In addition, IRGs, while officially specified as Initial Review Groups, are herein referred to as Integrated Review Groups to better describe their function.

3 We would be remiss if we failed to point out that our recommendations to operate and continually refine a more complex and flexible system will place a considerably greater burden on the CSR staff. Therefore, as needed, the CSR must be provided with the additional resources that will allow its staff to effectively manage and improve a system of peer review that lies at the heart of the enormous success of the U.S. scientific enterprise.

APPENDIX I

PROPOSED IRG STRUCTURE

Twenty-one IRGs are listed below, beginning with five concerned with the most fundamental biological processes or technologies that underlie approaches to the properties and treatment of all organ systems, disease processes, and general health issues. Examples of topics to be covered within each IRG are bulleted; the topics are not meant to be exhaustive. These topic lists do not represent study sections. As noted previously, study sections will be defined in Phase 2. Accordingly, many cross-cutting issues, such as patient-oriented research, complementary and alternative medicine, autonomic nervous system function related to all organs, rehabilitation, life-course changes, effects of environment, toxicology, molecular and cell biology and development of organs, have not been listed within the many IRGs.

In the proposed organization, the six IRGs created to integrate the review of neuroscience research and behavioral and social sciences research from the former Alcohol, Drug Abuse, and Mental Health Administration Institutes [Molecular, Cellular, and Developmental Neuroscience; Integrative, Functional, and Cognitive Neuroscience; Brain Disorders and Clinical Neuroscience; Social Sciences, Nursing, Epidemiology and Methods (titled Health of the Population in the new structure); Risk, Prevention, and Health Behavior; and Behavioral and Biobehavioral Processes] have been retained essentially intact in proposed IRGs. (see Appendix I and http://www.csr.nih.gov/review/reorgact.htm.) Three current IRGs (Biochemical Sciences, AIDS and Related Research, and Nutritional and Metabolic Sciences) are not continued, and their research applications widely dispersed. Another 7 of the 20 current IRGs (Biophysical and Chemical Sciences; Genetic Sciences; Immunological Sciences, Infectious Diseases and Microbiology; Oncological Sciences; Endocrinology and Reproductive Sciences; and Musculoskeletal and Dental Sciences), while altered in their scope, clearly relate to proposed IRGs. In general, the current Cell Development and Function IRG would be divided into the Molecular Approaches to Gene Function and the Molecular Approaches to Cell Functions and Interactions IRGs; the current Surgery, Radiology, and Bioengineering IRG into the Fundamental Bioengineering and Technology Development and the Surgery, Applied Imaging, and Applied Bioengineering IRGs; the current Pathophysiological Sciences IRG into the Digestive Sciences and the Pulmonary Sciences IRGs; and the current Cardiovascular Sciences IRG into the Cardiovascular Sciences and the Hematology IRGs. (The current IRGs are described at http://www.csr.nih.gov/review/irgdesc.htm).

IRGs that Encompass Fundamental Science that Underlies and Transcends all Physiological and Disease Systems

CHEMICAL BIOLOGY AND BIOPHYSICS IRG - This IRG will consider research applications focused on the detailed structures, chemistry, and physics of macromolecules and interacting small molecules.

Chemical dynamics and mechanisms

Chemical synthesis

Enzymology and catalysis

Protein chemistry, structure and folding

Nucleic acid chemistry, structure and folding

Oligosaccharide and polysaccharide biochemistry

Lipid chemistry

MOLECULAR APPROACHES TO GENE FUNCTION IRG – This IRG will consider research applications focused on the molecular mechanisms and regulation of the global processes of gene expression that are fundamental to all living cells.

Genome replication

Regulatory mechanisms of gene expression

Chromosome structure and dynamics

DNA recombination and repair

DNA transcription and translation

RNA processing, stability and degradation

Protein processing, stability and degradation

MOLECULAR APPROACHES TO CELL FUNCTIONS AND INTERACTIONS IRG - This IRG will consider research applications at the level of the functions, interactions, and regulation of cells and cellular organelles, focusing on fundamental cell biological processes.

Membrane transport, structure and dynamics

Membrane and protein trafficking

Organelles and compartments

Cytoskeleton, cell division, and cell movements

Bioenergetics

Intra- and intercellular signaling

Cell cycle control

Apoptosis

Cell junctions and extracellular matrix

Cell and tissue development

Cell and tissue maintenance, repair, and aging

Embryonic development and cell differentiation

Organization of cells into tissues

FUNDAMENTAL GENETICS AND POPULATION BIOLOGY IRG - This IRG will consider research applications focused on general issues in genetics, genomics and population dynamics.

Prokaryotic and general eukaryotic genetics and genomics

Mammalian and human genetics and genomics

Quantitative genetics and genetics of complex traits

Population structure and dynamics

Evolutionary genetics

Genetic/molecular epidemiology

Genome structure and function

FUNDAMENTAL BIOENGINEERING AND TECHNOLOGY DEVELOPMENT IRG - This IRG will consider research applications to develop fundamental, broadly applicable new methodologies and instrumentation.

Engineering underpinnings of biomedicine

Instrumentation development

Computational methods and informatics

Biostatistical methods

Mathematical modeling

Imaging techniques

Genome technology

General methods for gene therapy

IRGs that Encompass Science That Relates Specifically to Health or to a Specific Disease or Organ System

HEALTH OF THE POPULATION IRG - This IRG will consider research applications focused on broad social, environmental, cultural, and other contextual influences on health behavior.

Social influences on healthy behavior

Biomedical ethics

Population-based intervention trials

Population processes, composition, and distribution

Social sciences, epidemiology, and methods

Applied biostatistics

Genetic/molecular epidemiology

Outcomes research

Nursing sciences

Comment: This IRG reflects the rapid growth of a variety of disciplines related to the general theme of health promotion and disease prevention, including behavioral, social, and population-based approaches. Healthy behavior refers to studies related to behaviors important for the health of the population, including nutrition, physical activity, adherence to medical treatment regimes, smoking, substance use and violence. Biomedical ethics includes studies related to the ethics of the Human Genome Project and end-of-life issues. Biostatistical issues include those related to population studies.

RISK, PREVENTION, AND HEALTH BEHAVIOR IRG - This IRG will consider research applications focused on risk, prevention and health in the individual, studies of risk and protective processes, including genetic risk factors, and studies of interventions aimed at reducing risks.

Development, prevention, treatment of physical and mental diseases and disorders

Cognitive and affective processes and markers of disease and illness

Risk and protective processes and models

Behavioral and biobehavioral approaches to prevention and treatment

Intervention and risk factor modification studies

Personality, motivation, social roles and their relation to mental and physical health and substance abuse

Prevention and outcomes research

Behavioral medicine

Healthy behavior of the individual, including disease survivors

BEHAVIORAL AND BIOBEHAVIORAL PROCESSES IRG - This IRG will consider research applications focused on cognitive, perceptual, intra- and interpersonal processes; social influences on healthy behavior, studies of normal and disordered movement; and studies of developmental, psychopathological and substance use disorders.

Healthy adaptation across the lifespan

Basic biobehavioral, psychological, social and cultural processes governing the external expression of emotion (affect)

Language and other types of communication

Cognition and perception

Emotional, behavioral, and cognitive disorders

Developmental, psychopathological and substance use disorders

Normal and disordered motor function

IMMUNOLOGY IRG - This IRG will consider research applications focused on basic immunology and diseases that are principally immunological in their origin or manifestations.

Cellular, molecular and developmental immunology

Immunochemistry and immunogenetics

Autoimmune and hypersensitivity diseases

Immunodeficiency diseases, including studies of the immune defects due to HIV

Rheumatology

Tumor immunology and immunotherapy

Chronic fatigue and related disorders

Inflammation

Mucosal immunology

Innate immunology

Fundamental transplantation immunology, tolerance, and rejection

Comment: Where immunological responses to pathogens or specific disease pathogens or diseases are secondary, proposals should be assigned to IRGs more appropriate for the specific condition. Many studies of HIV will be most appropriately reviewed within the Infectious Diseases and Microbiology IRG, but those with a primary focus on HIV immunodeficiency would be reviewed here.

INFECTIOUS DISEASES AND MICROBIOLOGY IRG - This IRG will consider research applications focused on studies of infectious diseases, pathogenesis, prevention, and treatment.

Bacteriology and bacterial pathogenesis

Virology and viral pathogenesis

Mycology and fungal pathogenesis

Parasitology and parasitic diseases

Vaccines

HIV virology and pathogenesis

HIV opportunistic infections

HIV drug development

Host-pathogen interactions

Comment: Aspects of HIV that are best considered elsewhere include HIV epidemiology, neurological manifestations of HIV, studies of HIV immunodeficiency, and disease prevention (as related to risks and behaviors). Studies of oral, surgical or other "subspecialty" infections should be reviewed in this IRG. Infections as triggers for chronic diseases, including cancers, overlap with other IRGs and might be reviewed elsewhere; examples include hepatitis viruses causing liver failure and cancer, as well as papilloma viruses causing cancer.

ONCOLOGICAL SCIENCES IRG - This IRG will consider research applications focused on cancer, spanning pathogenesis to clinical trials of therapeutics.

Cancer genetics and cell biology, including metastasis

Pathology

Environmental carcinogenesis

Drug development and evaluation

Therapeutic oncology

Leukemias and lymphomas

Comment: Pathology is assumed to include immunopathology, diagnostic methods, and cytogenetics. Therapeutic oncology includes chemotherapy, hormonal therapy, combined modality therapies, radiotherapy, biological response modifiers, and chemoprevention. Tumor immunology and immunotherapy are represented within the Immunology IRG. It is intended that applied clinical immunotherapeutics, including tumor vaccines, would be reviewed here.

HEMATOLOGY IRG - This IRG will consider research applications focused on studies of blood cells and their diseases, and studies of the coagulation system and its pathology.

Stem cell biology and hematopoiesis (including cytokines)

Red blood cell biology and structure, including sickle cell anemia

Leukocyte biology

Bone marrow transplantation and transfusion

Coagulation biochemistry and disorders

Platelet biology and disorders

Vascular biology, thrombosis, and platelet function

Comment: Thrombosis proposals primarily addressing platelet biology and pathology would be included here, but those with emphasis on other aspects of thrombosis would be referred to the Cardiovascular Sciences IRG.

CARDIOVASCULAR SCIENCES IRG - This IRG will consider research applications focused on basic studies of heart and blood vessel development and physiology, studies of pathophysiology of the heart and vasculature, and clinical studies pertaining to specific cardiovascular diseases and their treatment.

Electrophysiology/arrhythmia

Cardiovascular pharmacology

Atherosclerosis/nutrition including lipid metabolism

Vascular biology/thrombosis

Cytokines/nitric oxide

Angiogenesis

Cardiac physiology

Bioengineering, including devices

Heart failure and transplantation

Hypertension

Renal diseases, including chronic renal failure and transplantation

Renal physiology

Cerebrovasculature

Comment: Proposals focusing on lipid metabolism could be reviewed either here or within the Endocrinology, Metabolism, and Reproductive Sciences IRG. Proposals in the area of thrombosis are included here, but those with a primary focus on platelet function and biology would be more suited for the Hematology IRG. Similarly, proposals on bioengineering related specifically to devices for cardiovascular disease (stents, pacemakers, etc.) would be assigned here, but those involving more general aspects of bioengineering would be reviewed elsewhere. Likewise, transplantation as applied to the heart would be assigned here, but studies of fundamental transplant biology, even as relevant to the heart, could be reviewed elsewhere.

ENDOCRINOLOGY, METABOLISM, AND REPRODUCTIVE SCIENCES IRG - This IRG will consider research applications focused on endocrine and reproductive systems and associated diseases, as well as basic intermediary metabolism and metabolic disorders.

Fetal, maternal, and neonatal physiology

Hormone physiology and biochemistry

Endocrine function and disease

Diabetes and diabetic complications

Neuroendocrinology

Genitourinary, including prostatic function and disease

Reproduction, fertility and contraception

Metabolism and metabolic diseases

Nutrient metabolism, obesity and clinical nutrition

Reproductive toxicology

Menopause

Comment: Diabetes affects many target organs. It is envisioned that studies of diabetes, pathophysiology, and control will be reviewed within this IRG. Those involving effects on other organs will be reviewed in other organ-specific IRGs if their focus is primarily on the function of the respective organs rather than on more general aspects of diabetes pathogenesis. Neuroendocrinology within this IRG will include those areas most closely allied with endocrinology, such as studies of the endocrine functions of the hypothalamic-pituitary axis. Other neuroendocrine studies relating to central nervous system (or brain) function of disease will be reviewed within appropriate neuroscience IRGs. Studies of metabolism included here span biochemistry of intermediary metabolism, cellular metabolism and physiology, and specific disorders involving these pathways.

BONE, MUSCLE, CONNECTIVE TISSUE, AND SKIN IRG - This IRG will consider research applications focused on studies of connective tissues, which include bone and dental tissues, skeletal muscle, skin, and the extracellular matrix, and their associated diseases.

Bone biology and disease

Extracellular matrix components and biology, including cartilage/ligament/tendon biology and disease

Craniofacial development and disorders

Dental sciences

Skeletal muscle biology and disease

Skin biology and disease

Sports medicine

Chiropractic medicine

Rheumatological disorders

Biomineralization

Comment: Areas of potential overlap with other IRGs include infections, tumors, and surgical procedures related to dentistry, immunological features of dermatological disease, and aspects of orthopedics. The primary focus of the application should determine the choice between overlapping IRGs. This IRG also encompasses musculosketal function, including aging, biomechanics and biomaterials, and tissue engineering as they pertain specifically to the musculoskeletal system.

DIGESTIVE SCIENCES IRG - This IRG will consider research applications focused on studies of the entire gastrointestinal tract and related organs.

Gastroenterological pharmacology and physiology

Esophageal, gastric, intestinal, and pancreatic diseases, including disorders of the gall bladder and biliary system

Liver diseases, including liver transplantation

Clinical nutrition/malnutrition (not including prevention)

Drug metabolism, pharmacology, and toxicology

Digestion, absorption, and nutrient transport

Comment: Liver diseases include the effects of alcohol and viral infections on the liver, and their treatment by transplantation and other modalities. Intestinal diseases include the entire spectrum from motility disorders to chronic inflammatory diseases, including the relevant immunology and pathogenesis studies. Nutrition proposals aimed at disease prevention would be referred to the Risk, Prevention and Health Behavior IRG. Those directed to this IRG relate to more traditional aspects of clinical nutrition (e.g. malabsorption, metabolic effects of malnutrition).

PULMONARY SCIENCES IRG - This IRG will consider research applications focused on basic studies of the lung and respiratory system development and physiology, studies of pathophysiology of the lung and respiratory system, and clinical studies pertaining to the specific pulmonary diseases and their treatment.

Lung development

Pulmonary physiology

Chronic airway diseases, including asthma, cystic fibrosis, obstructive airways diseases, and interstitial lung diseases, including pulmonary fibrosis

Environmental and occupational respiratory diseases (i.e. diseases of the lung)

Pulmonary toxicology

Lung transplantation

Adult Respiratory Distress Syndrome

Pulmonary vascular diseases

Sudden Infant Death Syndrome

Comment: Applications dealing with asthma might be reviewed within the Immunology IRG if the focus is on fundamental immunology. Proposals on cystic fibrosis might be reviewed with the Digestive Sciences IRG if the focus is on gastrointestinal manifestations or treatment of cystic fibrosis.

MOLECULAR, CELLULAR, AND DEVELOPMENTAL NEUROSCIENCE IRG - This IRG will consider research applications focused on the basic mechanisms that determine the structure and function of neurons, glia, and other excitable cells, and aspects of development in both the central and peripheral nervous system.

Cytoskeleton

Extracellular matrix

Membrane trafficking

Signal transduction

Structural analysis

Basic neuropharmacology

Medicinal chemistry

Cell cycle control

Apoptosis

Neurodegeneration

Basic cellular and molecular physiology

INTEGRATIVE, FUNCTIONAL, AND COGNITIVE NEUROSCIENCE IRG - This IRG will consider research applications focused on how the nervous system is organized and functions at an integrative, systems level.

Neural basis of emotional and motivational behavior

System function under varying behavioral states (e.g., sleep)

Biological rhythms

Autonomic nervous system homeostasis

Chemosensation, hearing, balance, touch, somatosensation

Motor systems and sensorimotor integration

Integration of multisensory information

Memory and other cognitive processes

Aging and cognition

Computational and theoretical models of cognitive processes

Neural coding of complex stimuli (e.g., spatial transformations, speech perception)

Attention and its effects on information processing in the nervous system

Vision

BRAIN DISORDERS AND CLINICAL NEUROSCIENCE IRG - This IRG will consider research applications focused on disease and injury to the nervous system, including issues of neural substrate, functional consequences, and rehabilitation.

Traumatic brain or spinal cord injury/regeneration/transplantation

Consequences of ischemia or hypoxia, convulsive disorders

Neurochemistry, neuropharmacology

Mental disorders and drug abuse, neurodegenerative diseases and other neuropathies

Clinical sleep research

Neuroinfections

Neurogenetics

Developmental disorders

Molecular neuropathology

Cerebrovasculature

SURGERY, APPLIED IMAGING, AND APPLIED BIOENGINEERING IRG - This IRG will consider research applications focused on surgical and related disciplines, and on applied imaging and applied bioengineering.

Anesthesia

Biomaterials

Bioengineering and radiologic imaging (application of e.g., MRI, PET scan)

Burns and trauma

Diagnostic bioimaging

Surgery

Comment: Surgery includes most of surgical research except some transplantation research; the latter would be reviewed in the context of the chronic diseases for which transplantation is a major therapeutic modality. Surgical infections will, in the main, be reviewed in the Infectious Diseases and Microbiology IRG. Applied imaging and applied bioengineering research applications reviewed in this IRG are those that are pertinent to multiple organ systems.

APPENDIX II
SOME RECENT CHANGES IN NIH REVIEW PROCESSES

Establishment of the Integrated Review Group (IRG) as the Functional Unit of Review

For decades, the individual study section was the functional unit of review in CSR (formerly DRG, the Division of Research Grants). However, CSR has recently adopted the IRG (a cluster of scientifically related study sections) as the functional unit of review. This organizational change groups study sections in a way that provides better opportunities for teamwork, a flexible distribution of applications, and a sharing of reviewer expertise. For example, the initial referral of applications is now made to the IRG, with subsequent assignment to individual study sections by IRG staff. In addition, IRGs can arrange concurrent meetings of several or all of their study sections, so that reviewers with specific expertise can participate in multiple study sections, as needed.

Establishment of IRG External Advisory Groups

For each IRG, an external advisory group is to be appointed, composed of 5 to 10 active, widely respected extramural researchers in disciplines related to those reviewed by the IRG. These advisory groups will examine all aspects of IRG function, and will provide advice regarding the boundaries between study sections, reviewer composition for each study section, and best reviewing practices. In addition, this advisory group will serve as a resource for the SRAs, helping them to identify and recruit reviewers, and deal with a variety of IRG-specific issues.

ADAMHA Integration

To complete the merger of the Institutes of the Alcohol, Drug Abuse, and Mental Health Administration into the NIH that began in 1992, review activities of the National Institutes on Alcoholism and Alcohol Abuse, Drug Abuse, and Mental Health have now been integrated into the CSR. Members of the scientific community, CSR personnel, and Institute staff developed 21 new neuroscience study sections that are grouped in 3 IRGs, 16 new behavioral and social sciences study sections that are grouped in 3 IRGs, and 8 new study sections within the AIDS IRG. (See http://www.csr.nih.gov/review/reorgact.htm. The reorganization accomplished by this team has been incorporated into the larger organizational framework proposed by our panel as described in Section II, and our panel recommends that the process they followed for forming study sections be emulated in Phase 2 of our work.

Increasing the Quality of Reviewers and Broadening Their Participation

The CSR is exploring flexible ways to overcome the present obstacles to reviewer recruitment. The Center is testing the effectiveness of having several members share a single appointment; and will evaluate the use of senior scientists to augment technical expertise with broad perspective. It will also assess ways to overcome the reluctance of researchers to serve on study sections because they perceive that their funding may be jeopardized by the requirement that their grant be reviewed in another study section or by a special emphasis panel. CSR is also working to expand the reviewer pool and improve the nomination process. The SRAs have been asked to broaden their nets when identifying new study section members, and they now provide more extensive information to the CSR Director about the source and rationale for specific reviewer nominations.

Communities That Feel Disadvantaged

In response to extensive outreach activities, three groups have been highlighted for attention: (1) clinical researchers, (2) behavioral and social sciences researchers, and (3) bioengineers and developers of technology and instrumentation.

To address long-standing concerns regarding the review of clinical research, CSR engaged an expert to act as liaison to the clinical research community (http://www.csr.nih.gov/events/research.htm). Previous analyses indicate that clinical researchers are disadvantaged when their applications are reviewed in study sections that had been assigned only a small number or proportion of clinical proposals. (A small proportion has been arbitrarily defined as less than 30% of the grant portfolio of the study section.) To solve this problem for about 50% of the clinical applications, the CSR has clustered clinical oncological sciences applications in one special emphasis panel and clinical cardiovascular proposals in another.

A second expert, who has recently completed his work, was retained to serve as liaison to the behavioral and social sciences community. He has reported that behavioral and social scientists tend to overemphasize methodological deficiencies in their reviews of grants, with the result that study sections rather than investigators are often designing research programs in these areas.

A special Working Group on Review of Bioengineering and Technology and Instrumentation Development Research was created to undertake a third activity: to identify the obstacles to fair, high-quality, rigorous review and develop a set of principles to guide CSR in establishing a review infrastructure that will fairly evaluate interdisciplinary research. This committee’s report http://www.csr.nih.gov/bioopp1/select.htm.) notes the need to develop a flexible organizational structure, to revise operating principles and practices, to promote system agility to adapt to the continually changing scientific landscape, and to declare the importance of multiple types of research.

Review of Fellowship Applications

To determine optimal ways to identify the most promising candidates for training support, an outside expert was engaged to study the variable venues and practices currently used to review fellowship applications within CSR. The resultant recommendation is to establish dedicated fellowship study sections to review the majority of these applications.

Training for SRAs

To ensure that CSR staff consistently apply uniform principles in managing the review process, a group of senior SRAs initiated a training program for their colleagues. New SRAs are presented with an extensive overview of policies and procedures, and all SRAs learn of changes to the system and are trained in specific aspects of peer review through a program of continuing education.

Development of Guidelines for Chairs and SRAs

The CSR Advisory Committee has developed a set of guidelines that outlines "best practices" for Study Section Chairs. The document outlines the need for Chairs to (i) become familiar with all the applications to be considered; (ii) consult with the SRA in assigning applications to reviewers; (iii) focus discussions on key issues; (iv) maintain fairness of the review process; and (v) promote consistent criteria-based scoring. Chairs are also encouraged to assist SRAs in identifying and assessing candidate reviewers and, where appropriate, to make initial contacts with candidates; to help train members; to help develop a successor; and to invite performance assessments of all participants, including that of the Chair, in the process.

New Review Criteria

Reviewers are now instructed to address each of five review criteria (significance, approach, innovation, investigator, and environment) and assign a single global priority score rating for each scored application (http://www.nih.gov/grants/guide/1997/97.06.27/notice-review-criter9.html). The score should reflect the reviewer’s judgment of the likelihood that the proposed research will have a substantial impact on advancing our understanding of biological systems, improving the control of disease, and/or enhancing health. This new NIH requirement to evaluate five criteria is intended to overcome an inappropriate focus on technical feasibility and provides an opportunity to rate all types of research equitably in terms of their potential impact. Innovation has been specifically included among the criteria to combat the present excess conservatism in the review system. The focus on the investigator is meant to take into account the skill of the scientist, based on his or her past accomplishments.

APPENDIX III
ROSTER
PANEL ON SCIENTIFIC BOUNDARIES FOR REVIEW

CHAIRMAN
ALBERTS, Bruce M., Ph.D.
President
National Academy of Sciences
Washington, DC 20418

MEMBERS
AYALA, Francisco J., Ph.D.
Donald Bren Professor
Department of Ecology and
Evolutionary Biology
University of California, Irvine
Irvine, CA 92697

BOTSTEIN, David, Ph.D.
Professor and Chair
Department of Genetics
School of Medicine
Stanford University
Stanford, CA 94305-5120

FRANK, Ellen, Ph.D.
Professor of Psychiatry and Psychology
Department of Psychiatry
School of Medicine
University of Pittsburgh
Western Psychiatric Institute and Clinic
Pittsburgh, PA 15213

HOLMES, Edward W., M.D.
Dean, School of Medicine
Vice Chancellor for Academic Affairs
Duke University Medical Center
Durham, North Carolina 27710

LEE, Ronald D., Ph.D.
Professor of Demography and Economics
Department of Demography
University of California, Berkeley
Berkeley, CA 94720

MACAGNO, Eduardo R., Ph.D.
Dean, Graduate School of Arts and
Sciences
Associate Vice President for Research
and Graduate Education
Professor of Biological Sciences
Columbia University
New York, NY 10027

MARRACK, Philippa, Ph.D.
Professor, Department of Medicine
National Jewish Medical and
Research Center
Professor, Departments of Biochemistry,
Biophysics, Genetics, Microbiology
Immunology, and Medicine
University of Colorado Health Sciences
Center
Denver, CO 80206

OPARIL, Suzanne, M.D.
Professor of Medicine and of
Physiology and Biophysics
School of Medicine
University of Alabama
Birmingham, AL 35294

ORKIN, Stuart H., M.D.
Leland Fikes Professor of
Pediatric Medicine
The Children’s Hospital
Harvard Medical School
Boston, MA 02115

RUBENSTEIN, Arthur H., M.B.B.Ch.
Dean and Executive Vice President Mount Sinai Medical Center
New York, NY 10029

SLAYMAN, Carolyn W., Ph.D.
Deputy Dean for Academic
and Scientific Affairs
Sterling Professor of Genetics
Professor of Physiology
School of Medicine
Yale University
New Haven, CT 06520

SPARLING, P. Frederick, M.D.
Chair, Department of Medicine
J. Herbert Bate Professor of Medicine
and Microbiology and Immunology
School of Medicine
University of North Carolina
Chapel Hill, NC 27599

SQUIRE, Larry R., Ph.D.
Research Career Scientist
Department of Veterans Affairs
Professor of Psychiatry,
Neurosciences, and Psychology
University of California, San Diego
San Diego, CA 92161

VON HIPPEL, Peter H., Ph.D.
Professor of Chemistry and
Member, Institute of Molecular Biology
American Cancer Society Research
Professor of Chemistry
University of Oregon
Eugene, OR 97403

CENTER FOR SCIENTIFIC REVIEW ADVISORY COMMITTEE MEMBER
YAMAMOTO, Keith R., Ph.D.
Professor and Chairperson
Department of Cellular and
Molecular Pharmacology
University of California
San Francisco, CA 94143-0450

EXECUTIVE SECRETARY
ENGEL, Linda W., M.S.
Associate Director for Planning
and Outreach
Center for Scientific Review
National Institutes of Health
Bethesda, MD 20892