![[DRG Home
Page]](../images/drghome.gif)
| Referral & Review |
REVIEW PROCEDURES FOR SCIENTIFIC REVIEW GROUP MEETINGSREVIEW PROCEDURES The guiding principles for the initial review of research project grant applications are based on the Public Health Service (PHS) Scientific Peer Review Regulations that state that peer review groups are to make recommendations concerning the scientific merit of applications. The specific criteria used to assess the merit of research project grant applications will vary with types of applications reviewed, such as Investigator Initiated Research Project Grants (R01), Academic Research Enhancement Awards (R15), the National Research Service Awards (F32, F33, etc.), Small Business Innovation Research Grants, etc. For the review of investigator-initiated research grant applications (e.g., R01 and R15), a streamlined procedure will be employed to determine whether the applications assigned to a study section are in the upper or lower half. This procedure is described in the document CSR Streamlined Review Procedures. Prior to the meeting of the study section, reviewers will be asked to identify applications that they feel are not in the upper half and will consequently not be discussed at the study section meeting. If two reviewers/discussants agree that an application is not in the upper half, it will be designated as such, and a list prepared by the SRA identifying proposed applications not in the upper half will then be sent to reviewers a few days prior to the study section meeting. After seeing this list any review group member not in conflict may disagree and identify an application that he/she believes is in the upper half and, therefore, should receive full discussion. At the beginning of the meeting, the list will be read aloud for final concurrence by the entire study section. If any member of the review group not in conflict questions the rating or wishes to comment on the application, it will be discussed and considered by the entire review group in the normal sequence of review. The Chairperson of the scientific review group introduces each application designated for discussion and calls upon the individuals assigned by the SRA to present their evaluations. The assigned discussants are then called upon for their comments and group discussion follows. If prior to substantial discussion the scientific review group determines that the application being discussed should actually not be placed in the upper half, it may recommend that the application not be scored. Such a designation requires unanimous agreement of the scientific review group. Otherwise, after sufficient discussion has ensued, the Chairperson calls for a priority rating to be assigned to the application. Ratings will be assigned by regularly appointed members of the scientific review group and by those serving as temporary members. Reviewers are encouraged not to abstain. However, a reviewer who feels unable to assess the merit of an application, as evidenced by his/her prior discussion or recommendation for deferral, should mark the vote sheet "AB". In addition, if there are comments or serious concerns regarding the use of human subjects or animal welfare or biohazards, a motion may be initiated that the application should be coded to reflect these comments or concerns, and an appropriate note will be included in the summary statement. If additional information is needed before a review group can make a recommendation, a motion for deferral may be entertained. The review group may, by majority vote, defer an application for additional information or, if information necessary to evaluate the application can be obtained only by visual inspection of the facilities, for a project site visit. Any member may nominate an application for deferral. NUMERICAL RATING Each scored application is assigned a single, global score that reflects the overall impact that the project could have on the field based on consideration of the five review criteria (significance, approach, innovation, investigator, and environment), with the emphasis on each criterion varying from one application to another, depending on the nature of the application and its relative strengths. The best possible priority score is 100 and the worst is 500. Individual reviewers mark scores to two significant figures, e.g., 2.2, and the individual scores are averaged and then multiplied by 100 to yield a single overall score for each scored application, e.g., 253. Abstaining members and those not present during the discussion do not assign a numerical rating and are not counted in calculating the average of the individual ratings. Reviewers are asked to recommend that half the applications not be scored and to spread final scores to achieve a median score of 300. (Any member of the scientific review group may request that an application be scored, in which case all members must score the application.) To the extent that the study section does not score some applications, the scoring range is altered. If half of the applications are not scored, then the remaining applications should be scored from 100-300. If only 25% of the applications are not scored then the remaining applications should be scored from 100-400. BUDGET The budget recommendation should be based upon the appropriateness of direct costs for the proposed research for each year of support requested. Attention should be given to the need for all personnel listed in the application and their percent effort in relation to the scope of works. Reviewers should keep in mind the applicant’s ability to move funds amongst budget categories, therefore, the appropriateness of the total budget and the requested duration of support in relation to the research proposed should be emphasized. Reviewers may identify areas of potential overlap with other supported research. However, potential overlap may be neither a reason for altering the budget nor may it affect the priority score. Information regarding potential overlap is included in the Scientific Review Administrator's note at the end of the summary statement. FOREIGN ORGANIZATIONS In addition to the regular review criteria, foreign applications are evaluated in terms of special opportunities for furthering research programs through the use of special talents, resources (human subjects, animals, diseases, equipment or technologies), populations or environmental conditions in the applicant country which are not readily available in the United States or which provide augmentation of existing United States resources. In addition, it should be noted whether similar research is being done in the United States and whether there is a need for additional research in the area of the proposal. These special review criteria are not applied to applications from domestic institutions that include a significant foreign component. RESEARCH INVOLVING HUMAN SUBJECTS Applicant organizations have the primary responsibility for safeguarding the rights and welfare of individuals who participate as subjects in research activities supported by the NIH. However, the NIH also relies on its scientific review groups and National Advisory Councils or Boards to evaluate all applications and proposals involving human subjects for compliance with the Department of Health and Human Services human subject regulations. There are several considerations for review of applications involving human subjects. These can be clustered into two broad areas: Protection of subjects from research risks; and the inclusiveness of the study population. Protection issues include questions regarding safety and welfare of the subjects, including data and safety monitoring where applicable. Inclusion issues reflect the appropriate involvement of women, minorities and children Assessment of scientific and technical merit of applications involving human subjects must include an evaluation of the proposed composition of the study population and its appropriateness for the scientific objectives of the study. If representation of women, minorities, or children in the study design is inadequate to answer the scientific question(s) addressed and justification for the selected study population is inadequate, reviewers should consider this to be a scientific weakness or deficiency in the study design and must consider this weakness in assigning a priority score. More detailed instructions for reviewing grant applications involving human subjects, and exemptions, are available at the following URL: http://grants.nih.gov/grants/peer/hs_review_inst.pdf Definitions: When considering applications that involve human subjects it is important for reviewers to keep a number of definitions of terms in mind: Human subjects: Federal regulations define "human subject" as a "living individual about whom an investigator obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information." The regulations extend to the use of human organs, tissue and body fluids from individually identifiable human subjects as well as to graphic, written, or recorded information derived from individually identifiable human subjects. A subset of research involving human subjects may qualify for exemption, but justification must be provided under the heading "Protection of Human Subjects from Research Risk". The use of autopsy materials is governed by applicable state and local law and is not directly regulated by the Federal human subject regulations. Clinical
research is defined as: (1)
Patient-oriented research, i.e., research conducted with human subjects (or
on material of human origin such as tissues, specimens and cognitive
phenomena) for which an investigator (or colleague) directly interacts with
human subjects. (Excluded from the
definition of patient-oriented research are in vitro studies that utilize
human tissues that cannot be linked to a living individual.) Patient-oriented research includes: (a)
mechanisms of human disease, (b) therapeutic interventions, (c) clinical trials,
and (d) development of new technologies; (2) Epidemiologic and behavioral
studies; or (3) Outcomes research and health services research. http://www.nih.gov/news/crp/97report/execsum.htm A Clinical Trial is operationally defined as a prospective biomedical or behavioral study of human subjects that is designed to answer specific questions about biomedical or behavioral interventions. An NIH-defined Phase III clinical trial is a broadly based prospective clinical investigation, usually involving several hundred or more human subjects, for the purpose of evaluating an experimental intervention in comparison with a standard or control intervention or comparing two or more existing treatments. Often the aim of such investigation is to provide evidence leading to a scientific basis for consideration of a change in health policy or standard of care. The definition includes pharmacologic, non-pharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. Community trials and other population-based intervention trials are also included. A valid analysis is required in phase III clinical trials. This means an unbiased assessment. Such an assessment will, on average, yield the correct estimate of the difference in outcomes between two groups of subjects. Valid analysis can and should be conducted for both small and large studies. A valid analysis does not need to have a high statistical power for detecting a stated effect. The principal requirements for ensuring a valid analysis are:
Research Conducted in a Foreign Country: For foreign awards, and domestic awards with a foreign component, the NIH policy on inclusion of women and minority groups in research is the same as that for research conducted in the U.S. If there is scientific rationale for examining subpopulation group differences within the foreign population, investigators should consider designing their studies to accommodate these differences. Children: For purposes of this policy, a child is an individual under the age of 21 years. This definition does not affect the human subject protection regulations for research on children (45 CFR 46) and their provisions for assent, permission, and consent, which remain unchanged. State laws define what constitutes a "child," for the purpose of determining whether or not a person can legally consent to participate in a research study. EXEMPTION FROM HUMAN SUBJECTS REGULATIONS If the applicant designates an exemption from the human subjects regulations, reviewers should evaluate the information provided to determine if the designated exemption is appropriate. With regard to exemption 4, although reviewers need not evaluate questions related to research risks or the inclusion of women and minorities, the appropriate inclusion of children DOES need to be addressed for these applications. PROTECTION OF HUMAN SUBJECTS If the proposed research involves human subjects, and does not qualify as being exempt, it is considered clinical research (see definition above) and reviewers must evaluate the plan to protect human subjects. The applicant’s research plan should include four elements under the heading "Protection of Human Subjects from Research Risk". Reviewers are asked to evaluate each of the four elements:
There is a fifth level of protection involving data and safety monitoring, if a clinical trial is proposed. All applications proposing clinical trials research (see definition above) should include plans for Data and Safety Monitoring that describe the entity to be responsible for the monitoring as well as the policies and procedures for adverse event reporting. An NIH defined Phase III clinical trial (see definition above) also requires establishment of a Data and Safety Monitoring Board to provide this oversight. Reviewers should look for this information within the applicants Protection of Human Subjects section and evaluate it accordingly. Based on the evaluation of whether the applicant has adequately addressed Human Subjects Protection according to these criteria and subsequent discussion, the study section may score the application with no concerns or with comments or concerns that may affect the score to a level commensurate with the seriousness of the concern. A "concern" is a scientific review group finding regarding human subjects that requires resolution by program staff prior to award; a "comment" is a scientific review group observation that will be communicated in the summary statement as a suggestion to the principal investigator. No awards will be made until all expressed concerns about human subjects have been resolved to the satisfaction of the NIH. Inclusion of Women and Minorities as Subjects in Clinical Research It is the policy of NIH that women and members of minority groups and their subpopulations must be included in all NIH-funded clinical research (see definition above), unless a clear and compelling rationale and justification establishes that inclusion is inappropriate with respect to the health of the subjects or the purpose of the research. Cost is not an acceptable reason for exclusion, except when the study would duplicate data from other sources. Women of childbearing potential should not be routinely excluded from participation in clinical research. The inclusion of women and members of minority groups, and their subpopulations, must be addressed in developing a research design appropriate to the scientific objectives of the study. The research plan should describe the composition of the proposed study population in terms of sex/gender and racial/ethnic group, and provide a rationale for selection of subjects. Such a plan should contain a description of the proposed outreach programs for recruiting women and minorities as participants. The objective should be to actively recruit and retain the most diverse study population consistent with the purposes of the research project. When an NIH-defined Phase-III clinical trial (see definitions above) is proposed, the Research Plan must include a description of plans to conduct valid analysis (see definition above) by sex/gender, racial/ethnic groups, and relevant subpopulations, if applicable. Accordingly, reviewers should consider these inclusion criteria in their evaluations and:
Additional information concerning the NIH Policy on Inclusion of Women and Minorities as Subjects in Clinical Research is available at:http://grants.nih.gov/grants/funding/women_min/women_min.htm. Inclusion of Children as Participants in Research It is the policy of NIH that children (i.e., individuals under the age of 21) must be included in all human subjects research supported by the NIH, not solely clinical research as is the case for women and minorities, unless there are scientific or ethical reasons not to include them. This policy applies to all research involving human subjects, including research that is otherwise "exempt."Proposals for research involving human subjects must include a description of plans for including children. If children will be excluded from the research, the application must present an acceptable justification for the exclusion. The section in the application titled "Inclusion of Children" should provide either a description of the plans to include children and a rationale for selecting or excluding a specific age range of child, or an explanation of the reason(s) for excluding children as participants in the research. When children are included, the plan must also include a description of the expertise of the investigative team for dealing with children at the ages included, of the appropriateness of the available facilities to accommodate the children, and the inclusion of a sufficient number of children to contribute to a meaningful analysis relative to the purpose of the study. Reviewers should assess each application as being "acceptable" or "unacceptable" in regard to the age-appropriate inclusion or exclusion of children in the proposed research project. Specific exclusionary circumstances and other pertinent information on the inclusion of children in NIH-supported research may be found at: http://grants.nih.gov/grants/guide/notice-files/not98-024.html RESEARCH INVOLVING VERTEBRATE ANIMALS Although the recipient institution and investigator bear the major responsibility for the proper care and use of animals, NIH staff, scientific review groups, and Councils and Boards share this responsibility. Care and use of vertebrate animals in research must conform to applicable law and Public Health Service policy, especially the Principles for Use of Animals. These principles can be summarized as two broad rules:
The evaluation by scientific review group members is to take into consideration the investigator's response to the following five points:
Research using nonhuman primates or chimpanzees requires special attention by Review and Institute staff, so their use must be identified during review and in the vertebrate animal section of the summary statement. There are two situations using animal tissue that do not invoke application of PHS Policy on Vertebrate Animal Use. These are 1) use of blood obtained by a veterinarian in normal medical practice and given to an investigator after testing; and 2) use of left over tissue, as from a slaughterhouse. In both cases, there has been no "custom" request and thus no live vertebrate animals are involved. Any comments or concerns that scientific review group members may wish to express regarding the appropriateness of the choice of species and numbers involved, the justification for their use, and the care and maintenance of vertebrate animals used in the project will be discussed in a special note (ANIMAL WELFARE) in the summary statement. A "concern" is a scientific review group finding regarding animal care or use that requires resolution by program staff prior to award; a "comment" is a scientific review group observation that will be communicated in the summary statement as a suggestion to the principal investigator. Questions may be directed to the Office for Protection from Research Risks. No award will be made unless the applicant institution has given the NIH Office for Protection from Research Risks an acceptable assurance of compliance with the PHS policy and all concerns or questions raised by the scientific review group have been resolved to the satisfaction of the NIH. If concerns are expressed regarding the proper use and care of animals, a recommendation may be made that no further consideration be given to the application. This can be done by either appropriate language in the summary statement (applications eligible for streamlined review) or by majority vote (applications not eligible for streamlined review). BIOHAZARDS The investigator and the sponsoring institution are responsible for protecting the environment and research personnel from hazardous conditions. As with research involving human subjects, reviewers are expected to apply the collective standards of the professions represented within the scientific review group in identifying potential hazards, such as inappropriate handling of oncogenic viruses, chemical carcinogens, infectious agents, radioactive or explosive materials, or recombinant DNA. If applications pose special hazards, these hazards will be identified and any concerns about the adequacy of safety procedures highlighted as a special note (BIOHAZARD) on the summary statement. No award will be made until all concerns about hazardous procedures or conditions have been resolved to the satisfaction of the NIH. AVOIDING CONFLICTS OF INTERESTS DURING SCIENTIFIC REVIEW GROUP MEETINGS At the beginning of each meeting, the Scientific Review Administrator orients the members by explaining the NIH conflict-of-interest policy. A member must leave the room when an application submitted by his/her own organization is being discussed or when the member, his/her immediate family, or close professional associate(s) has a financial or vested interest even if no significant involvement is apparent in the proposal being considered. If the member is available at the principal investigator's institution for discussions; is a provider of services, cell lines, reagents, or other materials, or writer of a letter of reference, the member must be absent from the room during the review. Members are also urged to avoid any actions that might give the appearance that a conflict of interest exists, even though he or she believes there may not be an actual conflict of interest. Thus, for example, a member should not participate in the deliberations and actions on any application from a recent student, a recent teacher, or a close personal friend. Judgment must be applied on the basis of recency, frequency, and strength of the working relationship between the member and the principal investigator as reflected, for example, in publications. Other examples are a project that closely duplicates work ongoing in the member's laboratory, or an application from a scientist with whom the member has had longstanding differences that could reasonably be viewed as affecting the member's objectivity. If an application is submitted naming a participating individual from another institution, that individual is not considered to have a relationship with the applicant institution that constitutes a conflict of interest. Consequently, (1) that named individual may review other applications from the applicant institution; and (2) other individuals from the institution of the named individual may be used as reviewers for the submitted application, so long as any real or apparent conflict of interest is resolved. The SRA will document that there is no conflict of interest. For peer review consultants who are not federal employees, all separate organizational components/schools of multi-component academic institutions, hospitals, health centers, and research institutes may be considered to be sufficiently independent such that an employee of one component can review an application from another component without a conflict of interest, so long as any other real or apparent conflict of interest is resolved. In practice, for example, this means that:
however,
In addition, so long as any real or apparent conflict of interest is resolved: If an individual supplies a resource or service to an applicant, and that resource or service is freely available to anyone in the scientific community, neither the institution nor the individual supplying the resource is in conflict. For fellowship and K award applications, peer reviewers who write reference letters for an applicant are in conflict and must leave the room for the review of the application; this does not, however, constitute an institutional conflict. If the applicant's sponsor is a member of the SRG, this constitutes a member conflict for the study section (i.e., the study section may not review the application). For conference grant applications, the originators, planning group members, and proposed speakers are in conflict, but their institutions are not, and this situation does not generate a study section conflict. Reviewers from institutions that are part of a multi-center network (e.g., accrual sites for a multi-center clinical trial) are not in conflict with other applications/proposals from other institutions in the network; furthermore, reviewers from institutions that provide members of an applicant's Advisory Board or Data and Safety Monitoring Board are not in conflict with other applications/ proposals from those institutions. A reviewer must leave the room during discussion of an application if he/she is a member of, or has a financial interest in a for-profit organization submitting the application. This includes ownership of stock in, or being a consultant for a for-profit organization. A reviewer should also leave the room during discussion of an application if being present would give the appearance of a conflict of interest. Examples would be, an application from a for-profit organization that provides substantial financial funding to the reviewer's organization or laboratory. Prior to the scientific review group meeting, each reviewer will receive a certificate of Conflict of Interest and Confidentiality and a list of applications that will be reviewed. Reviewers must notify the Scientific Review Administrator of any conflict of interest prior to the meeting and certify that the confidentiality of the review procedures will be maintained. At the end of the scientific review group meeting, the SRA will obtain written certification from all members that they have not participated in any reviews of applications when their presence would have constituted a real or apparent conflict of interest and that the confidentiality of actions will be maintained. In addition, each study section keeps a log, prepared by the Grants Assistant and maintained in the study section office, of which members left the room because of potential conflict of interest and for which applications. CONFIDENTIALITY AND COMMUNICATIONS WITH INVESTIGATORS All materials pertinent to the applications being reviewed are privileged communications prepared for use only by consultants and NIH staff, and should not be shown to or discussed with other individuals. Review group members must not independently solicit opinions or reviews on particular applications or parts thereof from experts outside the pertinent initial review group. Members may, however, suggest scientists from whom the SRA may subsequently obtain advice. Consultants are required to leave all review materials with the SRA at the conclusion of the review meeting. Privileged information in grant applications shall not be used to the benefit of the reviewer or shared with anyone. Under no circumstances shall consultants advise investigators, their organizations, or anyone else of recommendations or discuss the review proceedings. The investigator may be led into unwise actions on the basis of premature or erroneous information. Such advice also represents an unfair intrusion into the privileged nature of the proceedings and invades the privacy of fellow consultants serving on review committees and site visit teams. A breach of confidentiality could deter qualified consultants from serving on review committees and inhibit those who do serve from engaging in free and full discussion of recommendations. Except during site visits, there must be no direct communications between consultants and investigators. Consultants' requests for additional information and telephone inquiries or correspondence from investigators must be directed to the SRA, who will handle all such communications. SCIENTIFIC MISCONDUCT "Misconduct" or "misconduct in science" is defined at 42 CFR 50.102 as fabrication, falsification, plagiarism, or other practices that seriously deviate from those that are commonly accepted within the scientific community for proposing, conducting or reporting research. It does not include honest error or honest differences in interpretation or judgments of data. Review of grant/cooperative agreement applications and contract proposals for scientific merit will ordinarily not be delayed by pending or ongoing inquiry or investigation. To avoid influencing the review process, HHS awarding units will not inform members of scientific review groups about instances of possible misconduct or the status of ongoing investigations. However, if certain instances have received such extensive publicity that the review may be compromised, the CSR Research Integrity Officer (RIO) will discuss the matter with the Agency Research Integrity Liaison Officer (ARILO). Findings from completed investigations should be shared with scientific review group when an accurate disclosure of the facts in the case is necessary for an objective and thorough review. The scientific review group should not review an application about which an allegation of misconduct has surfaced from one of its members. The SRA should report the allegation to the CSR RIO. The RIO will involve appropriate CSR staff and the ARILO in determining the manner in which the allegation will be treated In all cases of suspected misconduct, it is essential that the SRA stress to the reviewers the seriousness of such allegations and the potential harm that may result if confidentiality is not strictly maintained. In addition, it is important for the SRA to assure the reviewers that the suspicions identified will be taken seriously and pursued by the HHS. In no instance shall the SRA or a reviewer communicate the scientific review group's concerns to the principal investigator or applicant institution. |