Referral & Review

How Scientists are Selected for Study Section Service

Balancing experience and diversity when developing study section member rosters is one of the most challenging tasks a Scientific Review Administrator (SRA) faces. Study section membership is generally a four-year commitment, involving three meetings per year.

  1. Selection Criteria

General Requirements

  • Candidates must be recognized authorities in their field.
  • Candidates must be a principal investigator on a research project comparable to those being reviewed. 
  • There must be diversity with respect to the geographic distribution, gender, race and ethnicity of the membership. 
  • Candidates must be dedicated to high quality, fair reviews.

Expertise Requirements

  • Expertise is the paramount consideration when developing/updating a study section roster. 
  • Each scientific area reviewed by the study section needs appropriate expert representation. 
  • The SRA must ensure that the study section does not become static. Care must be taken to ensure that the study section remains responsive to emerging areas of science and shifting scientific boundaries. 
  • It is important to consider that one-fourth of study section members will rotate off each year. This could dramatically affect the breadth of a study section's expertise without proper long-term planning.

Study Section-Specific Requirements 

  • Unique characteristics of study sections must be factored into selection of members. The breadth of science, the multidisciplinary or interdisciplinary nature of the applications, and the types of applications or grant mechanisms being reviewed play a large role in the selection of appropriate members.

Examples: 

  1. Study sections that review multidisciplinary or interdisciplinary applications have a greater need for scientists who have broader expertise or who have demonstrated the capacity to appreciate and evaluate areas of science outside their immediate area of expertise.
  2. Study sections covering clinically oriented research have a greater need for reviewers who are clinicians. 
  3. Study sections reviewing bioengineering or bioinformatics applications or applications involving partnerships with small businesses have a greater need for scientists who work in non-academic settings. 
  • Group dynamics should be considered when selecting study section members. 

Examples: 

  1. There is a need for balance in the level of seniority represented among members of a study section. Too many senior-level reviewers are just as problematic as too few. 
  2. There is a need to balance those who are generalists and provide the broader perspective needed for evaluation of the overall impact of a given project and those who are specialists and provide a more focused perspective needed to ensure proper evaluation of feasibility. 
  3. For study sections that cover multiple scientific areas or disciplines within the context of a common theme, there is a particular need for reviewers who bridge these areas or disciplines so as to prevent factions from developing within the study section.

Individual Reviewer Qualifications

  • Fairness and objectivity are the most important criteria for a reviewer. 
  • Reviewers need to be able to articulate their views succinctly, engage in productive exchanges, actively participate in the discussion of applications other than those specifically assigned, and demonstrate an ability to work collegially in a group setting.
  • Reviewers who are able to facilitate or help focus the discussion are particularly valued, as are those who remain actively engaged in ensuring the fairness and consistency of the scoring practices within the group throughout the meeting.
  1. The Nomination Process

Identifying Potential Reviewers

  • SRAs have many sources of information available to assist them in identifying potential study section members:

Examples: 

  1. Recent scientific literature in the area covered by the study section
  2. Scientific meetings that allow for the identification and evaluation of potential members
  3. The list of successful grant applicants within a given area of scientific expertise
  4. Present and former study section members and Chairs (although care must be taken to ensure this does not lead to over-representation of a given subset of scientists within a given scientific area)
  5. NIH program staff within the relevant Institutes served by the study section 
  6. Institute Advisory Councils 
  7. Major scientific societies served by a particular study section are increasingly offering to contribute the CVs of individuals they would recommend for service
  8. Individuals interested in serving on a study section are free to submit their CVs directly to the SRA of a given study section

Selecting Study Section Members

  • After identifying potential reviewers, further information is needed regarding: 
  1. Their NIH or other agencies grant history 
  2. Their publication history 
  3. Their professional status and/or record of accomplishments 
  4. Their review experience 
  • In terms of review experience, it is particularly important to determine: 
  1. Whether these potential study section members are currently serving on any other study section (concurrent service on two study sections as a member of one and an ad hoc reviewer on another is allowed). 
  2. Whether they are serving on an Institute's Council (concurrent service on an Institute's Council and a study section, even as a temporary reviewer, is not allowed). 
  3. Whether they have had prior review experience either as a temporary member or as a previous study section member (a second term is allowed, but only after an absence of at least a year). 
  • As a part of the selection process, most individuals are asked to first serve on the study section as a temporary reviewer, since the reviewer's objectivity and ability to work in a group are important considerations for membership. Service as a temporary reviewer is a mechanism for preparing reviewers for regular study section membership as well as a means for bringing needed expertise and a fresh perspective to a study section.

Preparing the Nomination Package

  • For individuals selected for "permanent" study section membership a nomination package is compiled by the SRA annually. 
  • The charter for each study section specifies the number of permanent study section members allowed, although temporary reviewers frequently constitute a significant percentage of the actual review panel at a given meeting. 
  • The number of permanent members on a study section is determined by the typical number of applications reviewed by that study section, the complexity of the applications reviewed, and the breadth of science covered by the study section. 
  •  The nomination package consists of:
  1. A cover letter that addresses both the past and present scientific review needs of the study section regarding the breadth of science covered and number of applications typically reviewed as well as the level of seniority and the geographic, gender, race and ethnic diversity of both the current and proposed membership. 
  2. The nomination slate, which identifies those being recommended for membership, their areas of expertise and terms of service. 
  3. Documentation in support of the nominations, including the curriculum vitae or NIH Biosketch of each candidate, their record of grant support and/or evidence of their stature in the field, prior review experience, and the rationale for their selection, including an indication of the validation of specific nominees from independent sources.

Obtaining Approval of the Nomination Slate

  • The nomination package prepared by the individual SRA is reviewed from varying perspectives within the Center for Scientific Review prior to its review within the Offices of the Director of NIH.
  • Rejection of the nomination package at any level sends it back to the SRA for revision and the process is repeated until final approval is obtained from the Director of NIH.
  • Approval of the nomination slate takes the following path: 
  1. The nomination package is prepared by the SRA and reviewed by the IRG Chief. Subsequent to approval the package is sent to the Division Director.
  2. After approval at the Division level, the CSR Committee Management Office (CMO) reviews the package. 
  3. Once the CMO approves the nomination package, it is sent to relevant Institute program staff for comment. 
  4. If no concerns are expressed it is then presented to the CSR Director for approval.
  5. The package is then sent to the central NIH Committee Management office for evaluation.
  6. If the nomination package is found acceptable, it is sent to the Director of NIH for final signature approval. 
  • If no problems are identified, the average time from preparation by the SRA to approval at the level of the CSR Director is generally six weeks and the average time from approval at the level of the CSR Director to final approval at the level of the NIH Director is approximately another six weeks (12 weeks total).

The process of preparing and approving nomination slates is designed to help ensure high quality study section membership. While the process is somewhat cumbersome and may not be perfect, empirically it has proven an effective way to select appropriate and effective review panels.