|
has
provided the following short article by Tony Scarpa that was published
in the January 6, 2006 edition of Science describing his proposed
innovations to the NIH grant review process, including: 'virtual'
study sections; downsized grant applications / appendices; and shortening
the 9 month review period. The latter idea will be tested in a pilot
study, beginning in February 2006. Also to be piloted is "knowledge-management
software for speeding the referral of applications to the appropriate
NIH institute or center and to study sections." Some more detail
of planned and ongoing initiatives at CSR are described in the text
below.
Toni Scarpa*
The National Institutes of Health now has a $28 billion annual budget,
with more than 80% going to support outside training and research,
including grants to more than 3000 universities, medical schools,
and other research institutions in the United States and abroad.
These funds are allocated in a competitive process, using peer review
by independent scientists to identify proposals with the most scientific
merit for possible funding. It is a system revered by many, but
it faces clear challenges.
Much has changed since the NIH grants program was established 60
years ago (1), when we received 800 grant applications. We now get
~80,000 applications a year. The NIH Center for Scientific Review
(CSR) reviews and assigns priorities to two-thirds of these applications,
recruiting each year more than 15,000 outside scientific experts
in various specialized fields. Our regular reviewers volunteer 1
to 3 months a year to read and to assess applications. They then
meet three times a year with other members of their review group
to discuss and to score applications.
After an application has been assessed by the CSR-managed study
section, CSR sends to the NIH institutes and centers a summary statement
containing a score (and, in most cases, a percentile ranking), the
peer reviewer's comments, and a résumé of review discussions.
In the second level of peer review, advisory councils make funding
recommendations on the basis of these summaries, taking into consideration
the institute or center's scientific goals and public health needs.
Other nations provide direct funding to their research institutions
for researcher salaries and facilities and have relatively smaller
grant programs to provide additional funds. Although U.S. researchers
are hired and formally paid by their academic institutions and medical
schools, they depend on NIH and a few other competitive grants programs
to support a large and growing fraction of their salaries and laboratory
expenses. A scientist who fails to obtain a grant is now less likely
to achieve tenure. A new investigator who does not attract grants
will soon be forced to consider another career.
Getting a grant is more difficult now as NIH is no longer seeing
rapid budget increases. The percentage of grant applications funded
is shrinking, partly as a result of multiple applications from the
same individuals and, to a lesser degree, as a result of an increase
in the number of researchers submitting applications. What makes
this situation particularly difficult is that the business plans
of most universities and medical schools assume continued growth.
In addition to increased competition for NIH funds, the scope, pace,
and complexity of biomedical research has increased dramatically
in recent years, and the needs of the research community have changed
significantly. An exciting and innovative idea can quickly become
dated, and researchers now have less protected time to write applications,
to mentor new investigators, or to serve on peer-review groups.
All of these realities have put tremendous pressures on the NIH
application review system. Our process must evolve to address these
new realities.
Competitive pressures have pushed researchers to submit more conservative
applications, and we must find ways to encourage greater risk-taking
and innovation and to ensure that our study sections are more receptive
to innovative applications. We also must find ways to address the
fact that clinical research applications may receive slightly less
favorable scores than other applications (2).
NIH peer reviews are only as good as the reviewers we are able to
recruit, and we must do more to enlist the best reviewers. Although
face-to-face meetings of reviewers are the ideal and will continue,
we are experimenting with new electronic technologies that permit
reviewers to have discussions with greater convenience and to spend
less of their precious time in traveling. For example, asynchronous
Internet-assisted discussions--secure chat rooms--allow reviewers
to "meet" and to comment independently of time as well
as place.
The investigator-initiated (R01) grant application may be the world's
longest of its type--25 dense pages with unlimited appendices. Producing,
reviewing, revising, and re-reviewing them takes an enormous amount
of time for applicants and reviewers. We should look into whether
these multipage applications and appendices are really useful, or
if they are an unnecessary burden on reviewers. The seemingly endless
appendices may never be read in full and some may never be read
at all.
Finally, many have asked if the present two-tiered, 9-month review
process should be shortened so that we can keep pace with the science
and be more responsive to our applicants. Our system can be particularly
frustrating for those who may need to make only minor revisions,
because results from our reviews typically come too late for them
to reapply for the next review round.
CSR has taken two steps to speed its review process. As of October
2005, we are posting summary statements of most reviews within 1
month after the study section meeting instead of 2 to 3 months after
the meeting.
In February 2006, we will begin a pilot study to cut 11⁄2
months from our review process as recommended by a Trans-NIH Committee
to Shorten the Review Cycle (3). Forty CSR study sections will participate
in this pilot, which will speed the reviews of R01 applications
submitted by new investigators. Resubmission deadlines will be extended
to allow these new investigators to resubmit immediately if only
minor revisions are necessary. Specifically, we will: (i) schedule
study section meetings up to a month earlier; (ii) provide scientists
their study section scores, critiques, and panel discussion summaries
within a week after the section meeting; (iii) shave days from the
internal steps involved in assigning proposals to study sections;
and (iv) extend resubmission deadlines by 3 weeks.
If the pilot succeeds, we will seek to expand it. One major step
needed before expansion should be completed by October 2006, when
all R01 applications must be submitted electronically. CSR also
plans to test knowledge-management software for speeding the referral
of applications to the appropriate NIH institute or center and to
study sections. We will also seek to use this software to speed
assignment of applications to reviewers. In initial tests, this
software appears quite promising.
Change can be difficult when so much is at stake, but it is urgently
needed. For NIH to meet its mission and for science to advance,
we must work together. I encourage anyone with a good idea or suggestion
to contact me.
References and Notes
1. R. Mandel, A Half Century of Peer Review, 1946-1996 (NIH Division
of Research Grants, Bethesda, MD, 1996).
2. T. A. Kotchen, T. Lindquist, K. Malik, E. Ehrenfeld, JAMA 291,
836 (2004).
3. "Pilot study to shorten the review cycle for new investigator
R01 applications," in NIH Guide to Grants and Contracts (Notice
NOT-OD-06-013, 30 November 2005, NIH Office of Extramural Research,
Bethesda, MD, 2005); (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-06-013.html).
|
