The Journey of an Application: The CDMRP Two-Tier Review Process

Hello, and welcome! Great! You submitted a funding application to the CDMRP! In this video, we review the journey of your application through our two-tier application review process and discuss how programs make funding recommendations. Let's get started!

The application you submitted will undergo a two-tier review process consisting of peer and programmatic reviews. Research programs publish approximate dates for the peer and programmatic reviews on the first page of all funding opportunities. Our two-tier review process balances funding the most meritorious research projects with projects that also offer the highest promise to fulfill the programmatic goals described in each funding opportunity. To meet this goal, each step of the review process involves dedicated objectives and outcomes so that the two tiers are complementary, not redundant.

The first tier of the review process is peer review. This is a criteria-based process where your application is individually evaluated based on its scientific and technical merit relative to an ideal application.

Peer reviewers use specific evaluation criteria described in each funding opportunity. Applicants can use the evaluation criteria as a rubric to guide their application writing processes.

It's important to note that funding opportunities describe both scored and unscored criteria. Although the unscored criteria do not receive individual criterion scores, peer reviewers do factor those criteria into the overall scores.

Applicants should also note whether the scored criteria are weighted equally or whether the criteria are listed in descending order of importance and take this into consideration when composing their applications.

CDMRP programs recruit peer reviewers after receipt of the full application submission, meaning there are no standing peer review panels. The peer review panels include the experts needed to evaluate the applications received that fiscal year. Peer reviewers include scientists, clinicians, consumers, and specialists, as needed.

CDMRP uses the term "consumers" to refer to a survivor, person living with, family member or caregiver of an individual with the disease, injury, or condition.

All CDMRP peer review panels include consumer reviewers, and they provide important perspectives on the potential impact of the proposed research. The lay abstract and the impact statements are two documents that consumer reviewers rely on heavily during peer review. Because of this, it is important that you clearly describe your proposed research and the anticipated impact in a way that can be easily understood by non-scientists. You will not know the identities of the individuals who specifically reviewed your application because CDMRP policy prohibits contact between peer reviewers, applicants, and program staff. However, after peer review, program staff publishes lists of that program's peer reviewers on the program- specific web page.

The outcome from peer review is the Summary Statement. This statement includes a score for overall merit, individual evaluation scores, and a written critique of the strengths and weaknesses of the application. If relevant, discussion notes from the live discussions during the peer review meeting may also be captured.

The overall score represents an overall assessment of the application merit. Scores range between 1 and 5, with 1 representing the highest merit and 5 the lowest merit. Scores are averaged among all peer review panel members and are reported on the first page of the Summary Statement.

Each application is also rated according to the peer review evaluation criteria published in the funding opportunity. The most important peer review criteria receive scores from each reviewer, using a scale of 1 to 10; 1 represents the lowest merit and 10 the highest merit, using whole numbers only.

Additional review criteria are not individually scored but are discussed by the panel and may impact the overall score.

The Summary Statement not only provides the primary source of feedback that you will receive regarding your application, but it also is used during the second tier of review, programmatic review. While you will not receive any communication from CDMRP after peer review is complete, the program will send the Summary Statement to you with the funding recommendation after programmatic review.

Programmatic review is conducted by a multidisciplinary panel composed of consumers, clinicians, laboratory scientists, and other key stakeholders representing the program's field. Although the programmatic panel members may have similar areas of expertise as peer reviewers, the panel members are kept separate. In other words, no one can serve as both a peer reviewer and programmatic panel member in the same fiscal year.

For information on the programmatic panel members for a particular program, you can view these lists on the respective program web pages.

Unlike peer review, programmatic review is a comparison-based process where the highest scoring applications from peer review are assessed based on the programmatic review criteria described in the respective funding opportunities.

The programmatic panel aims to recommend a broad portfolio of applications for funding that aligns with the investment strategy for the program cycle and reflects the program's priorities. The programmatic panel does not re-review the scientific or technical merit of the applications. In fact, depending on the program, the programmatic panel may not even receive the entire application. Instead, they may only receive the Summary Statement, the Abstract, and selected attachments that are needed to address the programmatic review criteria specified in the funding opportunity.

When working on your application, please carefully follow the instructions in the funding opportunity and include the correct information within the correct application component/attachment.

Please note, CDMRP programs do not use a pay line, and applications that receive the highest scores at peer review are not automatically recommended for funding at programmatic review. Rather, peer review ratings and evaluations are just a subset of the criteria used by programmatic reviewers.

In the example shown here, you can see all the programmatic criteria listed for this particular funding opportunity. Programmatic review criteria may include considerations such as adherence to the intent of the mechanism, relative innovation and/or impact compared to the other applications under consideration, relevance to program goals, and consideration of the portfolio composition.

As the last step of programmatic review, the programmatic panel recommends a list of awards to fund. For some programs, the programmatic panel may also compose an "alternate list" or a list of awards that are not initially recommended for funding. However, they may be reconsidered if additional program funds become available for investment in research during that fiscal year. For example, if one of the applications originally recommended for funding is withdrawn then an application on the alternate list could then be recommended for funding as a replacement.

Once the recommended applications receive final approval, all applicants receive notification of whether their application is recommended for funding or not, and the notification comes approximately three-to-six weeks after the conclusion of programmatic review.

If the front page of the funding opportunity says that programmatic review will be held in January, then applicants should be notified of their application status by mid-March, if not earlier. All applicants receive their peer review Summary Statement at this time, in addition to an information paper that provides the funding metrics for each funding opportunity and an overview of the two-tier review process. This is the only feedback reviewer receive about their application. Applicants not recommended for funding also receive brief feedback on the primary reason they were not selected in their notification letter. Investigators whose applications are not recommended for funding often want to know if they can address reviewer comments and resubmit. Although resubmissions are permitted for some other funding agencies, the CDMRP does not have a resubmission process like this.

Should your application not receive funding, you are welcome to update or revise your applications at your discretion to submit the project to a future funding opportunity, assuming the application still meets the needs of the program and intent of the funding opportunity. Any such submissions are treated as new applications, and will be reviewed as such, without any consideration of previous applications.

The single most important tip about applying to funding from the CDMRP is to carefully read the entire funding opportunity and the associated general application instructions. CDMRP does not provide one-on-one guidance to applicants, such as whether their research is something the program is interested in funding. Rather, CDMRP develops the funding opportunity and general application instructions to contain all the information applicants need to address in their applications, such as focus areas, required application elements, eligibility, and deadlines. All applicants receive the same information.

Again, the funding opportunity and general application instructions will have all the information that you need, and CDMRP will not provide any additional guidance regarding the content of your application.

Good luck on your applications! We look forward to reading them.