We recently published a book, Getting research funded: Five essential rules for early career researchers. This post provides excerpts from chapter 2: Be the funding body. Until the end of 2023, you can buy the book for 20% off by using the code AFL03 on the Routledge website.
Funders can seem a little like the Oracle of Delphi in ancient Greece. For four centuries, supplicants approached the Oracle, seeking wisdom and hoping for prophecies of good fortune. No one was entirely sure how she arrived at her decision. All were slightly fearful. None looked too closely or too directly at her. But all knew the outcome was final: there was no appeal process.
However, the process for funders making their decisions – and the rationale underlying it – isn’t necessarily as opaque as it may seem at first. As Lewis et al. (2019) made clear, ‘every funder has a mission statement that declares the scope of research they are interested in, and many funders have statements about their current priorities. Read. Those. Statements. Carefully. And incorporate them into your proposals’. As well as Reading. These. Statements. Carefully, you should also take time to understand how the funders work so that you have an idea of the process it will go through.
There are several different ways that organisations may select applications for funding. Generally, there’s a panel of reviewers or assessors and they judge the applications according to the funding scheme’s selection criteria. The composition of the reviewing panel can be quite diverse. Sometimes, the members are all academics. Other times, it’s a blend of academics, funding organisation executives, stakeholders or community leaders.
A significant number of funders of research opt for peer review when it comes to selecting applications. Peer review of grant applications differs from peer review of journal articles in one important way – it is ‘single-blind’. The reviewers know the identity of the applicants, but the applicants are not told the identity of their reviewers. The identity of the applicants is revealed because it can have a significant bearing on the feasibility of the research proposed. The identity of the reviewers is protected so that they can speak frankly, without fear of reprisals from the applicants.
How do the reviewers make the decision about the viability and excellence of your research? How can they tell if your project fits the purpose of the funder’s scheme? In this post, we look at the criteria the funders use to assess applications, and how these criteria may make or break your proposal at the different stages of the assessment process.
Abdoul et al. (2012) examined the use of criteria by a range of French and international funding organisations, including the regional Programme Hospitalier de Recherche Clinique (PHRCs), the Agence Nationale de la Recherche (ANR), the US National Institutes of Health, UK Medical Research Council (MRC), Canadian Institutes of Health Research (CIHR), the National Health and Medical Research Council (NHRMC) in Australia and the European Science Foundation (ESF).
In the 14 calls they looked at, all had a set range of criteria (between three and eight, with the median being five), and all had some form of scoring for these. Abdoul et al. created a ‘typology’, grouping together the criteria that funders in their study used. Hug and Aeschbach (2020) undertook a similar analysis, mapping criteria against the parts of the proposal. Both are a useful way of thinking about what funders see as important for judging the value and plausibility of applications. For Abdoul et al., the nine broad categories of criteria are as follows:
Most commonly used
- Originality: this is crucial. Will your project significantly shift understanding in your discipline? Has it been done before? What impact will it have?
- Usefulness: this is related to ‘originality’, but looks more at the overall ‘state of the art’ and how your proposal fits within it.
- Methodology: once again, this is an important factor and puts meat on the bones of your proposal. In weighted scoring systems, a high weight was given to this criterion. The review guidelines included specific questions about numerous methodological issues such as sample size estimation and quality of the study design.
- Feasibility: this included an assessment of the applicant’s track record and environment. Porter (2005) reported that many panellists see track record as an essential de facto indicator of whether the project is viable.
Less commonly used
- Ethical issues: including potential risks to patients. This, of course, is far more important in medical and health proposals. It was often assessed qualitatively, as opposed to score.
- Financial issues: the planning of the project and description of necessary resources. Are they appropriate, and do they offer value for money?
- Innovativeness: the technological, technical or methodological innovations used or investigated in the research project.
Rarely used
- Relevance to the call: This is important and emphasised by the officer from the funder at the beginning of the peer review meeting, but it is not usually a formal criterion.
- Writing or readability of the proposal: Although this is a critical factor in how reviewers and panellists assess applications, it is not usually a formal criterion for assessing an application.
Interestingly, in questioning reviewers and panellists about the criteria checklist that they are asked to use, ‘most of the reviewers completed the checklists at the end of the review process, as a means of supporting rather than of developing their opinion’. When asked by the researchers which of the criteria they found most important in assessing an application, ‘originality’, ‘methodology’, ‘scientific interest’ and ‘feasibility’ came out on top.
I co-authored this book with the fabulous Tseen Khoo and Jonathan O’Donnell, co-founders of The Research Whisper. When you get a chance, do have a look at their site for insights, hints and tips on all things research.
Tseen Khoo is co-founder of The Research Whisperer and a Senior Lecturer in Research Education and Development at La Trobe University, Australia.
Jonathan O’Donnell is co-founder of The Research Whisperer and a Senior Research Initiative Coordinator in the Faculty of Science, University of Melbourne, Australia. You’ll find more by Jonathan and Tseen at the Research Whisperer blog.
Photo by Glenn Carstens-Peters on Unsplash