Show simple item record

dc.contributor.authorSorrell, L
dc.contributor.authorMcardle, N
dc.contributor.authorBecque, T
dc.contributor.authorPayne, H
dc.contributor.authorStuart, B
dc.contributor.authorTurner, S
dc.contributor.authorWyatt, JC
dc.date.accessioned2022-11-08T13:14:40Z
dc.date.available2022-11-08T13:14:40Z
dc.date.issued2018-12-14
dc.identifier.issn2044-6055
dc.identifier.issn2044-6055
dc.identifier.othere022547
dc.identifier.urihttp://hdl.handle.net/10026.1/19933
dc.description.abstract

<jats:sec><jats:title>Objectives</jats:title><jats:p>To evaluate the influence of external peer reviewer scores on the National Institute for Health Research (NIHR) research funding board decisions by the number of reviewers and type of reviewer expertise.</jats:p></jats:sec><jats:sec><jats:title>Design</jats:title><jats:p>Retrospective analysis of external peer review scores for shortlisted full applications for funding (280 funding applications, 1236 individual reviewers, 1561 review scores).</jats:p></jats:sec><jats:sec><jats:title>Setting</jats:title><jats:p>Four applied health research funding programmes of NIHR, UK.</jats:p></jats:sec><jats:sec><jats:title>Main outcome measures</jats:title><jats:p>Board decision to fund or not fund research applications.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>The mean score of reviewers predicted funding decisions better than individual reviewer scores (area under the receiver operating characteristic (ROC) curve 0.75, 95% CI 0.69 to 0.81 compared with 0.62, CI 0.59 to 0.65). There was no substantial improvement in how accurately mean reviewer scores predicted funding decisions when the number of reviewers increased above 4 (area under ROC curve 0.75, CI 0.59 to 0.91 for four reviewers; 0.80, CI 0.67 to 0.92 for seven or more). Reviewers with differing expertise influenced the board’s decision equally, including public and patient reviewers (area under ROC curves from 0.57, CI 0.47 to 0.66 for health economists to 0.64, CI 0.57 to 0.70 for subject-matter experts). The areas under the ROC curves were quite low when using reviewers’ scores, confirming that boards do not rely solely on those scores alone to make their funding decisions, which are best predicted by the mean board score.</jats:p></jats:sec><jats:sec><jats:title>Conclusions</jats:title><jats:p>Boards value scores that originate from a diverse pool of reviewers. On the basis of independent reviewer score alone, there is no detectable benefit of using more than four reviewer scores in terms of their influence on board decisions, so to improve efficiency, it may be possible to avoid using larger numbers of reviewers. The funding decision is best predicted by the board score.</jats:p></jats:sec>

dc.format.extente022547-e022547
dc.format.mediumElectronic
dc.languageen
dc.language.isoeng
dc.publisherBMJ Publishing Group
dc.subjectpeer review
dc.subjectresearch funding
dc.subjectscore
dc.subjectDecision Making
dc.subjectNational Health Programs
dc.subjectPeer Review, Research
dc.subjectResearch Support as Topic
dc.subjectRetrospective Studies
dc.subjectUnited Kingdom
dc.titleInfluence of external peer reviewer scores for funding applications on funding board decisions: a retrospective analysis of 1561 reviews
dc.typejournal-article
dc.typeReview
plymouth.author-urlhttps://www.ncbi.nlm.nih.gov/pubmed/30552251
plymouth.issue12
plymouth.volume8
plymouth.publisher-urlhttp://dx.doi.org/10.1136/bmjopen-2018-022547
plymouth.publication-statusPublished
plymouth.journalBMJ Open
dc.identifier.doi10.1136/bmjopen-2018-022547
plymouth.organisational-group/Plymouth
plymouth.organisational-group/Plymouth/Faculty of Health
plymouth.organisational-group/Plymouth/Faculty of Health/Peninsula Medical School
plymouth.organisational-group/Plymouth/Users by role
plymouth.organisational-group/Plymouth/Users by role/Academics
dc.publisher.placeEngland
dcterms.dateAccepted2018-10-18
dc.rights.embargodate2022-11-9
dc.identifier.eissn2044-6055
dc.rights.embargoperiodNot known
rioxxterms.versionofrecord10.1136/bmjopen-2018-022547
rioxxterms.licenseref.urihttp://www.rioxx.net/licenses/all-rights-reserved
rioxxterms.typeJournal Article/Review


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record


All items in PEARL are protected by copyright law.
Author manuscripts deposited to comply with open access mandates are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author.
Theme by 
Atmire NV