Abstract
Objective To assess the validity and reliability of the Sheffield Peer Review Assessment Tool (SPRAT) Japanese version for evaluating doctors’ competencies using multisource feedback. Methods SPRAT, originally developed in the UK, was translated and validated in three phases: (1) an existing Japanese version of SPRAT was back-translated into English; (2) two expert panel meetings were held to develop and assure content validity in a Japanese setting; (3) the newly devised Japanese SPRAT instrument was tested by a multisource feedback survey, validity was tested using principal component factor analysis, and reliability was assessed using generalisability and decision studies based on generalisability theory. Results 86 doctors who had been practising for between 2 and 33 years participated as assessees and were evaluated with the SPRAT tool. First, the doctors identified 1019 potential assessors who were each sent SPRAT forms (response rate, 81%). The mean number of assessors per doctor was 9.7 (SD=2.5). The decision study showed that 95% CIs of ±0.5 were achieved with only 5 assessors. 85 of the 86 doctors achieved scores that could be placed with 95% CI above the 4 expected standard. Doctors received lower scores from more senior assessors (p<0.001) and higher scores from those they had known longer (p<0.001). Scores also varied with the job role (p<0.05). Conclusions Following translation and content validation, the Japanese instrument behaved similarly to the UK tool. Assessor selection remains a primary concern, as the assessment scores are affected by the seniority of the assessor, the length of the assessor–assessee working relationship, and the assessor's job role. Users of the SPRAT tool need to be aware of these limitations when administering the instrument.
DOI
10.1136/bmjopen-2014-007135
Publication Date
2015-06-01
Publication Title
BMJ Open
Volume
5
Issue
6
Publisher
BMJ
ISSN
2044-6055
Embargo Period
2024-11-19
First Page
e007135
Last Page
e007135
Recommended Citation
Sasaki, H., Archer, J., Yonemoto, N., Mori, R., & et al. (2015) 'Assessing doctors’ competencies using multisource feedback: validating a Japanese version of the Sheffield Peer Review Assessment Tool (SPRAT)', BMJ Open, 5(6), pp. e007135-e007135. BMJ: Available at: https://doi.org/10.1136/bmjopen-2014-007135
