What is the value of simulated patient assessment in structured clinical examinations of undergraduate students and how should it be used?

Abstract

<jats:title>Abstract</jats:title><jats:sec><jats:title>Introduction</jats:title><jats:p>The aim of this study was to explore correlations between clinical assessor and simulated patient (SP) scores drawn from summative Integrated Structured Clinical Examination (ISCE) and inform the best use of SP scores in future assessments.</jats:p></jats:sec><jats:sec><jats:title>Materials and Methods</jats:title><jats:p>This retrospective study explores summative clinical assessor and formative SP numeric scores drawn from summative ISCE assessments spanning three academic years (2017–18 to 2019–20). Analyses were carried out using R 3.5.1 (R Core Team, 2018), with the stats package.</jats:p></jats:sec><jats:sec><jats:title>Results</jats:title><jats:p>The sample consisted of 169 final‐year BDS students across the three cohorts and included 95 females (56.2%) and 74 males (43.8%). Data from eight substations where SPs were included, were explored. Kendall's Tau, a non‐parametric correlation, was used to investigate the relationships between the assessor and SP scores. Clinical assessor scores were out of a total of 20 points across various assessed domains within each substation. The formative SP assessment was out of 10 points with the same five affective domains related to communication included in each substation. Overall, the assessor and patient substation scores were not correlated (τ = 0.04, <jats:italic>p</jats:italic> = .272) indicating that communication skills alone, as assessed by patients, do not correlate with more holistic performance across other domains. There was significant positive correlation for two of the eight substations with the other substations showing very little correlation.</jats:p></jats:sec><jats:sec><jats:title>Conclusion</jats:title><jats:p>This study shows that assessment of student performance by SPs does not show a correlation with examiner scores and may provide additional information relating to affective skills of students. Notwithstanding the limitations of this study, the findings underscore the need to investigate further the value of involvement of SPs in clinical assessments to explore if scores by SPs can be used to enhance the validity of assessments if used summatively.</jats:p></jats:sec>