Validation of Multisource Feedback in Assessing Medical Performance: A Systematic Review

Abstract

Introduction Over the past ten years, a number of systematic reviews have evaluated the validity of multisource feedback (MSF) to assess and quality assure medical practice. The purpose of this study is to synthesise the results from existing reviews to provide a holistic overview of the validity evidence. Methods This review identified eight systematic reviews evaluating the validity of MSF published between January 2006 and October 2016. Using a standardised data extraction form, two independent reviewers extracted study characteristics. A framework of validation developed by the American Psychological Association (APA) was used to appraise the validity evidence within each systematic review. Results In terms of validity evidence, each of the eight reviews demonstrated evidence across at least one domain of the APA validity framework. Evidence of assessment validity within the domains of ‘internal structure’ and ‘relationship to other variables’ has been well established. However, the domains of content validity (i.e. ensuring MSF tools measure what they are intended to measure); consequential validity (i.e. evidence of the intended or unintended consequences MSF assessments may have on participants or wider society) and response process validity (i.e. the process of standardisation and quality control in the delivery and completion of assessments) remain limited. Discussion Evidence for the validity of MSF has, across a number of domains, been well-established. However, the size and quality of the existing evidence remains variable. In order to determine the extent to which MSF is considered a valid instrument to assess medical performance, future research is required to determine: 1) how best to design and deliver MSF assessments that address the identified limitations of existing tools, and 2) how to ensure involvement within MSF supports positive changes in practice. Such research is integral if MSF is to continue to inform medical performance and subsequent improvements in the quality and safety of patient care.