The evaluation of health recommender systems: A scoping review

ORCID

Abstract

BackgroundPeople often look online for information about health concerns, but the vast amount of available and unregulated information can cause misinformation and potential harm. Health recommender systems (HRSs) can address this issue by recommending personalised health information. Previous research has evaluated individual systems, but there is a lack of reviews synthesising their evaluation findings. Such a synthesis is needed to ensure that future recommender designs have a positive impact on target health or behavioural outcomes.ObjectiveThis review aimed to provide a summary of the evidence obtained from previous studies evaluating HRSs and highlight methodological considerations and gaps in the current research.MethodsThe review was developed using the PRISMA-ScR and PICOS frameworks. PubMed, ACM Digital Library, IEEE Xplore, Web of Science, ScienceDirect, and Scopus were searched for studies that evaluated at least one HRS and involved human participants. A descriptive analysis was conducted on included studies and key themes and gaps in the literature were assessed.Results36 papers evaluating 34 HRSs were included. The systems targeted 13 health conditions and recommended different item types. Evaluation designs varied, with sample sizes ranging from 1 to 8057, and study durations from a single session to three years. A variety of outcome measures was used, including accuracy, engagement, clinical or behavioural outcomes, and participant perspectives.ConclusionsThe number of studies about HRSs is increasing, but there is a distinct lack of robust scientific research. The heterogeneity of outcome measures made it difficult to draw conclusions about their efficacy, but the data suggest that HRSs can help with the self-management of a wide range of conditions. There is a need to strengthen the available early-stage evidence with further research, evaluating multiple outcome measures including clinical outcomes, usability, and acceptability over a longer period to show real-world impact.

Publication Date

2024-11-14

Publication Title

International Journal of Medical Informatics

ISSN

1386-5056

This document is currently not available here.

10.1016/j.ijmedinf.2024.105697" data-hide-no-mentions="true">

Share

COinS