ORCID

Document Type

Article

Abstract

Background:To provide patients with gastric cancer with adequate health education information for effective overall management is crucial, while traditional manners exposed certain challenges. Conversational agents have increasingly been adopted for health care use to provide innovative solutions for patient education.Objective:This study aimed to develop a health education embodied conversational agent to focus on gastric cancer disease using an action research approach and test its accuracy, usability, and user experience among patients and other related stakeholders.Methods:The AI-guided conversational agent was developed based on the OpenMEDLab 2.0 foundation model and the Retrieval-Augmented Generation (RAG) architecture. A 4-phase action research approach was adopted to implement this system at a gastric cancer center in China. Diagnose and plan phase used participatory observation and in-depth interviews to explore current health education models and patients’ needs for health education. Act and implement phase was used to develop and deploy the conversational agent. Evaluate phase comprised 3 rounds of alpha testing to assess accuracy and RAG knowledge hit rate, and 1 round of beta testing to assess the usability and relevance. Reflect phase conducted in-depth interviews to gain insights into users' experiences. Data collection was conducted from September 2023 through April 2025. Participants include patients, clinical nurses, nursing managers, surgeons, clinical psychologists, and dietitians. Thematic analysis and multiple-group chi-square tests were performed, respectively, for qualitative and quantitative data. A 2-sided P value of <.05 was considered statistically significant.Results:A total of 44 patients, 13 nurses, 3 nursing managers, 2 surgeons, 1 clinical psychologist, and 1 dietitian were recruited during the study procedure. Favorable outcomes in terms of accuracy and usability were achieved. The accuracy of the agent in 3 rounds was 67% (37/55), 71% (44/62), and 82% (31/38), respectively; RAG knowledge hit rates reached 86% (47/55), 98% (61/62), and 100% (38/38). Significant differences (P<.01) in RAG knowledge hit rates were observed. The mean chatbot usability questionnaire score was 91.9 (SD 3.6), while the mean content relevance score was 3.75 (SD 0.9). Three themes of user experiences were identified: perceived usefulness, ease of use, and intention to use, revealing potential in reducing staff workload and reinforcing patient education.Conclusions:This study provided insights into how the action research approach can inform the development and usability assessment of a gastric cancer health education conversational agent, also illustrating the value of RAG technology. Additional assessments and improvements are warranted to confirm the effectiveness and safety.

Publication Date

2026-09-10

Publication Title

Journal of Medical Internet Research

Volume

28

ISSN

1439-4456

Deposit Date

2026-09-22

Funding

This research was supported by Shanghai Health Science Popularization Special Program (JKKPYL-2025-A01) and Fosun Fund (FNF202513). The funders had no role in the study design; collection, analysis, and interpretation of data; writing of the paper; and/or decision to submit for publication.

Keywords

health education, generative artificial intelligence, digital health, user-centred design, action research, user-centered design

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Download

Share

COinS