Show simple item record

dc.contributor.authorMaramba, Inocencio Daniel
dc.contributor.authorChatterjee, Arunangsu
dc.contributor.authorNewman, Craig
dc.date.accessioned2019-04-11T08:53:24Z
dc.date.available2019-04-11T08:53:24Z
dc.date.issued2019-06
dc.identifier.issn1386-5056
dc.identifier.issn1872-8243
dc.identifier.urihttp://hdl.handle.net/10026.1/13696
dc.descriptionBackground The number of eHealth applications has exponentially increased in recent years, with over 325,000 health apps now available on all major app stores. This is in addition to other eHealth applications available on other platforms such as PC software, web sites and even gaming consoles. As with other digital applications, usability is one of the key factors in the successful implementation of eHealth apps. Reviews of the literature on empirical methods of usability testing in eHealth were last published in 2015. In the context of an exponentially increasing rate of App development year on year, an updated review is warranted. Objective To identify, explore, and summarize the current methods used in the usability testing of eHealth applications. Methods A scoping review was conducted on literature available from April 2014 up to October 2017. Four databases were searched. Literature was considered for inclusion if it was (1) focused on an eHealth application (which includes websites, PC software, smartphone and tablet applications), (2) provided information about usability of the application, (3) provided empirical results of the usability testing, (4) a full or short paper (not an abstract) published in English after March 2014. We then extracted data pertaining to the usability evaluation processes described in the selected studies. Results 133 articles met the inclusion criteria. The methods used for usability testing, in decreasing order of frequency were: questionnaires (n = 105), task completion (n = 57), ‘Think-Aloud’ (n = 45), interviews (n = 37), heuristic testing (n = 18) and focus groups (n = 13). Majority of the studies used one (n = 45) or two (n = 46) methods of testing. The rest used a combination of three (n = 30) or four (n = 12) methods of testing usability. None of the studies used automated mechanisms to test usability. The System Usability Scale (SUS) was the most frequently used questionnaire (n = 44). The ten most frequent health conditions or diseases where eHealth apps were being evaluated for usability were the following: mental health (n = 12), cancer (n = 10), nutrition (n = 10), child health (n = 9), diabetes (n = 9), telemedicine (n = 8), cardiovascular disease (n = 6), HIV (n = 4), health information systems (n = 4) and smoking (n = 4). Further iterations of the app were reported in a minority of the studies (n = 41). The use of the ‘Think-Aloud’ (Pearson Chi-squared test: χ2 = 11.15, p < 0.05) and heuristic walkthrough (Pearson Chi-squared test: χ2 = 4.48, p < 0.05) were significantly associated with at least one further iteration of the app being developed. Conclusion Although there has been an exponential increase in the number of eHealth apps, the number of studies that have been published that report the results of usability testing on these apps has not increased at an equivalent rate. The number of digital health applications that publish their usability evaluation results remains only a small fraction. Questionnaires are the most prevalent method of evaluating usability in eHealth applications, which provide an overall measure of usability but do not pinpoint the problems that need to be addressed. Qualitative methods may be more useful in this regard. The use of multiple evaluation methods has increased. Automated methods such as eye tracking have not gained traction in evaluating health apps. Further research is needed into which methods are best suited for the different types of eHealth applications, according to their target users and the health conditions being addressed.
dc.description.abstract

Background The number of eHealth applications has exponentially increased in recent years, with over 325,000 health apps now available on all major app stores. This is in addition to other eHealth applications available on other platforms such as PC software, web sites and even gaming consoles. As with other digital applications, usability is one of the key factors in the successful implementation of eHealth apps. Reviews of the literature on empirical methods of usability testing in eHealth were last published in 2015. In the context of an exponentially increasing rate of App development year on year, an updated review is warranted. Objective To identify, explore, and summarize the current methods used in the usability testing of eHealth applications. Methods A scoping review was conducted on literature available from April 2014 up to October 2017. Four databases were searched. Literature was considered for inclusion if it was (1) focused on an eHealth application (which includes websites, PC software, smartphone and tablet applications), (2) provided information about usability of the application, (3) provided empirical results of the usability testing, (4) a full or short paper (not an abstract) published in English after March 2014. We then extracted data pertaining to the usability evaluation processes described in the selected studies. Results 133 articles met the inclusion criteria. The methods used for usability testing, in decreasing order of frequency were: questionnaires (n = 105), task completion (n = 57), ‘Think-Aloud’ (n = 45), interviews (n = 37), heuristic testing (n = 18) and focus groups (n = 13). Majority of the studies used one (n = 45) or two (n = 46) methods of testing. The rest used a combination of three (n = 30) or four (n = 12) methods of testing usability. None of the studies used automated mechanisms to test usability. The System Usability Scale (SUS) was the most frequently used questionnaire (n = 44). The ten most frequent health conditions or diseases where eHealth apps were being evaluated for usability were the following: mental health (n = 12), cancer (n = 10), nutrition (n = 10), child health (n = 9), diabetes (n = 9), telemedicine (n = 8), cardiovascular disease (n = 6), HIV (n = 4), health information systems (n = 4) and smoking (n = 4). Further iterations of the app were reported in a minority of the studies (n = 41). The use of the ‘Think-Aloud’ (Pearson Chi-squared test: χ2 = 11.15, p < 0.05) and heuristic walkthrough (Pearson Chi-squared test: χ2 = 4.48, p < 0.05) were significantly associated with at least one further iteration of the app being developed. Conclusion Although there has been an exponential increase in the number of eHealth apps, the number of studies that have been published that report the results of usability testing on these apps has not increased at an equivalent rate. The number of digital health applications that publish their usability evaluation results remains only a small fraction. Questionnaires are the most prevalent method of evaluating usability in eHealth applications, which provide an overall measure of usability but do not pinpoint the problems that need to be addressed. Qualitative methods may be more useful in this regard. The use of multiple evaluation methods has increased. Automated methods such as eye tracking have not gained traction in evaluating health apps. Further research is needed into which methods are best suited for the different types of eHealth applications, according to their target users and the health conditions being addressed.

dc.format.extent95-104
dc.format.mediumPrint-Electronic
dc.languageen
dc.language.isoen
dc.publisherElsevier
dc.rightsAttribution 4.0 International
dc.rightsAttribution 4.0 International
dc.rightsAttribution 4.0 International
dc.rightsAttribution 4.0 International
dc.rightsAttribution 4.0 International
dc.rightsAttribution 4.0 International
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/
dc.subjecteHealth
dc.subjectmHealth
dc.subjectMobile
dc.subjectUsability
dc.subjectApplications
dc.subjectHealth informatics
dc.titleMethods of Usability Testing in the Development of eHealth Applications: A Scoping Review
dc.typejournal-article
dc.typeJOUR
plymouth.author-urlhttp://www.sciencedirect.com/science/article/pii/S1386505618313182
plymouth.volume126
plymouth.publication-statusPublished
plymouth.journalInternational Journal of Medical Informatics
dc.identifier.doi10.1016/j.ijmedinf.2019.03.018
plymouth.organisational-group/Plymouth
plymouth.organisational-group/Plymouth/Faculty of Health
plymouth.organisational-group/Plymouth/Faculty of Health/Peninsula Medical School
plymouth.organisational-group/Plymouth/REF 2021 Researchers by UoA
plymouth.organisational-group/Plymouth/REF 2021 Researchers by UoA/UoA03 Allied Health Professions, Dentistry, Nursing and Pharmacy
plymouth.organisational-group/Plymouth/REF 2021 Researchers by UoA/UoA23 Education
plymouth.organisational-group/Plymouth/Research Groups
plymouth.organisational-group/Plymouth/Research Groups/Institute of Health and Community
plymouth.organisational-group/Plymouth/Research Groups/Institute of Translational and Stratified Medicine (ITSMED)
plymouth.organisational-group/Plymouth/Research Groups/Institute of Translational and Stratified Medicine (ITSMED)/CCT&PS
plymouth.organisational-group/Plymouth/Users by role
plymouth.organisational-group/Plymouth/Users by role/Academics
dc.publisher.placeIreland
dcterms.dateAccepted2019-03-27
dc.rights.embargodate2020-3-30
dc.identifier.eissn1872-8243
dc.rights.embargoperiodNot known
rioxxterms.versionofrecord10.1016/j.ijmedinf.2019.03.018
rioxxterms.licenseref.urihttp://creativecommons.org/licenses/by/4.0/
rioxxterms.licenseref.startdate2019-06
rioxxterms.typeJournal Article/Review
plymouth.funderE-health Productivity and Innovation in Cornwall & Scilly::EU ERDF
plymouth.funderE-health Productivity and Innovation in Cornwall & Scilly::EU ERDF


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record

Attribution 4.0 International
Except where otherwise noted, this item's license is described as Attribution 4.0 International

All items in PEARL are protected by copyright law.
Author manuscripts deposited to comply with open access mandates are made available in accordance with publisher policies. Please cite only the published version using the details provided on the item record or document. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of content should be sought from the publisher or author.
Theme by 
Atmire NV