Evaluating the Accuracy and Usability of the WHO Skin NTD App as a capacity building tool in Kenya

Background

Skin Neglected Tropical Diseases (Skin NTDs) are prevalent in endemic areas with limited resources for management. The WHO SkinNTDs App aids Frontline Health Workers (FHWs) in diagnosing these conditions. Version 3.0 of the app has been tested for usability and user experience in Kenya and Ghana. This study evaluates a new beta version of the app, incorporating an AI algorithm to identify 13 Skin NTDs and 24 common skin conditions, under real-life conditions in Kenya. The study assesses the usability, user experience, and accuracy of the AI-enhanced SkinNTDs app.

Methods

A cross-sectional study was conducted in Kenya with 40 FHW from 5 counties, using the app in their daily routines from June to October 2024. Two workshops were held: the first to train participants on app usage, patient recruitment, and image uploading; the second to conduct the assessment study. Participants completed an online survey using the user Mobile App Rating Scale (uMARS) questionnaire, followed by four focus groups and 16 semi-structured interviews. To evaluate the AI algorithms’ accuracy, 605 images were uploaded by participants, and the AI diagnoses were compared to consensus diagnoses from 3 independent expert dermatologists.

Results

The AI algorithms were tested with 605 images of skin lesions. The NTD AI algorithm achieved an overall accuracy of 78%, with higher accuracy for endemic diseases in Kenya. The common skin conditions AI algorithm had an accuracy of 80%. The SkinNTDs App received high scores on the uMARS questionnaire (App quality mean 3.82/5) comparable to the previous version tested in Kenya and Ghana. Preliminary qualitative analysis indicated that the app empowered healthcare professionals, boosting their self-esteem and trust from colleagues and patients, while saving time and reducing referrals to specialists.

Conclusions

The beta version of the SkinNTDs App demonstrates commendable quality and potential as a capacity-building tool for FHWs in Kenya. Key messages • WHO SkinNTDs App enhances diagnostic accuracy and usability, empowering Kenyan healthcare workers and improving management of skin NTDs and common conditions. • AI-powered digital health solutions can significantly enhance the capacity building of healthcare workers in low and middle-income countries.