The toolkit's content was meticulously developed to ensure clarity and accuracy in classifying DHIs services and applications (see Fig. 3). The toolkit was organized into four main sections: introduction, guide to use, classification, and references. The introduction section provides important definitions related to the classification of DHIs and applications and services, an introduction to the toolkit for CDISAH, and a guide to using the toolkit. The toolkit is available for use and can be accessed via (https://forms.fillout.com/t/1hjbK9pz6gus).

Fig. 3

Domains of the toolkit for Classification of Digital Health Interventions Services and Applications

Classification criteria were defined clearly and precisely, adhering to WHO CDISAH three axes. Detailed explanations and practical examples were included to illustrate the application of each criterion, thereby enhancing user understanding and consistency in classification. Visual elements were added throughout all axes for better comprehension and usability [23, 24]. A summary map of the classifications was generated on the summary page and could be emailed to the user's specified email address.

The toolkit's logical and intuitive framework organizes content hierarchically, ensuring ease of navigation and accessibility for users (see Fig. 4).

Fig. 4

Navigation architecture among interfaces of the toolkit

Experts found the toolkit highly applicable to various healthcare settings, noting the benefit of generating summary reports for policymakers, though some recommended adding more customizable options for specific regional use cases. While clinicians and researchers commended the toolkit’s intuitive design, it was suggested that more comprehensive onboarding materials or tutorials be included for users with limited digital health experience. Based on these insights, refinements were made to improve navigation, add explanatory content, and enhance the user guide for broader accessibility.

Results of the pilot testing and reporting of the cardiac rehabilitation use case

A summary of the classification of five examples in the cardiac rehabilitation according to CDIASH is shown in Fig. 5.

Fig. 5

a Linkages across health System challenges, digital health interventions and Aktivplan application b Linkages across health system challenges, digital health interventions and MORE c Linkages across health system challenges, digital health interventions and shared achievements. d Linkages across health system challenges, digital health interventions and HERO e Linkages across health system challenges, digital health interventions and Active waiting

The participants of the pilot testing appreciated the clear purpose of the toolkit, which effectively guided them in classifying DHIs.

When comparing the WHO document and the tool, users generally appreciate the comprehensive and detailed nature of the WHO framework but find the step-by-step structure and ease of use provided by the tool to be more efficient and accessible for practical application. One participant noted,

“The WHO Digital Health Intervention (DHI) framework is comprehensive and highly informative, providing in-depth classifications that cover a wide range of digital health interventions. However, its format resembles a book, requiring users to frequently go up and down the pages or refer back to previous sections, which can be time-consuming and cumbersome. In contrast, my tool offers a more user-friendly experience: it allows users to proceed step-by-step, skip unnecessary steps, and directly read only the relevant examples and explanations.”

Overall, participants reported that the toolkit was intuitive and easy to navigate. The clarity of the classification criteria was frequently praised. One participant noted,

"The criteria were clearly defined, which made it simple to classify our digital health interventions."

They found the inclusion of practical examples particularly beneficial for enhancing understanding and application of the WHO classifications. One user stated,

"The inclusion of examples was particularly beneficial in helping me grasp the framework."

The picture choices were eye-catching and efficiently conveyed concepts. The capability to receive a summary at the end, along with the option to email the summary and create a PDF document, was positively reviewed.

However, participants suggested several improvements to enhance the toolkit. For instance, they recommended that the starting page clearly labeled the resource as a "toolkit" and articulate its purpose on the first page to improve user understanding. One user commented,

"I wasn’t sure what the initial page was meant for—it should be clearer."

This indicates a need for better communication regarding the toolkit's intent.

Participants also suggested ensuring consistent terminology in the top menu to enhance coherence. Another suggestion was to make the "Next" button visible without scrolling and allow direct navigation to categories. Additionally, they recommended adding guidance to indicate that a selection must be made before clicking "Next," along with an improved error message for clarity. One user noted,

"It was frustrating to have to scroll down to find the 'Next' button," indicating a need for improved navigation usability.

Improving navigation options, allowing users to easily return to previous pages, and ensuring the top menu functions properly were also identified as key areas for enhancement. A participant commented,

"On the overview page, I couldn't tell what I should select first or if it was parallel pathways without any connection."

Further recommendations included reducing excessive scrolling by utilizing wider text boxes and adjusting colors for better accessibility, particularly for users with color blindness. While the tool is convenient for step-by-step classification, one user commented:

“It may lack the flexibility of the WHO document's narrative approach, which allows for more extensive interpretation and explanation of digital health interventions. The tool might struggle with highly complex or multifaceted interventions that don't fit neatly into predefined categories. In such cases, the more detailed WHO framework may offer a better understanding of the nuances”.

During the classification of digital health interventions, several practical challenges emerged. Certain applications exhibited multi-purpose capabilities, making it difficult to assign them a single classification. For example, the MORE Platform functions as a research information system while also supporting data management services. Similarly, Aktivplan serves both as a planning tool and a behavioral health management system. In that circumstance the primary function was prioritized while acknowledging secondary functionalities. This ensured that applications were not misclassified based on a singular feature.

The application's potential to support patient adherence to rehabilitation or prescribed activity plans blurred the lines between personal use and clinical health monitoring, making its classification dependent on use-case context. To address these challenges, Aktivplan was categorized under Personal Health Tracking (1.4), and also under targeted communication to person (1.1) if used in clinical pathways. This approach ensures its classification reflects both general self-monitoring and potential clinical utility.

Applications such as Active Waiting and Shared Achievements promote physical activity and motivation but do not include direct clinical interventions. This made it challenging to differentiate them from apps used in rehabilitation or medical treatment. Consequently, we classified them under Personal Health Tracking (1.4), as they promote general activity rather than structured medical exercise.

A key challenge in classifying the HERO application was distinguishing between targeted health communication and patient self-management, as it provides both structured education and interactive guidance. Another challenge was determining whether it functions as a passive information tool or an active decision-support system, as it helps users navigate rehabilitation but does not fully integrate with electronic medical records. To address these issues, the application was classified primarily under Targeted Communication to Persons (1.1.2) and On-Demand Communication (1.6.1) for its informational role, while also recognizing its Personal Health Tracking (1.4.2).

Based on pilot testing feedback, several key enhancements were made to the toolkit's usability and user experience. Clear guidance was added for required selections, along with improved error messages. The overview page was reorganized for logical navigation, while layout adjustments reduced excessive scrolling. Accessibility was enhanced with better color contrast for users with vision deficiencies. The top menu was refined for better functionality, allowing users to return to previous pages easily. Finally, the summary feature was improved for clearer formatting and easier sharing via email and PDF export.