The initial search identified 531 different records, excluding duplicates. During the screening phase, 488 records were deemed irrelevant, leaving 43 for full-text review. The inter-rater agreement between independent screeners was considered sufficient (PABAK = 0.816). Following the full-text assessment, ten articles corresponding to nine studies met the inclusion criteria and were considered for analysis. Additionally, three other articles were retrieved from citation analyses (i.e., backward and forward searches), and one was obtained from the grey literature search. Ultimately, 14 articles, corresponding to 13 studies, were included in this review (Fig. 1) [30,31,32,33,34,35,36,37,38,39,40,41,42,43].

Flow diagram of the study selection process

Table 1 provides an overview of the studies included in this review. The articles, published between 2010 and 2023, are distributed across all continents: six in Europe, two in America, two in Asia, two in Oceania, and one in Africa. Notably, three studies focused on specific CPS, as declared by the authors: Doerper et al. and Aljamal et al. addressed "medication reconciliation," while King et al. concentrated on "transitions of care."

Most studies involved a comprehensive KPI development process, which included an appraisal of evidence from the literature, followed by an expert panel consensus. Delphi (regular or modified) was the most prevalent method for KPI development, with eight studies utilizing this approach. One study combined nominal and focus group techniques, while two studies adapted consensus indicators previously published by the American College of Clinical Pharmacy (AACP) and the American Society of Health-System Pharmacy (ASHP). Additionally, one study employed an online survey with pharmacists, and another study did not provide information regarding the KPI development approach.

Pharmacists were the predominant stakeholders in the Key Performance Indicator (KPI) development teams, represented in 12 (92%) studies. Additionally, four studies included a minority of other professionals such as physicians, nurses, and risk/quality assurance managers. One study did not provide information about the composition of stakeholders. Notably, none of the studies reported the participation of patients, payers, or policymakers.

Several studies emphasized the significance of standardizing and benchmarking KPIs. The reasons supporting this recommendation included: (1) enabling meaningful comparisons of CPS within and between organizations; (2) facilitating performance management of hospital pharmacists; and (3) enhancing transparency about the quality of hospital pharmacy services. Despite the acknowledgement of benchmark importance, none of the studies provided any.

Table 2 presents the scores for the methodological quality of the included studies using the AIRE instrument. Notably, there is a wide variation in the information and level of detail describing the methodological characteristics of the studies developing KPIs. In domain #1 (Purpose, relevance, and organizational context), 11 studies (85%) scored above the 50% threshold, with an overall mean score of 64%. The item "the quality domain the indicator addresses is described in detail," representing the care quality aspect of the indicator (e.g., patient safety, care effectiveness, timeliness, equality, patient-centeredness, etc.), received the lowest score among the studies. In domain #2 (Stakeholder involvement), 10 studies (77%) demonstrated good quality, with an overall mean score of 54%. Two items in this domain scored particularly low: stakeholders' involvement and formal endorsement of the indicator set. Domains #3 (Scientific evidence) and #4 (Additional evidence, formulation, and usage) received the lowest scores, with overall means of 39% and 35%, respectively. Only five studies (38%) and three studies (23%) exceeded the 50% threshold in these domains. In domain #3, the item "supporting evidence has been critically appraised" scored the lowest, while in domain #4, the lowest scores were related to risk adjustment, accuracy, and consistency of the measures, as well as the piloting of the indicators.

Overall, two studies achieved high methodological quality in all four AIRE domains, while one study scored below the 50% threshold across all four AIRE domains. In the overall standardized score, encompassing all items from the four domains, only five studies scored higher than the 50% threshold, whereas three studies scored around 35%.

In total, 225 KPIs were extracted from the 13 studies. A complete list of these 225 KPI is available at Supplementary File 2. The reported number of KPIs in each study ranged from 6 to 41 (median = 16). Following Donabedian’s SPO paradigm [44], 19 KPIs (8.4%) were classified as structure, 193 (85.8%) as process, and 13 (5.8%) as outcome indicators. Among the outcome KPIs, 4 (1.8%) were economic, 7 (3.1%) were clinical, and 2 (0.9%) pertained to the humanistic dimensions of the ECHO model. Only one study developed KPIs spanning all three SPO categories. Merely three studies, contributing to 49 KPIs, described them using Donabedian’s SPO framework. Among these 49 KPIs, authors' classifications aligned with ours in 41 instances, with the remaining 8 originally classified as outcome KPIs by the authors but as process KPIs by our team. Notably, none of the studies utilized the ECHO model to categorize the KPIs. A comprehensive description of the KPIs is provided in Supplementary File 3.

Structure KPIs concentrated on the organizational aspects of the service and included supportive activities (9 KPIs in 3 studies), availability of protocol/policy (5 KPIs in 2 studies), technical and human resources (4 KPIs in 4 studies), and professional qualification (1 KPI in 1 study).

Process KPIs could be categorized into ten services: medication reconciliation (75 KPIs in 12 studies), medication review (46 KPIs in 10 studies), pharmacist advice (25 KPIs in 8 studies), care team involvement (16 KPIs in 5 studies), patient education (14 KPIs in 7 studies), care plan implementation (7 KPIs in 4 studies), dispensing (4 KPIs in 4 studies), outpatient services (2 KPIs in 2 studies), student mentoring (2 KPIs in 2 studies), and care team members’ satisfaction (2 KPIs in 1 study).

Among the 7 clinical outcomes KPIs, the focus was on adverse drug reactions (3 KPIs in 2 studies), patient readmission (3 KPIs in 1 study), and length of stay (1 KPI in 1 study). The 4 economic outcomes KPIs included cost savings (3 KPIs in 2 studies) and cost of therapy (1 KPI in 1 study). Additionally, the 2 humanistic outcomes KPIs aimed at patient satisfaction (2 KPIs in 2 studies).

A higher prevalence of outcome KPIs in a study was not associated with the overall methodological quality assessed by the AIRE. Among the five studies scoring above the 50% threshold, only two developed any outcome KPI. Notably, the study that produced KPIs from all three SPO categories scored below the 50% threshold.