Introduction

Chronic non-cancer pain (CNCP) significantly reduces the quality of life of the individuals affected by it. As 28–88% of older adults may suffer from CNCP, this is a highly relevant public health concern.1 CNCP is associated with multimorbidity in 88% of cases,2,3 with depression and insomnia being frequent co-morbidities.4–6 Older adults suffering from CNCP, in particular, have more physical and cognitive deficits.7 CNCP affects more women than men and more people from poorer socioeconomic backgrounds, particularly those with past unemployment.8,9 Alcohol consumption and smoking also seem to correlate with CNCP.10,11

Because of its complexity, CNCP management should consider different treatment types. First-line therapies should be non-pharmacological and can include both psychological and somatic interventions. Pharmacological therapies should be considered second-line therapies.12 Despite this recommendation, however, many older adult inpatients continue to be prescribed drugs for their CNCP as a first-line treatment, putting them at greater risk of medication-related problems (MRPs).7

MRPs are particularly prevalent among older adult inpatients, largely due to their complex clinical presentations involving multimorbidity and polypharmacy.13 As older adult patients often experience clinical decompensation during hospitalisation, the effects of MRPs are more detrimental.14 The situation becomes even more complex in the context of CNCP, because older adult patients may experience acute-on-chronic pain, which may require additional pharmacological treatment.15 Older adults continue to be at a very high risk of MRPs when they are discharged from hospital due to suboptimal information transfer and communication issues.16,17 These intersecting challenges highlight the importance of improving medication safety practices for older adult patients with CNCP.

Quality indicators (QIs) are measurable items used to assess and track care, both within and between institutions.18 As such, QIs can help to standardise healthcare processes and improve the quality of care delivered.18 If incorporated within (electronic) trigger tools, QIs can help to efficiently identify patients at a higher risk of deficient care.19 QIs are thus essential to continuous improvements in healthcare system quality and must be developed with great rigour.18,20

Although there are different methods for developing QIs, most involve systematic searches of the relevant literature and/or the expert validation of the QIs selected.18 Our previous systematic literature search revealed no existing set of QIs to guide the pharmacological management of CNCP in older adults, but we did accumulate a list of individual QIs and developed new ones from the quality criteria found.21 However, QIs must also reflect expert opinions,18 especially in a domain where high-quality evidence from randomised controlled trials is scarce.21 The Delphi method is a suitable study design for achieving expert consensus on QIs. It is a qualitative, constructivist, consensus-building method that allows experts to negotiate a shared clinical reality and co-construct recommendations and rules.22 However, regional differences in drug availability or guidelines may cause experts to rate QIs differently, limiting their generalisability. The present study aimed to find a consensus on the face validity and feasibility of a proposed set of QIs for the pharmacological management of CNCP in older adults inpatients.

Materials and Methods Design

Campbell et al proposed that developing QIs first requires collating evidence using a systematic literature search and then establishing expert consensus on the QIs identified.18 To this end, they explicitly recommended using the Research and Development Corporation’s (RAND) and the University of California at Los Angeles’s (UCLA) Appropriateness Method, known as the RAM.18,23 Prior to our RAM Delphi study, we used an integrative literature review to systematically collect an initial set of potential QIs for the pharmacological management of CNCP in older adult inpatients.21

To find a consensus on the face validity and feasibility of this proposed set of QIs, we conducted a two-round Delphi study with three focus group discussions between the two rounds, as the RAM proposes. In the first round, experts rated the face validity and feasibility of the proposed set of QIs. Those for which no consensus could be found or for which we received many suggestions for improvement were discussed in three focus groups. Each expert was asked to participate in one focus group. Subsequently, QIs without a consensus decision and those that had been adapted were rated again. A visual representation of the RAM Delphi study methods used is shown in Figure 1. For the study overall, we followed the Recommendations for the Conducting and Reporting of Delphi Studies (CREDES).22 Our local ethics committee (Cantonal Ethics Committee Bern) declared that the study did not require its approval (Req-2024-00779) because the project does not fall under the Human Research Act, Art. 2, para. 1.

Figure 1 Schematic overview of the methods used in this RAND/UCLA Appropriateness Method Delphi study. QI = Quality indicator. Created in BioRender. Goetschi, A (2025) https://BioRender.com/i93a003.

Expert Selection

The RAM recommends creating an expert panel of 7–15, but acknowledges that the optimal composition depends on the desired geographic and professional representation.23 As we aimed for a diverse and international panel including nurses, pharmacists and physicians, we set our target panel size at 20–30 experts. Experts were required to hold a university degree in nursing, pharmacy or medicine and have proven expertise (clinical or academic) in the pharmacological care of older adult inpatients with CNCP. Experts were mainly identified from the authors’ professional networks and from relevant published articles found in our systematic literature search.21 If experts recommended suitable candidates, we contacted them equally. We contacted potential experts via Email and re-contacted them two weeks later if they had failed to respond.

Selection of QIs to Be Rated

We conducted a systematic literature search using an integrative review to screen 6,842 articles. This identified 11 validated QIs, 10 of which were included in the Delphi study.21 The QI excluded covered primary care practices and fell outside of our focus on inpatient care. Of the 243 other existing QIs identified, we included every QI mentioned in at least four independent studies or deemed of specific relevance by the research team, leading to a further 51 QIs being included in the Delphi study. Thus, our experts judged a total of 64 items in the first round (61 QIs plus three questions/clarifications).

Written Delphi Rounds

Our RAM study used two written Delphi rounds to establish an expert consensus. In both rounds, the experts rated the face validity and feasibility of each item on a Likert scale ranging from 1–9 (1 = extremely invalid/unfeasible; 5 = uncertain; 9 = extremely valid/feasible). Experts were able to comment on each QI if they thought it necessary and could suggest new QIs. ANG set up both questionnaires in Excel® (Microsoft® Excel® 2016 (16.0.5448.1000) MSO (16.0.5452.1000) 32 Bit) sheets (see Supplementary File 1) and they were pilot tested by three independent researchers: NS, UW and CMM. Both rounds also contained a disclaimer (see Supplementary File 1) stating that by participating in the study, experts agreed to further anonymous use of their data.

The experts rated 64 items in the first round, which took place from 10 to 31 June 2024. To facilitate experts’ decisions, we provided them with a synopsis of the results from the integrative review.21 We followed the RAM recommendations to define their consensus view:23

Include: a panel median score of ≥ 6.5, without disagreement in either face validity or feasibility. Exclude: a panel median score of < 3.5 in either face validity or feasibility, without disagreement. Uncertain: a panel median of ≥ 3.5 and < 6.5 in either face validity or feasibility, or disagreement. The “exclude” category had priority: if one category was “uncertain” and the other was “exclude”, then the QI was excluded.

We calculated disagreement using the Interpercentile Range Adjusted for Symmetry required for disagreement (IPRAS) proposed by the RAM.23

For the second Delphi round, which took place after the focus groups (described in the next paragraph), we applied the same procedures. Experts received an Excel spreadsheet showing the median first-round ratings for each QI, plus their own ratings. In addition, we asked the authors to list the most important QI in each pharmacological category (ie general drug therapy, opioids, NSAIDs, paracetamol, metamizole and co-analgesics). The second round took place from 14 to 28 August 2024 and involved rating the uncertain QIs again and rating the newly proposed QIs. We had decided a priori to limit the workload and time burdens put on the experts to two rounds of rating using the two questionnaires and participating in a 1.5-hour focus group. This approach overcame the possible limitations of a fixed two-round procedure, and we also excluded from the second round all the QIs that had not met the criteria for the “include” category.

Focus Group Discussions

Semi-structured focus group discussions occurred between the two written Delphi rounds. These enabled our experts to exchange thoughts and supported the co-construction of a shared clinical reality, recommendations and rules.22,23 We held three 1.5-hour focus group discussions, on 5, 6 and 9 August 2024, to ensure the participation of as many experts as possible. Experts were strongly encouraged to attend one meeting. Experts failing to attend a meeting were not excluded from the second Delphi round, however, and they received a synopsis of the issues discussed. Indeed, all the experts received this synopsis to ensure that they were aware of the points discussed by all three focus groups.

We tried to balance the composition of these discussion meetings using profession as the main criterion. Meetings followed the RAM recommendations to ensure that each expert had an opportunity to speak up and give their perspective.23 The focus group discussions were semi-structured, meaning that they followed the same pattern and all covered the same relevant topics. Discussions nevertheless remained open to any expert input, and schedules were adjusted if we felt the participants needed to discuss an issue in greater depth. According to the RAM recommendations we ensured that all experts could voice their opinions.23 In the event of disagreement, we attempted to facilitate a productive debate. If no consensus could be reached, we raised the issue in the next focus groups. If there was still no consensus, we let the experts decide in the second written round. All the discussions were led by ANG, who is a 27-year-old male pharmacist and PhD candidate in clinical pharmacy. He works clinically on geriatric and internal medicine wards and has experience in facilitating focus group discussions. Either NS or CMM were present at every meeting and took notes verbatim. Meetings were held using Zoom (©2024 Zoom Video Communications, Inc). Prior to starting the meetings, all the experts gave their consent to participate and to the recording of the event. We used the recordings to verify and complete the notes and then deleted them.

Results First Round

We initially contacted 52 experts, and 22 agreed to participate. In the first Delphi round, 19 (86%) experts returned their ratings (see Table 1 for the experts’ characteristics). Of the 61 QIs rated, 30 had panel median scores ≥ 6.5 for both face validity and feasibility. Ten QIs that received suggestions for improvement, were retained and discussed in the ensuing focus groups. Twenty QIs had uncertain ratings, and one was excluded (see Table 2 and Figure 2).

Table 1 Summary of the Characteristics of the Experts Who Agreed to Participate in Our Delphi Study. As Some Experts Were Working in Multiple Settings, the Percentages and Absolute Numbers May Add up to More Than 22

Table 2 Overview of the Median Expert Ratings for Face Validity and Feasibility, as Well as Disagreements Concerning Quality Indicators (QIs) Over the Two Delphi Rounds

Table 3 The Final Set of Quality Indicators (QIs) for the Pharmacological Management of Chronic Non-Cancer Pain in Older Adult Inpatients

Figure 2 Sankey plot depicting how the experts evaluated, modified, rated uncertain, included and excluded the 61 original QIs identified in a previous integrative review, plus an extra 13 new QIs, over the two written rounds and the focus groups. Created in BioRender. Goetschi, A (2025) https://BioRender.com/m32m131.

Focus Group

Eighteen of our 22 experts (81%) agreed to participate in one of our three focus group discussions; however, mostly due to clinical emergencies, only 9 (50%) of them actually did so. The experts discussed the “uncertain” QIs, the QIs with suggested improvements and the new QIs proposed by the experts in round one. As a result, 11 QIs were modified before being rated again, and 13 new QIs were proposed for rating in round two. The experts deemed one QI to be inappropriate, and we excluded it. Table 2 provides an overview of the changes made.

Regarding general drug therapy, the experts discussed when patients should have their medication re-evaluated and the value of numerical rating scales. QI 2 advised that a patient’s medication should be re-evaluated after every 3–6 months of therapy. In their discussions, the experts agreed that this period was too long. The consensus was that treatment re-evaluations would ideally occur as soon as 1 week after treatment initiation or change. As this would decrease the QI’s feasibility, the experts agreed to the wording “within 1 month”. In their opinion, this QI should emphasise that if an earlier evaluation were possible, then it should be carried out. The experts discussed the value of numerical rating scales at length (eg 0–10, where 0 is no pain and 10 is the worst pain imaginable). Overall, their consensus was that numerical rating scales were suboptimal for evaluating the adequacy of an ongoing treatment. This led to the development of QI N1, specifying that treatment goals should align with quality of life and functionality. The majority of experts nevertheless believed that numerical rating scales might be more suitable to evaluate treatment efficacy in acute on chronic pain (eg patients with a fracture and chronic low back pain), aligning with standard evaluations of acute pain management. This resulted in a modified version of QI 3. However, some experts voiced the opinion that acute CNCP should not lead to an immediate change in pain medication but rather to the initiation of a careful clinical evaluation of the patient. Experts also agreed that many older adults find numerical rating scales problematic, particularly those with a cognitive impairment.

When discussing the QIs on opioids, the experts focussed on therapeutic monitoring and the risks of addiction. Regarding opioid initiation, the experts stressed the importance of distinguishing safety and efficacy. Safety should be evaluated earlier—within one week, according to QI 5.2. Experts also underlined the importance of nurses monitoring the patient. Efficacy, on the other hand, should be measured with improvements in quality of life. The experts emphasised that if no improvements occurred, opioid therapy should be stopped. They also acknowledged the importance of screening for and monitoring addiction to opioids. However, they doubted the feasibility of implementing this widely, especially in a European context. They argued that clinicians had limited time and lacked the necessary knowledge about screening tools and how to interpret results.

The experts noted that a scarcity of evidence made discussing QIs on metamizole challenging; only a few countries use this active substance. There was discussion about whether metamizole should be a first-line drug, comparable to paracetamol, but due to the lack of evidence, especially on efficacy and safety, they agreed to recommend using metamizole solely in cases of non-inflammatory CNCP where paracetamol is insufficiently effective. In doing so, the experts acknowledged the increased risk of agranulocytosis when using metamizole, and they proposed two QIs as risk-reduction strategies.

The experts also discussed pharmacogenetic testing’s value in CNCP management. While most agreed that it might be relevant in some cases, they had doubts about the cost–benefit analysis and healthcare professionals’ ability to interpret the testing’s results. Some participants also voiced concerns that the pharmacogenetic testing of polymedicated older adults with CNCP might be inferior to phenotyping as drug–drug, drug–gene and drug–disease interactions become too complex to predict.

Second Round

Nineteen (86%) experts participated in the second round, rating the “uncertain”, modified and new QIs and prioritising them all according to their relevance. These ratings led to a final set of 51 QIs meeting our pre-specified inclusion criteria. QI 18 (prefer topical drugs) received a sufficiently high median rating to be included; however, multiple experts stated that it was less valid than QI 17 (prefer topical drugs for localised pain). Due to their similarity, we followed their recommendation and excluded it. All the median QI ratings are shown in Table 2. Twenty-three QIs received at least one priority rating. The final set of QIs, ordered by priority, face validity and then feasibility, is shown in Table 3. It consists of 10 QIs for general drug therapy, 18 for opioids, 11 for NSAIDs, 3 for paracetamol, 3 for metamizole and 5 for co-analgesics.

Discussion

This RAM Delphi study aimed to reach a consensus on the face validity and feasibility of a set of potential QIs for the pharmacological management of CNCP in older adult inpatients. After two written rounds rating the QIs and three focus group discussions between them, our experts included a total of 51 QIs, of which 23 were given a priority rating. To the best of our knowledge, this is the first set of expert-validated QIs prepared for the pharmacological management of CNCP in older adult inpatients.

We chose a RAM Delphi study approach based on a systematic literature search because this is a recommended process for developing a set of QIs.18 The Delphi method allows experts to construct a shared clinical reality based on an aggregated scientific evidence base.22 It is particularly useful because its anonymous rating rounds ensure that every expert’s opinions are heard, reducing the risk of a few dominant voices overpowering others. Having focus group discussions between the anonymous written rounds facilitates the exchange of opinions needed to reach a consensus.24 Predefined inclusion and exclusion criteria for QIs, set as per the RAM recommendations, provided the basis for a rigorous evaluation of the QIs. Finally, combining the results of a systematic literature search with input from experts working clinically with older adults enabled the development of a set of QIs that were both evidence-based and relevant to practice. Delphi study methodologies are not above critique, however, with some authors arguing that they are neither reliable nor valid enough.25,26 The RAM acknowledges this, stating that Delphi studies are more reliable and valid when there is a sound evidence base and when the items rated are objective.23 Keeney et al27 also indicated that judging the constructivist Delphi method using positivist criteria (reliability and validity) may be inappropriate. Instead they suggested quality criteria such as transferability, credibility, applicability and confirmability.27

The set of QIs developed in this study could be used in different ways. They could help to standardise the pharmacological care delivered to older adult inpatients with CNCP18 or could be used as trigger tools integrated into an electronic algorithm to flag patients at an elevated risk of MRPs or to suggest targeted medication reviews.19 To this end, further research validating our set of QIs in clinical practice seems warranted. Feedback on the set’s clinical applicability and completeness would have to be collected systematically: the set of QIs may be too big to be useful in clinical practice or may lack depth to support decision-making. Indeed, the set of QIs prepared involved no patient input, and some QIs may not adequately reflect aspects of the quality of care that are most relevant to them. Extrapolation to other situations may also be needed, as some of the QIs could also be applied to outpatient care. Another point to consider is new developments. As technologies evolve, such as pharmacogenetic testing, they may become more affordable and available, necessitating their integration into this set of QIs. It is also worth noting that important insights can be gained through a thorough examination of the patient’s medical and family history. These can serve as accessible alternatives or complementary approaches.

Although most of the QIs excluded from our final set were considered valid—sometimes highly valid—our experts doubted that current healthcare systems would be able to implement them, making them infeasible. One specific example of this was QI 12, which recommends multimodal treatments. This concept is reflected in most clinical guidelines.12,28,29 Even though our experts rated this QI with a median of 9 and gave it two priority ratings, its feasibility was deemed too low. Reasons for this included too few specialised healthcare professionals and health insurance reimbursement issues for the institutions concerned.

Some QIs included in this set may prove controversial. For instance, QI 1 recommends prescribing pharmacological treatments to all older adult inpatients with CNCP. Many guidelines recommend that pharmacological treatments should only be second-line treatments.12,28,29 On the contrary, however, our experts stated that pharmacological therapy should at least be available as a first-line safety net, and they noted that most older adult inpatients with CNCP were currently treated pharmacologically anyway, despite the guidelines.

Other QIs involving some uncertainties included those for metamizole. Metamizole is only licensed in a few countries, but it is used a lot in Switzerland, Austria and Germany, particularly for older adults.30,31 Although there is some evidence that metamizole may indeed be more tolerable than NSAIDS and just as efficient,32 many healthcare professionals still fear agranulocytosis, a very rare but potentially lethal adverse drug reaction.33 As metamizole is only used in a handful of countries, evidence on its long-term use remains limited.34 Evaluations of recommendations for its use, therefore, often rely on personal clinical experience. With new evidence on metamizole emerging, for example, regarding drug–drug interactions and efficacy, the QIs that we have proposed here may need updating.35,36

Limitations

Although this Delphi study was performed following a systematic literature search and conducted using the RAM’s rigorous methodology, it had some limitations. First, the quality of the consensus reached in any Delphi study relies on the quality of the available evidence and the panel members’ clinical expertise. Because CNCP in older adult inpatients has been poorly studied, the evidence base may be too weak to formulate robust QIs. However, as the rate of retention of the QIs from the literature was high, we believe that the selection presented to the experts was appropriate. Second, participation in the focus group discussions was low. Although we tried to make participating in a focus group as easy as possible, many experts failed to attend. We nevertheless fully informed every expert about the discussions, whether they had attended one or not, and allowed them to give full written input on our suggested QIs in both rounds. We believe, therefore, that even if some experts did not participate in the focus group discussions, they had sufficient possibilities to intervene. Third, the QIs present here are not generalisable, particularly regarding their feasibility. Our experts were predominantly from German-speaking countries, and their recommendations may reflect the clinical practices in those countries. The feasibility of QIs, in particular, may also vary between institutions.

Conclusion

This two-round Research and Development Corporation (RAND) and University of California at Los Angeles (UCLA) Appropriateness Method Delphi study involved three focus groups and 19 experts rating potential quality indicators (QIs) for the pharmacological management of chronic non-cancer pain (CNCP) in older adult inpatients. Our experts included 51 QIs in the final set. This is the first expert-validated set of QIs developed out of a systematic search of the relevant literature, and it could help to support the standardisation of the care provided to older adult inpatients with CNCP, increasing the overall quality of the care delivered to them. These QIs could also be used as trigger tools to prioritise patients for interventions. To this end and to generalise the QIs to other settings, our QIs may still need to be adapted to local clinical practices and imperatives, and they require further clinical validation using real-world data.

Abbreviations

CNCP, chronic non-cancer pain; CREDES, Conducting and Reporting of Delphi Studies; IPRAS, Interpercentile Range Adjusted for Symmetry required for disagreement; NSAID, non-steroidal anti-inflammatory drugs; QI, quality indicator; RAM, RAND/UCLA Appropriateness Method; RAND, Research and Development Corporation; UCLA, University of California, Los Angeles.

Acknowledgments

The authors would like to acknowledge the experts’ expertise and efforts, without which this study would not have been possible.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Disclosure

The salaries of Aljoscha Noël Goetschi and Carla Meyer-Massetti are covered by the endowed professorship from PharmaSuisse - the Swiss Pharmacists Association. The authors have no other conflicts of interest to report with regard to this work.

References

1. Helme RD, Gibson SJ. The epidemiology of pain in elderly people. Clin Geriatric Med. 2001;17(3):417–431. doi:10.1016/S0749-0690(05)70078-1

2. Mills SEE, Nicolson KP, Smith BH. Chronic pain: a review of its epidemiology and associated factors in population-based studies. Br J Anaesth. 2019;123(2):e273–e83. doi:10.1016/j.bja.2019.03.023

3. Scherer M, Hansen H, Gensichen J, et al. Association between multimorbidity patterns and chronic pain in elderly primary care patients: a cross-sectional observational study. BMC Family Prac. 2016;17(1):68. doi:10.1186/s12875-016-0468-1

4. Jank R, Gallee A, Boeckle M, et al. Chronic pain and sleep disorders in primary care. Pain Res Treatment. 2017;2017:1–9. doi:10.1155/2017/9081802

5. Banks SM, Kerns RD. Explaining high rates of depression in chronic pain: a diathesis-stress framework. Psychol Bull. 1996;119(1):95. doi:10.1037/0033-2909.119.1.95

6. Currie SR, Wang J. More data on major depression as an antecedent risk factor for first onset of chronic back pain. Psycholog Med. 2005;35(9):1275–1282. doi:10.1017/S0033291705004952

7. Goetschi AN, Verloo H, Wernli B, et al. Prescribing pattern insights from a longitudinal study of older adult inpatients with polypharmacy and chronic non-cancer pain. Eur J Pain. 2024;28(10):1645–1655. doi:10.1002/ejp.2298

8. El-Shormilisy N, Strong J, Meredith PJ. Associations among gender, coping patterns and functioning for individuals with chronic pain: a systematic review. Pain Res Manag. 2015;20(1):48–55. doi:10.1155/2015/490610

9. Janevic MR, McLaughlin SJ, Heapy AA, et al. Racial and socioeconomic disparities in disabling chronic pain: findings from the health and retirement study. J Pain. 2017;18(12):1459–1467. doi:10.1016/j.jpain.2017.07.005

10. Orhurhu VJ, Pittelkow TP, Hooten WM. Prevalence of smoking in adults with chronic pain. Tob Induc Dis. 2015;13(1):1–6. doi:10.1186/s12971-015-0042-y

11. Alford DP, German JS, Samet JH, et al. Primary care patients with drug use report chronic pain and self-medicate with alcohol and other drugs. J Gen Intern Med. 2016;31(5):486–491. doi:10.1007/s11606-016-3586-5

12. Gobet C, Naef F; NICE Guideline NG193. Chronic pain (primary and secondary) in over 16s: assessment of all chronic pain and management of chronic primary pain. Methods. 2021;203:10. doi:10.1016/j.ymeth.2021.10.004

13. Pramotesiri P, Putthipokin K, Ruangritchankul S. Drug related problems among older inpatients at a tertiary care setting. J Clin Med. 2024;13(6):1638. doi:10.3390/jcm13061638

14. Maher RL, Hanlon J, Hajjar ER. Clinical consequences of polypharmacy in elderly. Expert Opin Drug Saf. 2014;13(1):57–65. doi:10.1517/14740338.2013.827660

15. Laufenberg-Feldmann R, Jage J. Acute pain management in the elderly - Pharmaceutics and psychosocial approaches. Anasthesiol Intensivmed Notfallmed Schmerzther. 2011;46(5):354–362. doi:10.1055/s-0031-1277979

16. Forster AJ, Murff HJ, Peterson JF, et al. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Internal Med. 2003;138(3):161–167. doi:10.7326/0003-4819-138-3-200302040-00007/m12558354

17. Ambade PN, Hoffman Z, Vest T, et al. Factors influencing communication issues during hospital discharge for older adults in 11 high-income countries: a secondary analysis of the 2021 international health policy survey. BMJ Open. 2025;15(1):e089430. doi:10.1136/bmjopen-2024-089430

18. Campbell SM, Braspenning J, Hutchinson A, et al. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11(4):358–364. doi:10.1136/qhc.11.4.358

19. Murphy DR, Meyer AN, Sittig DF, et al. Application of electronic trigger tools to identify targets for improving diagnostic safety. BMJ Qual Saf. 2019;28(2):151–159. doi:10.1136/bmjqs-2018-008086

20. Lloyd RC. Quality Health Care: A Guide to Developing and Using Indicators. Jones & Bartlett Learning; 2017.

21. Goetschi AN, Verloo H, Schönenberger N, et al. Quality indicators for the pharmacological management of chronic non-cancer pain in older adult patients: an integrative review. J Evalu Clin Prac. 2025;31(5):e7025.

22. Jünger S, Payne SA, Brine J, et al. Guidance on conducting and reporting delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. doi:10.1177/0269216317690685

23. Fitch K, Bernstein SJ, Aguilar MD, et al. The RAND/UCLA Appropriateness Method User’s Manual. Santa Monica CA: Rand Corporation; 2001.

24. Day J, Bobeva M. A generic toolkit for the successful management of Delphi studies. Electron J Business Res Methods. 2005;3(2):10316.

25. Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs. 1994;19(1):180–186. doi:10.1111/j.1365-2648.1994.tb01066.x

26. Goodman CM. The Delphi technique: a critique. J Adv Nurs. 1987;12(6):729–734. doi:10.1111/j.1365-2648.1987.tb01376.x

27. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. InterJ Nurs Stud. 2001;38(2):195–200. doi:10.1016/S0020-7489(00)00044-4

28. Dowell D. CDC clinical practice guideline for prescribing opioids for pain—United States, 2022. MMWR Recommendations Rep. 2022;71.

29. Qaseem A, Wilt TJ, McLean RM, Forciea MA, Noninvasive Treatments for Acute. Subacute, and chronic low back pain: a clinical practice guideline from the American college of physicians. Ann Internal Med. 2017;166(7):514–530. doi:10.7326/m16-2367/m28192789

30. Gut S, Rauch M, Haschke M, et al. Use of metamizole and other non-opioid analgesics in Switzerland between 2014 and 2019: an observational study using a large health insurance claims database. Swiss Med Weekly. 2024;154(2):3535. doi:10.57187/s.3535

31. Hoffmann F, Meinecke P, Freitag MH, et al. Who gets dipyrone (metamizole) in Germany? Prescribing by age, sex and region. J Clin Pharm Therapeutics. 2015;40(3):285–288. doi:10.1111/jcpt.12261

32. Haschke M, Liechti ME. Metamizol: nutzen und Risiken im Vergleich zu Paracetamol und NSAR. Schweiz Med Forum. 2017;17(48):1067–1073.

33. Huber M, Andersohn F, Sarganas G, et al. Metamizole-induced agranulocytosis revisited: results from the prospective Berlin case–control surveillance study. Eur J Clin Pharmacol. 2015;71(2):219–227. doi:10.1007/s00228-014-1777-8

34. Kötter T, da Costa BR, Fässler M, et al. Metamizole-associated adverse events: a systematic review and meta-analysis. PLoS One. 2015;10(4):e0122918. doi:10.1371/journal.pone.0122918

35. Bachmann F, Duthaler U, Meyer Zu Schwabedissen HE, et al. Metamizole is a moderate cytochrome P450 inducer via the constitutive androstane receptor and a weak inhibitor of CYP1A2. Clin Pharmacol Ther. 2021;109(6):1505–1516. doi:10.1002/cpt.2141

36. Watermeyer F, Gaebler AJ, Neuner I, et al. Discovering interactions in polypharmacy: impact of metamizole on the metabolism of quetiapine. Br J Clin Pharmacol. 2024;90(11):2793–2801. doi:10.1111/bcp.16168