A total of 29 guidance documents were identified, including 5 on general modeling [10,11,12,13,14], 3 on cost-effectiveness modelling approaches [5, 15, 16], and 1 on observational data in general [17]. Regarding evidence synthesis, 13 considered indirect treatment comparisons (ITC) [18,19,20,21,22,23,24,25,26,27,28,29,30], 3 survival analysis [1, 8, 31], 3 treatment-switching methodologies [30, 32, 33], and 2 surrogacy [34, 35].

The review showed that besides the plausibility of survival extrapolations, plausibility of underlying model choices is also important for HTA agencies when assessing these models. The review further showed that the terms biologically plausible and clinically plausible are used interchangeably. In our view, biological plausibility is broadly defined by disease processes and treatment mechanisms of action, whereas clinical aspects are mostly defined by human interaction with the biological process. In general, biological and clinical aspects will jointly influence survival, therefore, in the remainder of the manuscript, the term plausibility will refer to both aspects.

In the sections below, we will define plausibility of survival extrapolations and propose a process for the operationalization of its assessment. Next, we will propose a survival extrapolation protocol template embedding the definition and the process for assessing plausibility of modeled survival extrapolations. Please note that the steps described in the next two paragraphs should be conducted partly in parallel (e.g., the last step of the assessment process should be conducted after the protocol is finalized).

3.1 Defining and Operationalizing Biological/Clinical Plausibility of Survival Extrapolations

We define clinically and biologically plausible survival extrapolations [1, 5, 8, 15], as “predicted survival estimates that fall within the range considered plausible a-priori, obtained using a-priori justified methodology.”

Expectations should align with the totality of evidence on biological and clinical aspects that may influence survival and with local HTA guidance, such as guidance on plausibility on trend in hazards, model approaches, and ITCs. They should be generated for the target setting, defined by, for instance, patient population, treatment pathway, and country. The target setting may reflect current medical practice or the trial setting, depending on the nature of the decision, available evidence, and the HTA agency [14, 17]. Guidance documents usually suggest that the target setting should reflect clinical practice [10, 11, 14].

For operationalization and quantitative assessment of biological/clinical plausibility, we propose a five-step process called DICSA building on the model development steps by Chilcott et al. [36].

3.1.1 DICSA Step 1

The first step of DICSA is to Define, describe, and understand the target setting of interest in terms of survival and the survival treatment effect and in terms of aspects influencing survival and the survival treatment effect. Here, the aspects may be related to disease processes, treatment pathway, or patient characteristics.

3.1.2 DICSA Step 2

The second step of DICSA relates to Information collection to support setting survival expectations. TSD 13 highlights the importance of reporting the data sources and the searches, so that users of the model can understand how sources of evidence came to be incorporated into the process and can judge whether the model is based on a plausible or acceptable set of evidence [15]. The following sources may provide valuable information:

Clinical guidelines and clinical input (qualitative and quantitative)

Existing health economic models

Existing (systematic) reviews

Historical (phase 1–4) trials

Real-world evidence (RWE) studies

Network meta-analysis and surrogacy evaluations

Routine monitoring sources

Data derived from the sources above do not necessarily need to relate to comparator- or active-treatment-specific information; they can also concern the general disease process or information from the same drug class in the same indication, later lines, and or other indications or may just describe the population in the target setting.

Data may be in the format of previously observed or expected survival estimates and the uncertainty around these may be more qualitative in nature (for example, whether hazards are expected to increase or decrease over a particular time period given the nature of the disease).

3.1.3 DICSA Step 3

The third step of DICSA involves comparing patient-, treatment-, and disease-related aspects between relevant information sources and the target setting. The relevance of information sources is related to maturity of survival data, comprehensiveness of reported aspects, and applicability of these aspects to the target setting (sources that closely align with the target setting are preferred). Preference may be given to data sources that establish a clear link between the reported aspects and mature survival, such as those featuring published risk equations or individual patient-level data with mature survival data.

If there are no published survival data over the disease course for standard of care (SoC), one may need to fall back to epidemiological publications reporting published standard mortality ratios (SMR). Please note, when relying on SMR-like metrics, one likely needs to assume the disease course has the same shape as the general population mortality. If these types of metrics are not available (e.g., for orphan diseases), the authors recommend the sponsor to start retrospective survival data collection, via databases or chart reviews, as it is unlikely health authorities will accept modeled survival benefits if the disease course is fully unknown.

For active therapies, the focus should be on the survival treatment effect, defined by, for example, hazard ratios (HR) over (different segments of) the disease course, instantaneous hazards, or incremental survival. Typically, prior to pivotal trial readout, limited information on relative treatment efficacy of the investigational therapy is available. Therefore, one may need to rely on phase 2 studies, historical trials in the target setting that ideally compare the same drug classes as those in the pivotal trial, later lines of therapy, or other similar indications.

Experts need to assess how the differences in factors, such as treatment pathway or patient characteristics, between information sources and defined target setting impact survival and associated treatment effect. For this, we suggest developing disease process and service pathway problem-oriented conceptual models, as suggested in NICE TSD 13. These conceptual models should inform the qualitative discussions with clinical experts on life time survival and incremental survival [15]. As an extension to the problem-oriented conceptual models suggested in TSD 13, we suggest developing a problem-oriented conceptual model for the treatment effect. The treatment-relevant aspects and corresponding considerations influencing the lifetime relative or incremental survival are presented in Table 1. PBAC also provides relevant guidance on factors influencing the treatment effect [37]. Figure 1 was developed to help capture the longitudinal impact of different aspects on the treatment effect. Patients enter the treatment effect conceptual model while starting treatment, either the investigational therapy of the pivotal trial or SoC. Clinical experts then need to qualitatively describe how the anticipated treatment effect might evolve while on treatment, during potential off-treatment periods (e.g., due to finite therapy), and while on the subsequent treatment lines. It is important to start with a reference treatment effect (e.g., OS HR) from the study protocol, historical trials, and or risk equations. If the ongoing trial might not be generalizable to the target setting, effect modification also needs to be considered.

Table 1 Aspects important for developing a problem-oriented conceptual treatment effect modelFig. 1

Illustrative problem-oriented model for factors influencing relative effectiveness in lifetime survival estimation. HR, hazard ratio; Tx, treatment

Where the treatment effect on an early endpoint is carried forward to a treatment effect on OS by assuming surrogacy, it is important that the experts consider the potential that an overall survival benefit might not materialize, e.g., surrogacy paradox [38]. For instance, patients achieving a pathological complete response (pCR) might be less aggressively treated in subsequent treatment lines than those who did not achieve a pCR, which might explain that pCR is not a valid surrogate and that a treatment effect on pCR does not automatically mean a treatment effect on OS [39].

3.1.4 DICSA Step 4

The fourth DICSA step concerns Setting expectations about survival in the target setting and corresponding plausible ranges by means of a structured expert elicitation (SEE) process. The information generated in DICSA steps 1–3, potentially formalized in an early model in combination with SEE techniques, can be used to generate predicted lifetime survival and incremental survival and corresponding plausible ranges [4, 10, 31, 40,41,42,43,44,45,46,47,48,49,50]. For all relevant treatment arms, SEE can include questions such as:

What percentage of patients are expected to be alive at 2 years, 5 years, 10 years, and 20 years in the target setting for SoC?

Given the percentages for SoC, what are the corresponding percentages for the active treatment?

For designing high-quality SEE, including expert selection, we refer to Bojke et al. and to the work of Ren et al. specific to the use of SEE for assessment of the plausibility of survival extrapolations [40,41,42, 51]. SEE can be used either directly within model extrapolations (such as is proposed in the survextrap package) or to set expectations for comparing extrapolations that do not directly use the data [52]. DISCA step 4 addresses the latter of these options. For more details on the SEE approach, please see the Supplementary Materials.

3.1.5 DICSA Step 5

The final step of DICSA is to Assess how trial-based extrapolations align with the set expectations by comparing modeled survival extrapolations to the range of values a priori considered to be plausible in DICSA 4. If they are aligned, the survival extrapolations are plausible. If not, the model extrapolations seem implausible. However, there are instances where model extrapolations might be plausible and DICSA 4 predictions may prove to be implausible, as plausibility depends on the knowledge of the day. For instance, for diseases with rapidly changing clinical practices, and trials investigating a treatment with a new mechanism of action, expectations formed at DICSA step 4 may no longer be current at DICSA step 5. Therefore, it is important to update the literature review (DICSA 2) at this point. When modeled extrapolations go outside the plausible ranges, it is crucial to assess why this might be the case. Here it is important to start with the standard of care arm survival expectations and extrapolations, as the survival extrapolations and expectations of the active arm depend on those for standard of care. Such an assessment may justify, despite violations, a conclusion that the modeled survival extrapolations are indeed plausible. For example, trial recruitment may not have reflected the predicted population, subsequent treatment use may be different from expected, or a therapy with a new mechanism of action might over- or under-perform the expectations.

If model extrapolations are implausible, one should consider using a different survival extrapolation model, adjusting for subsequent treatments, applying population adjustment techniques, and/or using SEE techniques to directly inform survival extrapolations.

The assessment and any potential justifications should be transparently reported in the HTA technical report submitted to health authorities as well as the original set expectations.

Figure 2 provides an overview of how the authors came to their findings.

Fig. 2

Flow diagram of how HTA review and more general definitions of biological plausibility [9] led to the findings presented in this study

An infographic of DICSA is presented in the Supplementary Materials.

3.2 Survival Extrapolation Protocol Template

For HTA agencies, it is essential that both the underlying model assumptions related to survival extrapolations and the resulting survival extrapolations are plausible. Therefore, DICSA should be ideally embedded in an extrapolation protocol. Table 2 provides a survival extrapolation protocol template. The survival extrapolation protocol template was developed to reflect and integrate the key considerations outlined in the HTA guidance documents identified from NICE, HAS, PBAC, and CDA and supplemented by the authors’ experience.

Table 2 Proposed survival extrapolation protocol template