Illustrative Data set

Figure 1 shows the network of comparisons of trials of antipsychotic medication for the prevention of relapse in people with schizophrenia. Each “edge” in the network indicates that the treatments at either end have been compared in an RCT, and the number on the edge indicates the number of trials. The data set includes 17 trials comparing nine treatments including placebo; eight of the trials are placebo controlled. There are 36 possible pair-wise contrasts between the nine treatments, and the present data set provides direct evidence on 11 of them. The methods used to identify these studies, and the criteria for inclusion and exclusion in the data set have been described previously 7. Briefly, a systematic search of the literature was undertaken to identify double-blind RCTs of antipsychotics used for relapse prevention in people with schizophrenia who are in remission. The review was conducted during the update of a clinical guideline on schizophrenia, commissioned by the National Institute for Health and Clinical Excellence in the UK 7. Although ziprasidone was not considered during the formulation of guideline recommendations, as it is not licensed in the UK, ziprasidone trials were included in the systematic review (and subsequently this analysis) to strengthen inference about the relative effect between other treatments. The analysis described in this article was similar to the one used to populate the decision-analytic economic model that informed the guideline recommendations 7.

Network diagram. The presence of connecting lines indicates which pairs of treatments have been directly compared in randomized trials: the numbers on the lines indicate the numbers of trials, those by the treatment names are the numbers assigned in Table 1 and in the mixed treatment comparison model. Three of the four trials comparing haloperidol and olanzapine were aggregated (see text).

The data available from each trial are the number of patients in each of the three outcome states at the end of follow-up. The outcome states are: relapse, discontinuation of treatment because of intolerable side effects, and discontinuation for other reasons, which might include inefficacy of treatment that did not fulfill all criteria for relapse, or loss to follow-up. Patients not reaching any of these end points at the end of follow-up were considered as censored observations, and still in remission. Individual patient data with times of transition were not available. Study follow-up varied from 26 to 104 weeks. The available data are shown in full in Table 1. Three trials comparing olanzapine and haloperidol were pooled as if they were a single study, because the original publication trials did not report all three outcomes separately.

Table 1. Trials of treatments for schizophrenia Trial Duration (weeks) Placebo1 Olanzapine2 Amisulpride3 Zotepine4 Aripiprazole5 Ziprasidone6 Paliperidone7 Haloperidol8 Risperidone9 1. Beasley 2003 42 28, 12, 15; 102 9, 2, 19; 224 2. Dellva 1997 – 1 46 7, 0, 4; 13 10, 2, 16; 45 3. Dellva 1997 – 2 46 5, 2, 5; 14 6, 10, 15; 48 4. Loo 1997 26 5; 5; 39; 72 4, 1, 26; 69 5. Cooper 2000 26 21, 4, 24; 58 4, 16, 21; 61 6. Pigott 2003 26 85, 13, 12; 155 50, 16, 18; 155 7. Arato 2002 52 43, 11, 7; 71 71, 19, 28; 206 8. Kramer 2007 47 52, 1, 7; 101 23, 3, 17; 104 9. Simpson 2005 28 11, 6, 44; 71 8, 5, 33; 55 10. Tran 1998 52 87, 54, 170; 627 34, 20, 50; 180 11. Study S029 52 28, 9, 26; 141 29, 14, 25; 134 12. Tran 1997 28 20, 17, 36; 172 53, 17, 18; 167 13. Speller 1997 52 5, 3, 2; 29 9, 5, 2; 31 14. Csernansky 2000 52 65, 29, 80; 188 41, 22, 60; 177 15. Marder 2003 104 8, 0, 4; 30 4, 3, 4; 33 Each cell gives: numbers of patients who relapse, discontinue because of side effects, discontinue for other reasons, and the denominator at initially at risk. Citations for included studies can be found in 7. The Tran 1998 data represent pooled results from three trials in which discontinuation data were not separately for each trial. Statistical Model

We begin with a general formulation for competing risks based on standard results from survival analysis 8. If λm(t) is the cause-specific hazard at time t for outcome m, then the conditional probability that failure at time t is of type m, given there is a failure at time t is

(1)

The probability that failure occurs before time D and is of type m is: (the probability of surviving to t, times the probability of failure at t, times the conditional probability that failure is of type m), integrated over all times t between zero and D. This is:

(2)

A form of “proportional hazards” assumption can now be made, which might be better termed “proportional competing risks,” in which the ratio (Equation 1) is constant over all t (i.e., πm(t) = πm). Under this restriction, Equation 2 becomes:

(3)

which is the probability of failure before time D times the probability that failure was of type m. In what follows, we assume constant underlying hazards, but Equation 3 shows that with proportional competing risks we are free to fit more complex survival distributions. This suggests some useful extensions to which we return in the discussion.

We now number the treatments from 1 to 9 (as shown in Table 1 and Fig. 1). Placebo is selected as the reference treatment 1. This is an arbitrary choice, but made to ease interpretation. We define the three outcomes as: m = 1 relapse, 2 = discontinuation caused by side effects, and 3 = discontinuation for other reasons. Then, each outcome is modeled separately on the log hazard rate scale. Considering a trial j that compares treatments k and b, the cause-specific log hazard for outcome m for treatment T is:

(4)

where δj,b,k,m is the trial-specific log hazard ratio of treatment k relative to treatment b for outcome m. This can be interpreted as meaning that the b arm of the trial estimates the baseline log hazard µj,m, while the k arm estimates the sum of the baseline hazard and the log hazard ratio. Note that b is not necessarily treatment 1, nor is it the same treatment in every trial; instead, it is simply the treatment with the lowest index in that trial. Thus, in a trial comparing treatments 2 and 3, b = 2. The trial-specific log hazard ratios are assumed to come from a common normal distribution, following the standard “random effects” approach:

(5)

The mean of this distribution is a difference between mean relative effects dk,m and db,m, which are the mean effects of treatments k and b, respectively, relative to (placebo) treatment 1, for outcome m, and we define d1,m = 0. This formulation of the problem expresses the consistency equations 9, by which the dimensionality of the 11 treatment contrasts on which there are direct data (Table 1 and Fig. 1), are reduced to functions of the eight contrasts between the active treatments and placebo. The between-trial variance of the random effect distribution, , is specific to each outcome m. Three models for the variance are considered below.

We may write the model as Equation 4 because all the trials in this example are two-arm trials. An advantage of our approach, however, is that it can be readily extended to multi-arm trials, and Equation 5 should in fact be interpreted as a “fragment” of a multivariate normal distribution:

(6)

The underlying assumption in Equations 5 and 6 is therefore that every trial may be considered as if it were a multiarm trial on all nine treatments, that trial-specific relative treatment effects are sampled from the multivariate normal in Equation 6, and that treatments are missing at random. (Note that missing at random means missing without regard to treatment efficacy; it does not mean that treatment arms are equally likely to be included in a trial).

The linking function that relates the arm-specific log hazards θj,k,m to the likelihood is developed as follows. Figure 2 shows a Markov transition model with a starting state (remission) and three absorbing states (relapse, discontinuation caused by side effects, and discontinuation caused by other reasons). Based on Equation 3, if we assume constant hazards λj,k,m acting over the period of observation Dj in years, the probability that outcome m had occurred by the end of the observation period for treatment T in trial j is:

Markov model showing competing risk structure. Patients in trial j, on treatment T, move from remission to relapse, discontinued treatment (side effects), and discontinued treatment (other reasons), at rates exp(θj,T,1), exp(θj,T,2), and exp(θj,T,3), respectively.

(7)

The probability of staying in the remission state (m = 4) is now simply 1 minus the sum of the probabilities of arriving at the three absorbing states, that is,

(8)

The data for each trial j and treatment T constitute a multinomial likelihood with four outcomes: moving to one of the three absorbing states, or remaining in the initial remission state. If rj,T,m is the number of patients on treatment T observed to reach end point m, and nj,T is the total number at risk on treatment T in trial j, then:

(9)

Three different models were fitted, differing solely in the specification of the between-trial variation in relative treatment effects, . In the fixed effects model, , and the model collapses to: θj,T,m = µj,m + dk,m − db,m, b = 1,2 . . .  8, k ≥ b.

In the random effect single-variance model, the between-trial variance , reflecting the assumption that the between-trial variation is the same for each outcome: a vague inverse gamma prior was put on the variance, 1/σ2∼ gamma (0.001, 0.001). In the random effect different variances model, each outcome has a different between-trial variation, and the vague uniform prior is put on each: . A sensitivity analysis based on uniform priors was also examined: σ ∼ uniform (0, 5). This gave virtually identical posteriors for the treatment effects, but resulted in posterior distributions with “spikes” at σm values at or close to zero and spikes in the posterior mean treatment effects. Gamma priors, which give zero weight to infinite precision and hence to zero SD, were therefore used in the primary analyses reported below.

Finally, in each of the three models, vague Normal (0, 1002) priors were put on all the trial baselines µj,m and mean treatment effects dk,m. The model for treatment effects (Equations 4 and 5) is therefore identical to that previously proposed for mixed treatment comparisons (MTCs) except that the multinomial likelihood (Equation 9) and linking function (Equation 7) are used, as is appropriate for the data at hand, in place of the binomial likelihood and logit link function proposed in most of previous work on these kinds of evidence structures 9-13.

Model Selection

Choice of models was based on the deviance information criterion 14. This is a deviance measure of goodness of fit, , equal to the posterior mean of minus twice the log likelihood, penalized by an estimate of the effective number of parameters in the model, pD. The DIC can be seen as a Bayesian measure analogous to the Akaike information criterion used in classical analysis, but which can also be applied to hierarchical models. Here, we adjust the standard deviance formula by subtracting the deviance of the saturated model (a constant). The contribution of each multinomial observation (trial j treatment T) to the deviance is:

(10)

where is an estimate of the probability of outcome m, for example, the estimate generated on some MCMC cycle, and is the posterior mean of the sum of the deviance contributions over all data points, . In a model that fits well, is expected to roughly approximate the number of data points. In this data set, with 15 two-arm trials each reporting three outcomes (the fourth, “censored” outcome is predicted from the number at risk less the other outcomes), the number of independent data points is 90; pD is equal to , where is the sum of the deviance contributions, evaluated at the posterior mean of the fitted values 15.

Computation

Models were estimated using the freely available Bayesian MCMC software WinBUGS 1.4.3 16. Convergence for all models occurred within 10,000 to 25,000 iterations as assessed by the Brooks–Gelman–Rubin criteria 17. Results are based on 150,000 samples, from three separate chains with disparate starting values for fixed effect, and five chains for the random effect models, taken after the first 60,000 were discarded. We also established that, in each model, all the chains converged to the same posterior. The code for each model is available on the ISPOR Web site as Supporting Information, in Appendix A at: http://www.ispor.org/Publications/value/ViHsupplementary/ViH13i8_Ades.asp.