The results of this study show that an incorrect covariate scope reduction may lead to substantial omission bias, especially in covariate coefficients when the parameters of the model are truly correlated and this correlation structure is (correctly) included in the estimation model. With these misspecified models, the exclusion of a relevant covariate-parameter relationship biased the estimation and, for covariate coefficients and IIV parameters, also somewhat reduced their precision. Inclusion bias, on the other hand, had minor, and mostly negligible effects on model elements of misspecified FEM models, except in some cases in which an IIV correlation was (correctly) included in the estimation. Finally, the addition of a non-relevant covariate in FREM did not induce any inclusion bias, and the estimates of all model parameters showed basically no difference from the “true” values used as a starting point for the simulations.

According to common recommendations in the literature, judicious scope reduction of the covariate-parameter relationships considered in the analysis should be performed to avoid problems with correlated covariates, long runtimes, and model instabilities. It is typically assumed that once a specific covariate-parameter relationship is excluded from the analysis (based on whatever rationale), such a scope reduction will not have an effect on the parameter estimates of the model, perhaps except for inflated IIV estimates. However, apart from the work from Ivatouri et al. [9], the impact the exclusion of relevant covariates has on parameter estimates has not been systematically explored. In the analysis presented here, we considered different scenarios of a simple one-compartment model with first-order absorption including three model parameters (CL, V, and Ka), with and without IIV correlation, and a single covariate that, when included, was added either on one or on two parameters (CL and V). Permutations of these settings were used to generate simulated datasets, which were then used to re-estimate the model elements, either as FEM models or as two FREM models. By comparing combinations of these models’ setups, many scenarios were generated. This way, possible biases of parameters’ estimates due to model misspecifications could be captured separately for parameters that were truly affected by the covariate (i.e. CL and/or V, depending on the case considered) and for the parameter that was not affected by it (Ka).

In terms of omission bias of typical model parameters, our results show that the situation is rather complex. Using FEM, an incorrect scope reduction, omitting a relevant covariate on a single model parameter (e.g. V), can actually bias all parameters on which that covariate has an effect (e.g. CL and V). On the other hand, the parameter unaffected by the covariate (Ka in our case) was not impacted by this omission bias. However, the lack of bias of the specific Ka estimates found in our model with three parameters cannot necessarily be generalized to more complex models, in which many more parameters and covariates need to be considered and might be somehow correlated (see discussion below about parameters and covariate correlations). When using FREM, model parameters resulted in estimates generally close to the true values.

Regarding estimated covariate coefficients, an incorrect scope reduction introduced a considerable bias in the estimated coefficients. This was clearly higher than that observed for typical parameter values. Furthermore, a considerable bias in the covariate coefficients estimated by misspecified FEM models was especially observed in the scenarios in which an IIV correlation was correctly included. Conversely, the results obtained using FREM led to covariate coefficients that were close to the correct scenarios, independently of the model parameters correlations considered.

The estimated IIV parameters were also affected by an incorrect scope reduction. In this case, the observed increase in the IIV estimates is perhaps not unexpected since the model misspecification involves the exclusion of a parameter that could have explained some of the variability in the data, had it been included in the estimation model.

Finally, individual parameter distributions were investigated (data not shown) and indicated only minor differences between the true underlying model and the estimated misspecified models. This indicates that e.g. model-informed precision dosing methods are less sensitive to omission/inclusion bias compared to the fixed effects estimates (conditional on the amount of available individual data).

The impact of IIV correlations on the typical parameters and covariate coefficients is of particular interest. Both omission bias and, to some extent an inclusion bias, are higher when an IIV correlation is correctly included in the model and the covariate model is misspecified. Another way to express this observation is that if the estimation model has both an incorrect scope reduction and a misspecified IIV correlation, there is no omission bias. These results then suggest that the possible effects of scope reductions need to be considered even more carefully when applied to models for which IIV correlations are expected. Similarly, if goodness-of-fit plots suggest that an IIV correlation is present in the data when a covariate scope reduction has been carried out, these signals should be interpreted with caution, especially if the covariate coefficients (and other parameters of the model) change drastically when the IIV correlation is included. This issue with IIV correlation was also discussed in the paper by Sanghavi et al. [4]: there, the authors suggest that the most appropriate choice in such cases would probably be to apply the covariate effect on only one of the model parameters, chosen based on scientific plausibility and purpose of the model developed. Our results, however, indicate that this recommendation is not always suitable and that the less-harmful strategy may be to include the covariate effect on both parameters. As stated earlier, our study represents a simple case, in which we considered the inclusion of one single covariate on model parameters. Despite the inclusion of different covariates on model parameters (e.g., renal function on CL and WT on V) has not been explored here, we expect that in such case the results would scale proportionally to the correlation between the two covariates, assuming a linear relationship between the two. Another consideration based on these results is whether or not to include IIV correlations in the base model prior to any scope-reduced covariate analysis. However, given the limited number of models investigated, more evidence is needed before making such a strong recommendation. Since FREM models include the covariates on all parameters it avoids the issue of omission bias even though all IIV correlations are estimated.

As a next step in the analysis, we then considered FREM and the possible introduction of an inclusion bias. FREM is a “saturated model” in the sense that all covariate-parameters relationships are included regardless of plausibility. Therefore, this method might be particularly sensitive to inclusion bias. However, this was not confirmed by our results, which actually suggest that inclusion of a non-relevant covariate on one or more parameters does not affect model elements. Parameter values and their IIVs were not biased, and covariate coefficients were correctly estimated to values close to zero and thus did not influence the overall model performance.

For a more complete comparison, inclusion bias was evaluated also for FEM estimation models that included non-relevant covariates on model parameters. Although FEM models generally did not induce inclusion bias, in cases in which the (correct) IIV correlation was included in the estimation model, covariate coefficients were under-estimated.

One unexpected observation in our data was a tendency for bias even in the correct scenarios for the typical parameters as well as for covariate coefficients. Attempts to resolve this, for example by changing estimation settings with different CTYPEs, increasing the number of iterations and perturbing the initial estimates, were unsuccessful. However, since this bias was around 1%, it is expected that the conclusions from this work are unaffected.

In light of the results of this analysis, a consideration on covariate plausibility needs to be made across different covariate model building strategies. Data-driven covariate selection methods, such as SCM, aim at selecting only the covariate-parameter relationships needed to adequately describe the data, which means that covariate-parameters relationships are removed from the corresponding potential “full model” based on the signals in the data. Except for runtime gains, this may mean that the reasons for a covariate scope reduction with such methods are fewer than with pre-specification methods. A beneficial aspect of data-driven selection methods is that mechanistically implausible covariates are less likely to be included in the final model. However, if they are selected, they can, from a communication perspective, be perceived as provocative. Regarding pre-specification methods using FEM, a modeler would naturally tend to avoid such parameter-covariate combinations by performing a covariate scope reduction. However, this tendency, combined with the general encouragement in the literature for a covariate scope reduction, and the need to increase model stability (when many covariates are to be included in the model using fixed effects), will likely lead to the exclusion of more covariates than would be wise, according to the results presented in this work. Since FREM is less sensitive to model instabilities due to the number of covariates, the need for a covariate scope reduction is smaller than with FEM models. On the other hand, FREM by default requires that all covariates that are included in the analysis are affecting all parameters considered regardless of mechanistic plausibility. However, instead of interpreting the inclusion of such covariate-parameter combinations as a suggestion of their mechanistic plausibility or clinical relevance, their inclusion should rather be regarded as a safe way to protect against omission bias. Nevertheless, even if inclusion bias does not seem to be an issue, it is as always important to carefully consider the included covariate effects with the aim of identifying false positive relationships. FREM is therefore likely to be a more robust pre-specification covariate modelling method than other pre-specification approaches that rely on FEM.