1. Steyerberg, EW, Vickers, AJ, Cook, NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiol Camb Mass. 2010;21(1):128–38.
Google Scholar | Crossref2. Janssens, ACJW, Martens, FK. Reflection on modern methods: revisiting the area under the ROC curve. Int J Epidemiol. 2020;49(4):1397–1403.
Google Scholar | Crossref | Medline3. Vergouwe, Y, Moons, KGM, Steyerberg, EW. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol. 2010;172(8):971–80.
Google Scholar | Crossref | Medline4. van Klaveren, D, Steyerberg, EW, Vergouwe, Y. Interpretation of concordance measures for clustered data. Stat Med. 2014;33(4):714–6.
Google Scholar | Crossref | Medline5. van Klaveren, D, Gönen, M, Steyerberg, EW, Vergouwe, Y. A new concordance measure for risk prediction models in external validation settings. Stat Med. 2016;35(23):4136–52.
Google Scholar | Crossref | Medline6. Gönen, M, Heller, G. Concordance probability and discriminatory power in proportional hazards regression. Biometrika. 2005;92(4):965–70.
Google Scholar | Crossref7. Shah, ND, Steyerberg, EW, Kent, DM. Big data and predictive analytics: recalibrating expectations. JAMA. 2018;320(1):27–8.
Google Scholar | Crossref8. Wessler, BS, Paulus, J, Lundquist, CM, et al. Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015. Diagn Progn Res. 2017;1(1):Article 20.
Google Scholar | Crossref | Medline9. Van Calster, B, McLernon, DJ, van Smeden, M, Wynants, L, Steyerberg, EW, Topic group ‘evaluating diagnostic tests and prediction models’ of the STRATOS initiative. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17(1):230.
Google Scholar | Crossref | Medline10. Van Calster, B, Nieboer, D, Vergouwe, Y, De Cock, B, Pencina, MJ, Steyerberg, EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167–76.
Google Scholar | Crossref | Medline11. Pepe, MS, Cai, T. The analysis of placement values for evaluating discriminatory measures. Biometrics. 2004;60(2):528–35.
Google Scholar12. Krzanowski, WJ, Hand, DJ. ROC Curves for Continuous Data. Boca Raton (FL): CRC Press; 2009.
Google Scholar | Crossref13. Pepe, MS. The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford (UK): Oxford University Press; 2004.
Google Scholar14. Steyerberg, EW, Vergouwe, Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31.
Google Scholar | Crossref | Medline15. Fisher, RA. Statistical Methods for Research Workers. Edinburgh (UK): Oliver and Boyd; 1930.
Google Scholar16. Brown, MB. A method for combining non-independent, one-sided tests of significance. Biometrics. 1975;31(4):987.
Google Scholar | Crossref17. Morris, TP, White, IR, Crowther, MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074–102.
Google Scholar | Crossref | Medline18. R Core Team. R : A Language and Environment for Statistical Computing [Internet]. Vienna (Austria): R Foundation for Statistical Computing; 2019. Available from: https://www.R-project.org/
Google Scholar19. Sadatsafavi, M, Sin, DD, Zafari, Z, et al. The association between rate and severity of exacerbations in chronic obstructive pulmonary disease: an application of a joint frailty-logistic model. Am J Epidemiol. 2016;184(9):681–9.
Google Scholar | Crossref | Medline20. Adibi, A, Sin, DD, Safari, A, et al. The Acute COPD Exacerbation Prediction Tool (ACCEPT): a modelling study. Lancet Respir Med. 2020;8(10):1013–21.
Google Scholar | Crossref | Medline21. Albert, RK, Connett, J, Bailey, WC, et al. Azithromycin for prevention of exacerbations of COPD. N Engl J Med. 2011;365(8):689–98.
Google Scholar | Crossref | Medline22. Criner, GJ, Connett, JE, Aaron, SD, et al. Simvastatin for the prevention of exacerbations in moderate-to-severe COPD. N Engl J Med. 2014;370(23):2201–10.
Google Scholar | Crossref | Medline23. Adibi, A, Sin, DD, Safari, A, et al. The Acute COPD Exacerbation Prediction Tool (ACCEPT): a modelling study. Lancet Respir Med. 2020;8(10):1013–21.
Google Scholar | Crossref | Medline24. Allison, PD . Measures of fit for logistic regression. Statistical Horizons LLC and the University of Pennsylvania. January 13, 2020. Report No. 1485–2014. Available from: https://support.sas.com/resources/papers/proceedings14/1485-2014.pdf
Google Scholar25. Hosmer, DW, Hosmer, T, Le Cessie, S, Lemeshow, S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965–80.
Google Scholar | Crossref | Medline26. Harrell, FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed. New York: Springer; 2015.
Google Scholar | Crossref27. Austin, PC, Steyerberg, EW. The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression models. Stat Med. 2019;38(21):4051–65.
Google Scholar | Crossref28. Van Hoorde, K, Van Huffel, S, Timmerman, D, Bourne, T, Van Calster, B. A spline-based tool to assess and visualize the calibration of multiclass risk predictions. J Biomed Inform. 2015;54:283–93.
Google Scholar | Crossref | Medline29. Morris, DE, Pepe, MS, Barlow, WE. Contrasting two frameworks for ROC analysis of ordinal ratings. Med Decis Making. 2010;30(4):484–98.
Google Scholar | SAGE Journals30. Saha-Chaudhuri, P, Heagerty, PJ. Non-parametric estimation of a time-dependent predictive accuracy curve. Biostat Oxf Engl. 2013;14(1):42–59.
Google Scholar | Crossref | Medline31. Bansal, A, Heagerty, PJ. A tutorial on evaluating the time-varying discrimination accuracy of survival models used in dynamic decision making. Med Decis Making. 2018;38(8):904–16.
Google Scholar | SAGE Journals32. Vergouwe, Y, Nieboer, D, Oostenbrink, R, et al. A closed testing procedure to select an appropriate method for updating prediction models. Stat Med. 2017;36(28):4529–39.
Google Scholar | Crossref | Medline33. Jameson, JL, Longo, DL. Precision medicine—personalized, problematic, and promising. N Engl J Med. 2015;372(23):2229–34.
Google Scholar | Crossref | Medline34. Janssens, ACJW . The ROC plot: the picture that could be worth a 1000 words. J Clin Epidemiol. 2020;126(5).
Google Scholar35. Van Calster, B, Wynants, L, Collins, GS, Verbakel, JY, Steyerberg, EW. ROC curves for clinical prediction models part 3: the ROC plot: a picture that needs a 1000 words. J Clin Epidemiol. 2020;126:220–3.
Google Scholar | Crossref