Development of Prognostic Models in the Presence of Competing Risks and Model Uncertainties: an Application to Peadiatric in-hospital Mortality

Abstract

This report delves into the practical applications of advanced modeling in the realm of prognostic modeling for pediatric in-hospital mortality. This research commenced with a comprehensive systematic review aimed at identifying predictive models for in-hospital mortality among pediatric patients in resource-limited settings. While the review unearthed twenty-one prognostic models from fifteen studies, it also unveiled significant methodological concerns. These included issues such as poor reporting, suboptimal handling of missing data, inadequate sample sizes, and misjudged categorization of continuous predictors, which collectively cast doubt on the models' predictive capabilities. Subsequently, the research progressed to external validation, assessing the predictive ability of the identified models using data from pediatric patients in 20 county referral hospitals between 2014 and December 2021. Of the 21 models, only 4 met the criteria for external validation. The validation metrics encompassed discriminatory ability (c-statistics) and model calibration (slope and intercepts). The findings consistently revealed a trend of underestimating the risk of mortality in all four models, highlighting the potential for misclassifying high-risk patients. To rectify the miscalibration issue, the focus shifted towards recalibrating these models. Two recalibration strategies were explored, with logistic recalibration proving more effective. However, the improvements, while notable, did not meet the necessary clinical standards, primarily due to a lack of consideration of model uncertainty during the development of the individual models in their original studies. Addressing the pivotal problem of model uncertainty, a stacking of model predictive distributions methodology was introduced. This innovative approach merged predictive distributions from four distinct models (which were refitted), enhancing the accuracy and reliability of mortality risk predictions. When comparing the performance of the individual models with the stacked posterior distribution, the latter surpassed individual models, offering improved discrimination and calibration, promising significant advancements in predictive

accuracy. The focus then shifted to the Fine-Gray Sub-distribution Hazard model in the context of competing risks, with Monte Carlo simulations revealing the impact of patient follow-up duration on model accuracy. The findings underscored the challenges in managing competing risks and the limitations of established approaches, particularly in epidemiological research. In conclusion, this study embarks on an innovative journey into the development, validation, and recalibration of prognostic models for predicting in-hospital mortality in pediatric patients. It underscores the importance of ensemble techniques in mitigating model uncertainties and improving predictive accuracy. Despite remarkable progress, further research is needed to address the intricacies of competing risks and enhance model reliability.