Data-Driven System for Perioperative Acuity Prediction

Abstract

The widely used American Society of Anesthesiologist’s’ (ASA) Physical Status classification is subjective and requires time-consuming clinician assessment. Machine learning can be used to develop a system that predicts the ASA score a patient should be given based on routinely available preoperative data. The problem of ASA prediction is reframed into a binary classification problem for predicting between ASA 1/2 versus ASA 3/4/5. Retrospective ASA scores from the Vanderbilt Perioperative Data Warehouse are used as labels, allowing the use of supervised machine learning techniques. Routinely available preoperative data is used to select features and train four different models: logistic regression, k-nearest neighbors, random forests, and neural networks. Of the selected features, ICD9 codes were tested by incorporating temporality and hierarchy. The area under the curve (AUC) of the receiver operating characteristic (ROC) of each model on a holdout set is compared. The Cohen’s Kappa is calculated for the model versus the raw data and the model versus our anesthesiologist. Results: The best performing model was the random forest, achieving an AUC of 0.884. This model results in a 0.63 Cohen’s Kappa versus the raw data, and a 0.54 Kappa against our anesthesiologist, which is comparable to unweighted Kappa values found in literature. The results suggest that a machine learning model can predict ASA score with high AUC, and achieve agreement similar to an anesthesiologist. This demonstrates the feasibility of using this model as a standardized ASA scorer.