KR102551204B1 - System and method for supporting clinical decision for acute kidney injury using recurrent neural network model - Google Patents

Abstract

A clinical decision support system using an RNN machine learning model according to an embodiment of the present invention includes an input unit that sequentially receives patient-related information; A first prediction model using an RNN machine learning algorithm and predicting the possibility of acute kidney injury (AKI) in a patient based on input patient-related information; a second prediction model using an RNN machine learning algorithm and predicting a change in serum creatinine level of a patient based on a prediction result of the first prediction model; and a plot generation unit that generates at least one plot that visually represents an effect of each characteristic variable on a prediction result with respect to at least one characteristic variable included in the patient-related information. According to the embodiment, it is possible to predict the possibility of AKI occurrence and changes in serum creatinine levels in inpatients in real time using an RNN machine learning model, and a plot that visually shows the relationship between the results and feature variables that affect the results of the prediction model. can provide insight into clinical decision making.

Description

Clinical decision support system and method for acute kidney injury using recurrent neural network prediction model

The present invention relates to a clinical decision support system and method, and more particularly, predicts the possibility of acute kidney injury and changes in serum creatinine levels in real time by using an externally verified recirculatory neural network as a predictive model, It relates to a system and method for supporting medical personnel's clinical decision-making by displaying it in a visual plot.

Acute Kidney Injury (hereinafter referred to as “AKI”) is a disease in which renal function is weakened as kidney cells are damaged, and is a relatively common disease that occurs in 5-10% of all hospitalized patients. When kidney function is weakened due to AKI, waste products are not excreted and accumulated in the body, and if the treatment period is missed, the risk of dialysis increases and may lead to death.

The occurrence of AKI can cause complications to patients, which leads to an increase in medical costs. Therefore, despite the need for many efforts to predict the occurrence of AKI early and manage high-risk patients, the incidence of AKI has been increasing over the past few decades.

Due to the recent rapid development of machine learning-based artificial intelligence technology and the increase in electronic health record (EHR) data, it is difficult to predict or manage the occurrence of a disease by using the cumulative data of patients for a specific disease. It became possible. In particular, machine learning models can incorporate many features compared to conventional regression models, enabling the use of non-linear algorithms, and as a result, random forests and neural networks in the field of AKI occurrence prediction. ), various studies employing machine learning methods such as models have been reported.

However, most of the currently developed AKI prediction models have a simple design as an alarm system for timely diagnosis of AKI according to established diagnostic criteria. In addition, it is difficult to apply in general situations because it does not go through external validation, and it is difficult to apply in general situations due to the black box characteristics of the neural network model (i.e., it is difficult to interpret what criteria and methods the artificial intelligence algorithm derived the results from). However, it is not directly helpful in making clinical decisions.

KR 10-2165840

Therefore, the present invention predicts the possibility of acute kidney injury (AKI) in hospitalized patients and changes in serum creatinine levels in real time using an RNN machine learning model externally verified through an independent test set, and displays them in a visual plot. The purpose of this study is to provide a system and method that can support the clinical decision-making of medical personnel by making it possible to intuitively know the relationship between variables and results.

A clinical decision support system using an RNN machine learning model according to an embodiment includes an input unit that sequentially receives patient-related information from day 1 to day n; A first prediction model using an RNN machine learning algorithm and predicting the possibility of acute kidney injury (AKI) in a patient based on input patient-related information; a second prediction model using an RNN machine learning algorithm and predicting a change in serum creatinine level of a patient based on a prediction result of the first prediction model; And with respect to at least one feature variable included in the patient-related information, generating at least one plot for visually representing the effect of each feature variable on the possibility of acute renal injury predicted by the first predictive model Includes a plot generator.

According to an embodiment, the first predictive model calculates the possibility of acute renal injury occurring in a patient within a predetermined period from the present time based on sequentially input patient-related information from day 1 to day n, and outputs the result as a result. However, the result may be calculated using the patient's condition information up to day n-1 and the patient-related information input on day n.

According to an embodiment, the patient's condition information on any day m may be obtained using the patient's condition information up to day m−1 and the patient-related information input on day m.

According to one embodiment, the second predictive model calculates the serum creatinine level within a predetermined time interval from each date using patient-related information input on any day m and state information of the patient up to day m-1. The calculation is output as a result, but the patient's condition information up to day m-1 can be obtained using the patient's condition information up to day m-2 and the patient-related information input on day m-1.

According to an embodiment, the first predictive model may output a predicted probability representing a possibility of acute renal impairment in a patient, or classify a patient expected to develop acute renal impairment based on the predicted probability.

According to one embodiment, the second predictive model may predict the serum creatinine level for each time interval based on the output value of the first predictive model.

According to one embodiment, the first prediction model and the second prediction model are learned using data of patients admitted to a specific institution, and are externally trained using data of patients admitted to another institution independent of the institution. can be empirically verified.

According to an embodiment, the at least one characteristic variable included in the patient-related information includes static variables that do not change during the patient's hospitalization period or dynamic variables that change during the patient's hospitalization period, and the static variables are the patient's gender , age, BMI level, history of drug administration prior to hospitalization, and at least one of comorbidities, and the dynamic variables include at least one of drug administration during hospitalization, in-hospital test results, and vital signs, and may be updated at least once a day. can

According to an embodiment, at least one of the plots generated by the plot generator may include partial dependence plots, individual conditional expectation plots, accumulated local effect plots, Alternatively, a variable importance plot using sharp values may be included.

According to one embodiment, the partial dependence plot (PDP) and the cumulative local effect plot (ALEP) may represent average values of the probability of occurrence of acute kidney injury in hospitalized patients with respect to the selected feature variable.

According to one embodiment, the variable importance plot using the Shapley value may indicate the importance of variables contributing to the occurrence of renal injury in an individual patient and in the entire group in order.

According to one embodiment, the Individual Conditional Expectancy Plot (ICEP) may represent the likelihood of acute renal injury of an individual patient with respect to selected characteristic variables.

According to one embodiment, the cumulative local effect plot (ALEP) can accurately represent the probability of acute renal injury in hospitalized patients as a whole for a selected characteristic variable even when there is correlation between the variables.

According to one embodiment, the variable importance plot using the Shapley value may indicate the importance of variables contributing to the occurrence of renal injury in an individual patient and the entire group in order.

According to one embodiment, at least one plot generated by the plot generation unit may include a graph showing a change in serum creatinine level per hour according to administration and discontinuation of a nephrotoxic drug.

A clinical decision support method using an RNN machine learning model according to an embodiment includes sequentially receiving patient-related information from day 1 to day n; predicting a possibility of acute kidney injury (AKI) of the patient based on the input patient-related information using a first prediction model using an RNN machine learning algorithm; predicting a change in serum creatinine level of a patient based on a prediction result of the first prediction model by using a second prediction model using an RNN machine learning algorithm; And with respect to at least one feature variable included in the patient-related information, generating at least one plot for visually representing the effect of each feature variable on the possibility of acute renal injury predicted by the first predictive model Include steps.

A computer program stored in a computer-readable recording medium for implementing a method for supporting clinical decision making using an RNN machine learning model according to an embodiment may be provided.

According to an embodiment of the present invention, a system and method capable of predicting the possibility of acute kidney injury (AKI) in hospitalized patients and changes in serum creatinine levels in real time using an RNN machine learning model are provided. According to an embodiment, a visual plot may be provided so that a clinician can intuitively know the relationship between feature variables that affect the result of the predictive model and the result. This predictive model is externally verified using an external data set independent of the data set used for learning, and therefore can be universally used for other patient groups with high predictive reliability.

BRIEF DESCRIPTION OF THE DRAWINGS To describe the technical solutions of the embodiments of the present invention or the prior art more clearly, drawings required in the description of the embodiments are briefly introduced below. It should be understood that the drawings below are for the purpose of explaining the embodiments of the present specification and not for limiting purposes. In addition, for clarity of explanation, some elements applied with various modifications, such as exaggeration and omission, may be shown in the drawings below.
1 shows the structure of a clinical decision support system based on an RNN machine learning model according to an embodiment.
Figure 2 shows the structure of a first predictive model for predicting the possibility of AKI occurrence and a second predictive model for predicting changes in serum creatinine level.
3 is a flowchart illustrating a process of classifying a study group for learning and external verification of a predictive model according to an embodiment.
4A to 4Y show Partial Dependence Plots and Individual Conditional Expectation Plots for some feature variables presented to interpret the behavior of the predictive model.
5A to 5O are graphs showing cumulative local effect plots (ALEPs) in which the risk of acute renal injury according to values of some characteristic variables of the predictive model is displayed according to time from a more predicted time point.
6 is a variable importance plot using a Shapley value showing the order of importance of feature variables of a predictive model.
7 is a graph showing actual changes in serum creatinine levels per hour and predicted results according to administration and discontinuation of some nephrotoxic drugs.
8 is a flowchart illustrating each step of a clinical decision support method based on an RNN machine learning model according to an embodiment.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings and the contents described in the accompanying drawings, but the scope to be claimed is not limited or limited by the embodiments.

The terminology used in this specification has been selected as a general term that is currently widely used as much as possible while considering functions, but it may vary according to the intention or custom of a person skilled in the art or the emergence of new technology. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in the explanation part of the corresponding specification. Therefore, it should be clarified that the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, rather than simply the name of the term.

Further, the embodiments described herein may have aspects that are entirely hardware, part hardware and part software, or entirely software. In this specification, terms such as “unit”, “module”, “device” or “system” refer to computer-related entities such as hardware, a combination of hardware and software, or software. For example, a unit, module, device or system may refer to hardware constituting part or all of a platform and/or software such as an application for driving the hardware. there is.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

1 shows the structure of a clinical decision support system based on an RNN machine learning model according to an embodiment.

Referring to FIG. 1 , the clinical decision support system 10 according to the embodiment includes an input unit 100 that sequentially receives patient-related information from day 1 to day n; A first prediction model 200 for predicting the possibility of occurrence of acute kidney injury (AKI) in a patient based on the input patient-related information; a second prediction model 300 for predicting a change in serum creatinine level of a patient based on the prediction result of the first prediction model 100; and a plot generation unit 400 that generates at least one plot to visually represent an effect of at least one characteristic variable included in the patient-related information on a prediction result of the predictive model.

Patient-related information from day 1 to day n is sequentially input through the input unit 100 . Here, 'sequentially' means that patient-related information of the corresponding date is sequentially input for each date. That is, on the first day of hospitalization, the first-day patient-related information is input, and on the second day of hospitalization, the second-day patient-related information is input to the system.

Each patient-related information includes at least one feature variable, and each feature variable may be divided into a static variable that does not change during the patient's hospitalization period or a dynamic variable that changes during the patient's hospitalization period. For example, static variables refer to variables that hardly change at least during the period the patient is hospitalized, such as the patient's gender, age, BMI (calculated based on the patient's height and weight), drug administration history before hospitalization, and comorbidities. , dynamic variables include drug administration during hospitalization, in-hospital test results (blood creatinine level, leukocytes, hemoglobin, platelets, albumin, sodium, potassium, calcium, glucose, carbon dioxide, urea levels, etc.), vital signs (blood pressure, pulse, temperature, etc.) ) is a variable that changes continuously during the patient's hospitalization and is measured at least once a day.

These characteristic variables (particularly, dynamic variables) are used as a basis for predicting the possibility of future AKI occurrence or changes in serum creatinine levels in a predictive model. The table below shows 107 characteristic variables representing information related to inpatients used for the development of a predictive model in one embodiment, classified by category. Among them, all characteristic variables included in the categories of vital signs and medications and some characteristic variables of clinical conditions correspond to dynamic variables.

category variable Count demographics
(Demographics) Age, sex, BMI, weight, height 5 Laboratory tests Creatinine, WBC, hemoglobin, platelet, albumin, sodium, potassium, chloride, AST, ALT, BUN, total CO ₂ , bilirubin, calcium, glucose, CK (3 categories), lipase (3 categories), troponin I (3 categories) , and baseline eGFR 25 Vital signs Systolic blood pressure, diastolic blood pressure, pulse rate, mean arterial pressure, body temperature (mean, maximum, minimum per day) 15 Comorbidities Ischemia heart disease, hypertension, diabetes mellitus, heart failure, liver disease, cerebrovascular disease, chronic obstructive pulmonary disease, chronic kidney disease, previous AKI, cancer, HIV, and CCI 12 Medications NSAIDs, diuretics, acyclovir, aminoglycoside, amphotericin, beta blocker, calcium channel blocker, cisplatin, vancomycin, colistin, cyclosporin, renin-angiotensin blocker, vasopressors, statins (medication use during and before admission) 28 Clinical conditions Departments (14 categories), mechanical ventilation (current/recent), ICU admission, surgery (major/minor), anesthesia (general/non-general), surgery duration 22

The first prediction model 200 predicts the patient's possibility of acute renal injury for each AKI stage (mild symptom stage 1 or severe stage 2-3) based on the input patient-related information. Here, acute kidney injury (Acute Kidney Injury) was defined according to the criteria of KDIGO (Kidney Disease: Improving Global Outcomes, AKI Clinical Practice Guideline for AKI), and since urine volume data cannot be used in the experiment, the AKI stage is serum creatinine (Cr ) was defined on a numerical basis. At the top of FIG. 2, the structure of the first prediction model for predicting the possibility of AKI occurrence is shown. As shown, the first predictive model calculates the possibility of AKI occurring in a patient within a predetermined period from the present time based on a plurality of sequential input data (patient-related information from day 1 to day n) and outputs as a result It has a many-to-one input/output structure.

RNN (recurrent neural networks) machine learning algorithm determines the current output by reflecting the previous state information generated based on the previous input data for sequential input data (that is, the current output is the previous state It is a very useful model for processing sequential data according to the flow of time, such as data by date of inpatients) and data by date of inpatients. The first predictive model calculates the possibility of acute renal injury occurring in a patient within a predetermined period from the present time based on sequentially input patient-related information from day 1 to day n, and outputs the result as a result.

Specifically, the output result of the model is calculated using patient status information up to day n-1 (ie, previous status result value) and patient-related information input on day n (ie, current input value). Here, the condition information of the patient on day m, which is an arbitrary date, is obtained using the patient's condition information up to day m-1 and the patient-related information input on day m.

For example, the first prediction model receives information related to a patient on day 1 of hospitalization (including dynamic feature variables measured on day 1 and static feature variables that the patient already had before hospitalization) on day 1, but since there is no previous state information, day 1 Calculate the likelihood of AKI occurring in a patient within 7 days (an arbitrarily set period) using only patient-related information. On the second day, the probability of AKI occurring in the patient within 7 days is calculated using the information related to the patient on the second day of hospitalization and the patient's condition information obtained on the first day (a different concept from the result value). Similarly, on day n, the probability of AKI occurring within 7 days is calculated using the patient-related information on day n of hospitalization and the patient's condition information obtained on day n-1.

Using such an RNN-based prediction model, it is possible to distinguish patients with a high risk of AKI from those with a low risk. According to an embodiment, the first predictive model estimates the probability of acute renal impairment in a patient and classifies a patient expected to have acute renal impairment with a high possibility of acute renal impairment and a patient predicted to not have acute renal impairment with a low probability. As will be described later, the second prediction model predicts the serum creatinine level for each time interval based on the acute renal injury prediction result in the first prediction model.

As shown in FIG. 2 , the first prediction model may include additional components for optimizing input/output of data, such as a batch norm layer and a linear layer. In addition, a cross-entropy loss function and an Adam optimizer can be used to train the model, and a class weight parameter can be applied to the loss function to deal with class imbalance. Additionally, drop-out and early stopping approaches can be utilized to prevent overfitting. Since these components or techniques are commonly used in RNN machine learning models, detailed descriptions will be omitted.

Referring back to FIG. 1 , the second predictive model 300 predicts a change in the patient's serum creatinine level by using the prediction result of the first predictive model 200 . Like the first predictive model 200, the second predictive model 300 also uses an RNN machine learning algorithm, and outputs the creatinine level after a certain time from each date as a result based on sequentially input patient-related information It has a many-to-many input/output structure.

Specifically, the second predictive model 300 calculates the serum creatinine level within a predetermined time interval from day m using patient-related information input on any day m and state information of the patient up to day m-1. do. Similar to the first predictive model 200, the patient's condition information at a given point in time is obtained using the patient's condition information one day before that point in time and the patient-related information input at that point in time.

For example, the second predictive model receives information related to a patient on day 1 of hospitalization (including dynamic feature variables measured on day 1 and static feature variables that the patient already had before hospitalization) on day 1, but since there is no previous state information, day 1 Using only patient-related information, serum creatinine levels after 24 hours, 48 hours, and 72 hours (arbitrarily set time intervals) from the present time are respectively predicted and output. On the 2nd day, the serum creatinine levels 24 hours, 48 hours, and 72 hours after the current point are predicted using the patient-related information on the 2nd day of hospitalization and the patient's condition information obtained on the 1st day (a different concept from the result value). print out

According to an embodiment, a two-step hierarchical RNN prediction model may be applied to improve the accuracy of the second prediction model. That is, based on the results of the first prediction model, different types of second prediction models may be applied. For example, serum creatinine levels may be predicted for each time interval for a patient predicted to have acute renal injury and a patient not expected to have acute renal injury in the first prediction model. This structure is shown in FIG. 2 .

According to an embodiment of the present invention, the first predictive model and the second predictive model determine hyperparameters of the model through multiple cross-validation using a training data set, are internally verified through an independent test set of the same institution, and other It was externally validated using the institution's independent test data set. External validation is a procedure for generalizing experimental results to other subjects or situations other than the current experimental conditions, and external validity measures the extent to which a learned model can be generally applied to other subjects. it means.

3 is a flowchart illustrating a process of classifying a study group for learning and external verification of a predictive model according to an embodiment. Among the data of 182,976 patients who were hospitalized for more than a week at SNUBH during the period from January 2013 to December 2017, considering the exclusion criteria, 69,081 patients were finally used for training the machine learning model, and 7,675 patients were used for internal validation. has been used Among the data of 299,491 patients who were hospitalized for more than one week at SNUH during the period from January 2013 to December 2017, the data of 72,352 patients considering the exclusion criteria were used for external verification. Here, SNUBH and SNUH are independent institutions.

On the other hand, the performance of AKI prediction systems is improving due to the development of artificial intelligence technology, but due to the black box characteristics of the neural network model, it does not directly help field medical staff in making clinical decisions. This is because machine learning-based models process a large amount of input data at once through complex multiple hidden layers and weight parameters, making it difficult for users to interpret what criteria and methods the results were derived from. This makes it difficult for users to fully trust the results of the predictive model, and makes it difficult for medical staff to decide what actions to take based on a simple list of numbers.

Therefore, to apply the predictive results in real medical practice, clinicians need to understand how risk assessments of predictive models are derived. Accordingly, an embodiment of the present invention provides a framework for interpreting the results of a predictive model and supporting clinical decision making through intuitive analysis.

Referring back to FIG. 1 , the plot generation unit 400 determines the effect of each characteristic variable on the possibility of occurrence of acute kidney injury predicted by the first predictive model for at least one characteristic variable included in the patient-related information. It is configured to generate at least one plot for visual presentation.

According to one embodiment, the plots are Partial Dependence Plots (PDP), Individual Conditional Expectation Plots (ICEP), Accumulated local effects plots (ALEP), Variable Importance Plots. (Feature Importance Plots) may be included. These correspond to representative model-agnostic interpretation methods. Through PDP and ALE, the marginal effect or conditional effect of each feature on the prediction output can be expressed, and the variable importance plot Through , the average importance of each variable can be compared, and through ICEP, the relationship between each feature and output can be intuitively expressed for each instance.

Figures 4a to 4y show partial dependence plots (PDPs) and individual conditional expectation plots (ICEPs) for some feature variables presented to interpret the behavior of the predictive model. The lines in each graph represent the average value of the probability of acute renal impairment in hospitalized patients for the selected characteristic variable and the individual patient's probability of acute renal impairment for the selected characteristic variable. These plots can visually represent the effect of the selected feature variables on the AKI occurrence probability and the actual change in the AKI occurrence probability according to the level of each feature variable.

5A to 5O are cumulative local effect plots (ALEPs) that can confirm the influence of specific variables presented to interpret the behavior of the predictive model. The cumulative local effect plot minimizes the effect on the correlation between variables, allowing a more accurate description than the partial dependence plot (PDP). In addition, the cumulative local effect plot is drawn separately according to the time difference from the forecasting point to show the change in predictive power according to the date.

6 is a variable importance plot showing the average importance of each variable in the predictive model. This is calculated through the average Shapley value, and the higher the value, the greater the influence on acute kidney injury.

According to one embodiment, at least one plot generated by the plot generation unit 400 may include a graph representing a change in serum creatinine level per hour according to administration and discontinuation of a nephrotoxic drug.

7 is a graph showing actual changes in serum creatinine levels per hour and predicted results according to administration and discontinuation of some nephrotoxic drugs. 7 shows four cases in which NSAIDs are used as nephrotoxic drugs, respectively.

7, the green line represents the actual creatinine level, the blue line represents the predicted value of the creatinine level during maintenance of the nephrotoxic drug, and the orange line represents the predicted value of the creatinine level after discontinuation of each nephrotoxic drug. When such a plot is provided, it can help clinicians make decisions (i.e., clinical decisions) about actions or interventions to prevent the occurrence of AKI.

8 is a flowchart illustrating each step of a clinical decision support method based on an RNN machine learning model according to an embodiment. Part or all of each step may be implemented as a computer program and executed by a computer processor.

Referring to FIG. 8 , a step of sequentially receiving patient-related information from day 1 to day n (S100) is performed. Each patient-related information includes at least one feature variable, and each feature variable may be classified as a static variable that does not change during the patient's hospitalization period or a dynamic variable that changes during the patient's hospitalization period. For example, static variables refer to variables that hardly change at least during the period the patient is hospitalized, such as the patient's gender, age, BMI (calculated based on the patient's height and weight), drug administration history before hospitalization, and comorbidities. , dynamic variables include drug administration during hospitalization, in-hospital test results (blood creatinine level, leukocytes, hemoglobin, platelets, albumin, sodium, potassium, calcium, glucose, carbon dioxide, urea levels, etc.), vital signs (blood pressure, pulse, temperature, etc.) ) is a variable that changes continuously during the patient's hospitalization and is measured at least once a day.

Next, a step (S200) of predicting the possibility of occurrence of acute kidney injury (AKI) of the patient based on the input patient-related information using the first prediction model using the RNN machine learning algorithm is performed, followed by RNN machine learning. Using a second predictive model using an algorithm, based on the predicted result of the first predictive model, a change in serum creatinine level of the patient is predicted (S300).

The first prediction model and the second prediction model are as described above with reference to FIGS. 1 and 2 . The first predictive model calculates the possibility of AKI occurring in a patient within a predetermined period from the present time based on a plurality of sequential input data (patient-related information from day 1 to day n) and outputs the many-to-one result as a result. to-one) input/output structure, and the second predictive model has a many-to-many input/output structure that outputs the creatinine level after a certain time from each date as a result based on sequentially input patient-related information. have Both predictive models use RNN machine learning algorithms, and the current output can be determined by reflecting previous state information based on previous input data for sequential input data.

Then, with respect to at least one feature variable included in the patient-related information, at least one plot for visually representing the effect of each feature variable on the possibility of occurrence of acute kidney injury predicted by the first predictive model is generated. A step (S400) is performed.

As described above, the plot may include a partial dependence plot (PDP), an individual conditional expectation plot (ICEP), a cumulative local effect plot (ALEP), and a variable importance plot, through which the user can determine the result of the predictive model. It enables interpretation and provides a framework to support clinical decision-making through intuitive analysis. According to another embodiment, the plot may include a graph showing changes in serum creatinine levels per hour according to administration and discontinuation of various nephrotoxic drugs.

The clinical decision support method based on the RNN machine learning model according to these embodiments may be implemented as an application or implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like.

Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes such as those produced by a compiler. The hardware device may be configured to act as one or more software modules to perform processing according to the present invention and vice versa.

Experimental example

This experimental example was performed by applying the patient data of the training group and the patient data of the test group introduced with reference to FIG. 3 . After considering various exclusion criteria, 69,081 patients were finally included in the training cohort, 7,675 patients were included in the internal test cohort, and 72,353 patients were included in the external test cohort. The characteristics of the study population are shown in the table below.

Variables SNUBH training set (n = 69,081) SNUBH validation set (n = 7,675) SNUH validation set (n = 72,352) P value Age, years 59.8 ± 16.5 ^a 59.6 ± 16.6 57.1±16.0 <.001 male sex 36732 (53.2%) 4114 (53.6%) 37405 (51.7%) <.001 Body mass index, kg/m ² 23.8 ± 3.5 23.8 ± 3.5 23.2 ± 3.5 <.001 Stay length, days 8.7 ± 14.2 8.5 ± 10.4 6.9 ± 11.7 <.001 ICU admission 4478 (6.5%) 475 (6.2%) 2340 (3.2%) <.001 Charlson comorbidity index 1.0 ± 1.4 1.1±1.4 1.1±1.4 <.001 Specific preexisting comorbidities Hypertension 9455 (13.7%) 995 (13.0%) 6624 (9.2%) <.001 Diabetes mellitus 7187 (10.4%) 769 (10.0%) 6513 (9.0%) <.001 Ischemic heart disease 7452 (10.8%) 790 (10.3%) 6289 (8.7%) <.001 heart failure 1500 (2.2%) 169 (2.2%) 793 (1.1%) <.001 Baseline eGFR, ml/min/1.73 m ² 94.9 ± 38.9 94.6 ± 37.4 89.4 ± 21.9 <.001 No CKD or CKD 1 or 2 (≥60) 60201 (87.1%) 6683 (87.1%) 65310 (90.3%) <.001 CKD G3a (45-59) 4839 (7.0%) 550 (7.2%) 4188 (5.8%) CKD G3b (30-44) 2629 (3.8%) 279 (3.6%) 1940 (2.7%) CKD G4 (15-29) 1412 (2.0%) 163 (2.1%) 914 (1.3%) Hemoglobin, g/dl 13.0 ± 2.1 13.0 ± 2.1 13.0±2.0 .54 Albumin, g/dl 4.0 ± 0.6 4.0 ± 0.6 4.1 ± 0.5 <.001 Bilirubin, mg/dl 0.9 ± 1.5 0.9 ± 1.6 0.9 ± 1.5 .032 Calcium, mg/dl 8.8 ± 0.6 8.8 ± 0.6 9.1 ± 0.6 <.001 Glucose, mg/dl 126.4 ± 55.5 125.8 ± 54.1 121.1 ± 48.9 <.001 Sodium, mEq/L 138.9 ± 3.8 138.8 ± 3.8 139.8 ± 3.3 <.001 Potassium, mEq/L 4.1 ± 0.4 4.2 ± 0.5 4.2 ± 0.4 <.001 Chloride, mEq/L 102.5 ± 4.0 102.5 ± 4.0 103.5 ± 3.5 <.001 AST, IU/L 47.3 ± 245.3 46.7 ± 213.2 32.4 ± 108.7 <.001 ALT, IU/L 42.6 ± 192.1 42.0 ± 154.1 30.5 ± 100.4 <.001 Total CO ₂ , mEq/L 24.7 ± 3.3 24.7 ± 3.3 25.6 ± 3.6 <.001 Platelet, 10 ³ /μl 232.9 ± 85.0 232.3 ± 86.7 235.8 ± 479.1 <.001 White blood cell count, /mm2 8.2 ± 7.1 8.3 ± 8.3 7.3±5.5 .105 Medication use RAS blockers 5494 (8.0%) 610 (7.9%) 6039 (8.3%) .007 Diuretics 3437 (5.0%) 381 (5.0%) 3723 (5.1%) .147 NSAIDs 16974 (24.6%) 1874 (24.4%) 13290 (18.4%) <.001 Systolic blood pressure, mmHg 128.7 ± 16.5 128.6 ± 16.5 124.8 ± 15.8 <.001 Diastolic blood pressure, mmHg 74.1 ± 10.8 74.0 ± 10.7 76.2 ± 10.7 <.001 Heart rate, per minute 78.6 ± 14.4 78.7 ± 14.3 77.7 ± 14.2 <.001 Body temperature, ℃ 36.7 ± 0.5 36.7 ± 0.5 36.5 ± 0.4 <.001

According to the table above, the average age of patients in the training cohort was 59.8 years, which was higher than the average age of patients in the external testing cohort, 57.1 years, and the average baseline eGFR was higher in the training cohort than in the external testing cohort. More patients in the training cohort had hypertension or diabetes, but the average comorbidity index was higher in the external testing cohort. The cumulative incidence of AKI (including both mild stage 1 and severe stage 2-3) during the 2 weeks after admission was 5.91% in the training cohort and 3.63% in the external testing cohort. The cumulative incidence of severe AKI (stage 2 or 3) was 1.58% in the training cohort and 1.11% in the external testing cohort.

In this experimental example, the performance of the first predictive model for AKI (including both mild stage 1 and severe stage 2-3) was AUC 0.882 in the internal test cohort and AUC 0.838 in the external test cohort, but the external test cohort When the model was updated using 10% of the AUC, the result was improved to 0.864. The performance of the first predictive model for AKI (stage 2 or 3) was AUC 0.927 in the internal test cohort and AUC 0.901 in the external test cohort, but when the model was updated using 10% of the external test cohort, it was AUC 0.914. showed improved results.

Based on the results of the first model, the second predictive model was verified, and the performance evaluation results of the second predictive model are shown in the table below.

Subgroup Validation method MSE at different time points 24h 48h 72h Positive prediction Internal validation 0.04 0.04 0.06 External validation 0.06 0.06 0.09 External validation (refitting) 0.05 0.06 0.08 Negative prediction Internal validation 0.03 0.04 0.04 External validation 0.06 0.06 0.08 External validation (refitting) 0.03 0.05 0.05

The results of the above table are verified by the second predictive model tracking the creatinine (Cr) level in the positive and negative predictive groups based on the predicted results of the first predictive model on the possibility of AKI occurrence. The measured creatinine levels were significantly different in each of the positive and negative predictive groups. The mean square error (MSE) values in the predictive models of AKI (all stages) ranged from 0.03 to 0.06 for internal validation and 0.05 to 0.09 for external validation. When the model was updated using 10% of the data from the external test cohort, the MSE of the model improved to 0.03 to 0.08. MSE values at different prediction points (24, 48, and 72 hours) were generally similar.

According to the embodiments of the clinical decision support system and method described above, it is possible to predict the possibility of acute kidney injury (AKI) in hospitalized patients and changes in serum creatinine levels in real time using an RNN machine learning model, and the predictive model By providing plots that visually represent the relationship between feature variables and outcomes that affect the outcome of the outcome, it is possible to provide clinicians with insight into how variables affect predicted outcomes. In addition, the predictive model of the embodiment is externally verified using an external data set independent of the data set used for learning, and thus has high predictive reliability and versatility that can be used for other patient groups.

Although the above has been described with reference to embodiments, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. You will understand.

Claims (15)

an input unit that sequentially receives patient-related information from day 1 to day n;
A first prediction model using an RNN machine learning algorithm and predicting the possibility of acute kidney injury (AKI) in a patient based on input patient-related information;
a second prediction model using an RNN machine learning algorithm and predicting a change in serum creatinine level of a patient based on a prediction result of the first prediction model; and
A plot for generating at least one plot for visually showing the effect of each characteristic variable on the possibility of occurrence of acute renal injury predicted by the first predictive model for at least one feature variable included in the patient-related information Including a generator,
The first predictive model,
Based on the entered patient-related information, the probability of AKI occurring in the patient within a set period from the present time is calculated and output as a result. AKI-free prospective patients are classified,
The second predictive model,
Based on the AKI prediction result of the first predictive model, the serum creatinine level within a set period from the present time is calculated and output as a result, but different types of predictive models for patients expected to develop AKI and patients without AKI, namely, A clinical decision support system using an RNN machine learning model, characterized in that the serum creatinine level is calculated by applying a two-step hierarchical RNN prediction model.
According to claim 1,
The first predictive model calculates the possibility of acute renal injury occurring in a patient within a predetermined period from the present time based on sequentially input patient-related information from day 1 to day n, and outputs the result as a result,
The result is a clinical decision support system using an RNN machine learning model, characterized in that calculated using the patient's condition information up to day n-1 and patient-related information input on day n.
According to claim 2,
Clinical decision-making support using an RNN machine learning model, characterized in that the patient's condition information on any day m is obtained using the patient's condition information up to day m-1 and the patient-related information input on day m system.
According to claim 1,
The second predictive model calculates the serum creatinine level within a predetermined time interval from each date using the patient-related information input on any day m and the condition information of the patient up to day m-1, and outputs the result as ,
The patient's condition information up to the m-1 day is obtained using the patient's condition information up to the m-2 day and the patient-related information input on the m-1 day, using an RNN machine learning model. Clinical decision support system.
According to claim 2,
The first predictive model is an RNN machine learning model, characterized in that outputting a predicted probability indicating the possibility of acute renal injury in the patient or classifying a patient who is expected to have acute renal injury based on the predicted probability Clinical decision support system using .
According to claim 5,
The second prediction model predicts the serum creatinine level for each time interval based on the output value of the first prediction model, clinical decision support system using an RNN machine learning model.
According to claim 1,
The first prediction model and the second prediction model are learned using data of patients admitted to a specific institution and externally verified using data of patients admitted to another institution independent of the institution. A clinical decision support system using RNN machine learning model.
According to claim 1,
The at least one characteristic variable included in the patient-related information includes static variables that do not change during the hospitalization period of the patient or dynamic variables that change during the hospitalization period of the patient,
The static variables include at least one of the patient's gender, age, BMI level, drug administration history prior to hospitalization, and comorbidities,
The dynamic variables include at least one of drug administration during hospitalization, in-hospital test results, and vital signs, and are updated at least once a day, clinical decision support system using an RNN machine learning model.
According to claim 1,
At least one of the plots generated by the plot generation unit may include partial dependence plots, individual conditional expectation plots, accumulated local effects plots, or sharp value plots. A clinical decision support system using an RNN machine learning model, characterized in that it includes a variable importance plot using ).
According to claim 9,
The partial dependence plot (PDP) and the cumulative local effect plot (ALEP) are a clinical decision support system using an RNN machine learning model, characterized in that they represent the average value of the probability of acute renal injury in hospitalized patients for the selected feature variable. .
According to claim 9,
The variable importance plot using the Shapley value is a clinical decision support system using an RNN machine learning model, characterized in that it indicates in order the importance of variables contributing to the occurrence of renal damage in individual patients and groups.
According to claim 9,
The individual conditional expectation plot (ICEP) is a clinical decision support system using an RNN machine learning model, characterized in that it indicates the possibility of acute renal injury of the individual patient for the selected feature variable.
According to claim 1,
At least one of the plots generated by the plot generation unit includes a graph representing a change in serum creatinine level per hour according to administration and discontinuation of a nephrotoxic drug, characterized in that, clinical decision making using an RNN machine learning model support system.
sequentially receiving patient-related information from day 1 to day n;
predicting a possibility of acute kidney injury (AKI) of the patient based on the input patient-related information using a first prediction model using an RNN machine learning algorithm;
predicting a change in serum creatinine level of a patient based on a prediction result of the first prediction model by using a second prediction model using an RNN machine learning algorithm; and
Generating at least one plot for visually representing the effect of each characteristic variable on the possibility of occurrence of acute renal injury predicted by the first predictive model with respect to at least one characteristic variable included in the patient-related information. Including,
Predicting the possibility of acute kidney injury (AKI) of the patient,
Based on the entered patient-related information, the probability of AKI occurring in the patient within a set period from the present time is calculated and output as a result. A step of classifying expected patients without AKI,
The step of predicting the change in the serum creatinine level of the patient,
Based on the AKI prediction result of the first predictive model, the serum creatinine level within a set period from the present time is calculated and output as a result, but different types of predictive models for patients expected to develop AKI and patients without AKI, namely, A clinical decision support method using an RNN machine learning model, characterized in that the step of calculating the serum creatinine level by applying a two-step hierarchical RNN prediction model.
A computer program stored in a computer-readable recording medium for implementing the clinical decision support method using the RNN machine learning model according to claim 14.

Images