CN114023440A

CN114023440A - Model and device capable of explaining layered old people MODS early death risk assessment and establishing method thereof

Info

Publication number: CN114023440A
Application number: CN202111312537.8A
Authority: CN
Inventors: 刘晓莉; 张政波; 周飞虎; 虎磐; 毛智; 刘超
Original assignee: Chinese PLA General Hospital
Current assignee: Chinese PLA General Hospital
Priority date: 2021-11-08
Filing date: 2021-11-08
Publication date: 2022-02-08

Abstract

The application discloses an evaluation model, a device and an establishment method for early death risk of hierarchical old MODS (moderate resolution imaging System), wherein an evaluation module is based on an XGboost model fused with a SHAP (short Range analysis) method, and is divided into a low-age old evaluation submodule aiming at MODS patients with the ages of more than or equal to 65 years and less than 80 years and a high-age old evaluation submodule aiming at MODS patients with the ages of more than or equal to 80 years; an assessment module to conduct a mortality risk assessment based on input features corresponding to at least some of the plurality of features; and the evaluation module ranks the importance of the input features to the evaluation result and calculates the contribution of each input feature to the evaluation result as the contribution degree of the risk factor. The model and the device for evaluating the early death risk of the layered old MODS are helpful for doctors to obtain more accurate evaluation on the disease emergency and the risk degree of patients, and are suitable for medical institutions in more different regions and centers.

Description

Model and device capable of explaining layered old people MODS early death risk assessment and establishing method thereof

Technical Field

The invention relates to machine learning, in particular to an early-stage death risk assessment model and device for senile multi-organ failure based on interpretable machine learning models for two types of elderly people groups, and an establishment method thereof.

Background

The elderly population is increasing exponentially, the cost of medical services is increasing year by year, and the intensive care unit ICU monitoring pressure is increasing. Due to hypofunction and the presence of various chronic diseases in elderly patients, the fatality rate during ICU is much higher compared to adults. Extensive studies with the coronavirus COVID-19 indicate that advanced age is a high risk factor for death, and patients are eventually lost life due to the development of multiple organ failure MODS. Numerous studies have shown that the clinical characteristics of elderly patients vary greatly with age. They are mostly shown to be debilitating (i.e. a gradual decline in various bodily functions, reduced reserves during stress), diminished cognitive abilities and immune aging compared to adult patients. Some studies at present indicate that the treatment effect of younger patients is poor and the clinical common disease severity scores such as sequential organ failure score SOFA and acute physiological and chronic health score APACHE-II have larger deviation from the real disease severity of the elderly patients. The main reasons include: the variation of the abnormal range of parameters, the non-consideration of factors characterizing the characteristics of elderly patients and the linear assessment of complex diseases, and the presence of certain pathological and physiological changes in the elderly patients of low age (65-80 years) and those of advanced age (over 80 years). Thus, the use of the same evaluation criteria may result in a deviation of the evaluation that results in neglecting the seriousness of the disease.

In recent years, the development of more accurate early disease risk assessment models/scores is greatly promoted by the research of disease prediction models based on electronic health files, but more research is limited by a limited sample set (such as a single center, a small sample and from one country), so that the universality and the robustness of the models cannot be guaranteed. Although some prediction models based on deep learning achieve good prediction performance, the black box principle of the prediction models is limited, so that the prediction models cannot be understood and trusted by doctors, and the popularization and the use of the prediction models are influenced to a certain extent.

Disclosure of Invention

In view of the above problems, the present application is directed to elderly patients with multiple organ failure of low age and advanced age, respectively developing a prediction model capable of early evaluating the death risk during ICU hospitalization based on multi-center data sets from hospitals in different countries and multiple regions, and synchronously presenting reasoning analysis reasons of the model for facilitating understanding of doctors so as to really help doctors to perform assisted diagnosis and treatment.

In one aspect, the present application provides an interpretable hierarchical model for evaluating early mortality risk of old people MODS, which includes an evaluation module; the evaluation module is based on an XGboost model fused with a SHAP method, and comprises a plurality of characteristics;

the evaluation module is divided into a low-age old evaluation submodule for MODS patients with the ages of 65 years or more and less than 80 years old and a high-age old evaluation submodule for MODS patients with the ages of 80 years or more;

for the young age assessment sub-module, the first 20 features of the plurality of features from high to low in importance are: GCS, charsen co-morbid index, whether mechanical ventilation is performed, respiratory rate, urea nitrogen, shock index, heart rate, BMI, body temperature nadir, total urine volume during the day, body temperature nadir, age, rate of norepinephrine use, blood oxygen saturation, blood potassium, blood glucose, hematocrit, lymphocytes, carbon dioxide partial pressure, and creatinine;

for the senior assessment sub-module, the first 20 features of the plurality of features from high to low in importance are: whether mechanical ventilation, GCS, respiratory rate, charsen co-morbid index, blood oxygen saturation, heart rate, body temperature nadir, shock index, total urine volume during the day, body temperature nadir, creatinine, aspartate aminotransferase, carbon dioxide partial pressure, Code status, albumin, oxygen partial pressure, BMI, lactate, rate of use of norepinephrine, and lymphocytes are performed;

the assessment module performs a mortality risk assessment based on input features corresponding to at least some of the plurality of features; and the evaluation module ranks the importance of the input features to the evaluation result and calculates the contribution of each input feature to the evaluation result as the contribution degree of the risk factor.

Preferably, the plurality of features are from class 6 data; the 6 types of data are respectively:

personal information, comprising: age, gender, BMI index;

frailty and nerves, which include: GCS, Charlson comorbidity index, Code status;

vital signs, which include: heart rate, respiratory rate, mean arterial pressure, systolic pressure, diastolic pressure, central venous pressure, body temperature, blood oxygen saturation, shock index;

a liquid output comprising: the urine volume;

a laboratory examination comprising: oxygen partial pressure, inhaled oxygen concentration, carbon dioxide partial pressure, oxygenation index, albumin, alkaline phosphatase, alanine transaminase, aspartate transaminase, base excess, prothrombin time, partial thromboplastin time, bicarbonate, bilirubin, brain natriuretic peptide, blood urea nitrogen, creatinine, chloride, fibrinogen, glucose, hematocrit, hemoglobin, international normalized ratio, lactate, lymphocytes, magnesium ions, neutrophils, platelets, potassium ions, Ph, sodium ions, troponin, and leukocytes;

a treatment, comprising: whether mechanical ventilation is performed, whether continuous renal replacement therapy is performed, the rate of norepinephrine use, whether dobutamine is used, whether dopamine is used, and whether epinephrine is used.

Preferably, for the elderly assessment sub-module, the input features are the first 10, 15, 20 or 79 features of the plurality of features from high to low in importance;

for the senior assessment sub-module, the input features are the first 10, 15, 20 or 79 features of the plurality of features from high to low in importance.

Preferably, a data processing module is included;

the data processing module obtains input characteristics from the data of the old patient on the first day in the ICU through processing so as to input the input characteristics into the evaluation module.

On the other hand, the application provides an interpretable hierarchical early-stage MODS death risk assessment device, which comprises a computing unit, a processing unit and a processing unit, wherein the computing unit is used for executing the interpretable hierarchical early-stage MODS death risk assessment model; the model adopts a SHAP method fused with the model to obtain the assessment of the contribution degree of the risk factors of the individual patients; the first color is used for representing that the factor is in the abnormal state at present and has harmful influence on the outcome of the patient, the second color is used for representing that the factor is in the normal state at present and has no harmful influence on the outcome of the patient, and the influence degree on the outcome is larger when the SHAP value is larger. The computing unit can be a CPU, a singlechip, a computer, intelligent equipment and the like.

In another aspect, the present application provides a method for establishing an interpretable model for evaluating an early mortality risk of a hierarchical MODS for elderly people, comprising:

constructing a data set, processing data, constructing a model and evaluating the model;

in the data set construction, obtaining research data sets of low-age and old aged multi-organ failure patients in a plurality of intensive care data sets according to the sequential organ failure scores; determining study variables including personal information, frailty and nerves, vital signs, urine volume, laboratory examinations, and treatments;

in data processing, cleaning, integrating, sampling and interpolating data from the plurality of data sets, and further sorting the data to obtain a plurality of statistical characteristics;

in the model construction and evaluation, after model training, parameter tuning and internal verification are carried out based on the multi-center data set, the performance of the model is evaluated by adopting 7 evaluation indexes and 1 functional index, and the performance of the model is evaluated by adopting the modes of internal verification, external verification, time sequence verification and subgroup analysis.

Preferably, in model construction and evaluation, the evaluation model is trained, evaluated and optimized for MODS patients aged 65 years or more and less than 80 years old and MODS patients aged 80 years or more, respectively.

Preferably, the plurality of intensive care data sets comprises: MIMIC-III, eICU-CRD, AmsterdamUMCdb, and MIMIC-IV; the study population was screened for patients based on the SOFA score and the established inclusion protocol to obtain data sets for patients with MODS aged 65 years or older but less than 80 years old and patients with MODS aged 80 years or older, respectively.

Preferably, in training the model, data from MIMIC-III and elcu-CRD are fused as one large sample, multi-center training set, with 80% of the patient data being used for model training and with cross-validation to adjust the hyper-parameters of the predictive model, and the remaining 20% of the patient data being used for internal validation of model performance;

in the model performance evaluation, the 7 evaluation indexes are: AUROC, specificity, sensitivity, accuracy, F1 value, accuracy, AUPRC; the 1 function index is an interpretable function index.

Preferably, the internal verification is evaluated using 20% of the patient data from MIMIC-III and eICU-CRD in the data set consistent with the training set; external validation is carried out by adopting all patient data of AmsterdamUMCdb distributed in a inconsistent way with training data for evaluation; the timing verification uses all patient data of the MIMIC-IV updated over the training set time span for evaluation; subgroup analysis verified the groups as caucasian, african and hispanic, respectively; so as to comprehensively evaluate whether bias exists or not, and the universality and the robustness of the model.

According to the model and the device for evaluating the early death risk of the hierarchical old MODS, through large samples and multi-center training, the performance is consistently superior to other compared models and scores, the model and the score are verified and evaluated in multiple modes, the model and the device show good universality and robustness, the risk probability that a patient has bad outcomes is provided, and meanwhile, the analysis reason can be obtained, so that the device is beneficial to doctors to obtain more accurate evaluation on the disease emergency and the risk degree of the patient, and is beneficial to early taking action to treat the patient with potential benefits, and the device is suitable for being used by medical institutions in more different regions and centers.

Drawings

FIG. 1 is a flowchart illustrating an implementation of a method for establishing a hierarchical model for early mortality risk assessment in old age MODS according to the present application;

FIG. 2.4 study populations of data sets and corresponding proportions;

FIG. 3. inclusion and exclusion criteria for study populations in MIMIC-III database;

FIG. 4. inclusion exclusion criteria for study population in eICU-CRD database;

FIG. 5, Ams-inclusion exclusion criteria for study populations in the UMC database;

FIG. 6 shows the inclusion and exclusion criteria of the study population in MIMIC-IV (2014-2019) database;

FIG. 7. predictive model vs. baseline model and clinical score in Young-old population (65-80 years old) (internal validation);

FIG. 8 comparison of predictive models to baseline models and clinical scores in the Old-Old population (over 80 years Old) (internal validation);

FIG. 9 comparison of predictive models to baseline models and clinical scores in Young-old population (external validation, Ams-UMC);

FIG. 10 comparison of predictive models to baseline models and clinical scores in the Old-Old population (external validation, Ams-UMC);

FIG. 11 comparison of predictive models to baseline models and clinical scores in Young-old population (external validation, MIMIC-IV);

FIG. 12 comparison of predictive models to baseline models and clinical scores in the Old-Old population (external validation, MIMIC-IV);

FIG. 13. calibration curves for the predictive model in Young-Old and Old-Old populations; (a) young-old: cohort 1-2 for internal authentication, Cohort 2-2 for external authentication at Ams-UMC, and Cohort 3-2 for external authentication at MIMIC-IV; (b) Old-Old: cohort 1-2 for internal authentication, Cohort 2-2 for external authentication at Ams-UMC, and Cohort 3-2 for external authentication at MIMIC-IV;

FIG. 14. analysis of the ethnic subgroups of the Young-Old and Old-Old populations in the external validation of the predictive model (a) Young-Old; (b) Old-Old;

FIG. 15. Top20 significance signature ranking for Young-old prediction model;

FIG. 16. Top20 significance signature ranking for the Old-Old prediction model;

FIG. 17. reasoning-interpretable analytic presentation of the Young-old predictive model; (a) non-viable patients; (b) a patient is alive;

FIG. 18. an inferential interpretable analytic presentation of the Old-Old predictive model; (a) non-viable patients; (b) a patient is alive;

fig. 19 is an ICU early risk assessment device for elderly organ failure patients based on interpretable machine learning.

Detailed Description

The method is based on electronic health files with massive and abundant dimensional information of patients, adopts a machine learning model fused with an interpretable method, respectively develops multi-center-training and verified multi-organ function failure death risk prediction models of the elderly with universality, robustness and interpretability for the elderly (65-80 years old) and the elderly (80 years old and above), obtains risk factors related to hospital bad outcomes and reasoning processes of the models, and finally respectively packages the models into devices capable of automatically and early evaluating the death risk of the elderly based on the respective models. The development process is as follows: (1) constructing a large sample multi-center data set which can support the development of a model with excellent evaluation performance, and constructing Research data sets aiming at the old of the low age and the old of the high age respectively according to clinical diagnosis standards and clinical and literature knowledge based on a single-center open-source Intensive Care Database Medical Information Mart for Intensive Care III (MIMIC-III) and a multi-center open-source Intensive Care Database eICI collectivity Research Database (eICI-CRD) with large time span; (2) cleaning and sorting data, including data combination, data sampling, abnormal value removal, interpolation and statistical characteristic construction, and respectively constructing 6 types of data of personal information, nerve and weak body function information, vital signs, urine volume, laboratory examination and treatment according to the data acquisition and change characteristics; (3) respectively training and adjusting the model based on 4 machine learning models (integrated learning models XGboost, logistic regression LR, random forest RF and naive Bayes NB model), and evaluating the performance of the model through 7 evaluation indexes (AUROC under a subject working characteristic curve, specificity, sensitivity, accuracy, F1 value, accuracy, AUPR under an accurate-recall curve) and 1 functional index (interpretability) to obtain the model with the optimal performance. And comparing with 3 clinically common scores (acute physiological assessment score APSIII, systemic infection related organ failure score SOFA, simplified acute physiological score SAPS); (4) evaluating the universality and robustness of the prediction model by adopting internal verification, external verification (Holland intensive care data set AmsterdamUMCdb, Ams-UMC), time sequence verification (MIMIC-III update data set MIMIC-IV 2014-2019), a calibration curve, subgroup analysis (white, black and Hispanic), and inclusion of partial features (79-10); (5) risk assessment factors associated with the occurrence of inpatient adverse outcomes in patients with low-aged and elderly multiple organ failure were obtained based on the interpretable method, SHAP. And the process is packaged, so that the risk of adverse outcome of the aged multi-organ failure patients in the intensive care unit ICU can be automatically and early evaluated conveniently, and the risk is convenient for doctors to understand and operate. So as to help the doctor to more comprehensively and early realize the potential body state of the patient and provide help for the next decision treatment.

The method is based on the electronic health record, adopts an integrated learning method to evaluate the early death risk of the aged multi-organ failure patients, and specifically comprises the following steps:

step 1: data set building block

And acquiring the use right of MIMIC-III, eICU-CRD, Ams-UMC and MIMIC-IV databases, and acquiring MODS patient groups researched in 4 databases by utilizing the patient diagnosis and treatment information recorded in detail in the electronic health file. Elderly patients with MODS are identified according to clinical organ failure assessment criteria (systemic infection-associated organ failure score, SOFA). Patients with MODS were obtained according to the inclusion and exclusion criteria of figure 2. Further combining the clinician's experience (characteristics of elderly patients) with the patient information recorded in the database, features are determined which are subsequently used to develop a predictive model. The method comprises the following steps: demographics (3 dimensions total), neurological and frailty (3 dimensions total), vital signs (9 dimensions total), laboratory examinations (32 dimensions total), clinical (intervention: 6 dimensions total), and urine volume (1 dimension total). Finally, the study population and data were divided according to two age ranges (65-80 years and over 80 years). And the patients who died in the hospital were labeled as positive samples, the rest as negative samples.

Step 2: data processing module

The data from the 4 data sets are cleaned up and merged separately for the study population and study data set determined in step 1. The method comprises the steps of standardization, cleaning, abnormal value removing, sampling and interpolation of sparse and non-uniform format data. Wherein, the interpolation adopts the population median corresponding to the corresponding characteristics, if the deletion ratio is more than 30%, a flag characteristic flag is added to indicate whether a record (1, 0) exists or not; based on the structured data, model input features are constructed, namely statistical features (original values, maximum values, minimum values, mean values and sums) are constructed, so that 79 research features (3 personal information, 3 weak and nerve information, 12 vital signs, 1 urine volume, 54 laboratory examinations and 6 treatment information) are obtained.

And step 3: model construction and evaluation module

The method adopts an integrated learning XGboost model which integrates a SHApley Additive ex plants (SHAP) method to evaluate the death risk of MODS patients in two age stages. The two prediction models are respectively trained and optimized by fusing MIMIC-III and eICU-CRD (multi-center and large sample), and are internally verified and evaluated; then 7 evaluation indexes and 1 function index AUROC, specificity, sensitivity, accuracy, F1 value, accuracy, precision, AUPRC and interpretability are adopted for evaluation; further using Ams-UMC full data for external validation of the model; data from 2014 to 2019 of MIMIC-IV are used for timing verification of the model; we simultaneously incorporated 3 machine learning models (logistic regression LR, random forest RF and naive bayes NB model) and 3 commonly used clinical scores (acute physiological assessment score APSIII, systemic infection-related organ failure score SOFA, simplified acute physiological score SAPS) for comparison with the model selected in this application; meanwhile, risk factors and ranking related to death of the low-age elderly patients and the high-age elderly patients suffering from MODS are obtained based on the SHAP method; further obtained calibration curves for the model and an assessment of ethnic performance (white vs. african and hispanic) of clinical major interest; and the model was evaluated to reduce the change in the performance of the incorporated variables (79 to 10 features in total). And finally, packaging the fully evaluated model and the relevant modules of the data processing link to obtain a risk evaluation model and a risk evaluation device for two age groups.

The application provides an early death risk assessment model and device with interpretable function for the elderly patients with low age and the elderly patients with organ failure after entering ICU, which comprises the following steps: obtaining 3 individuals' information on the first day of patient stay in the ICU, assessing frailty and neurological function associated with aging 3, vital signs 11, liquid output urine volume 1, laboratory test 55 and treatment information 6; the variables with a plurality of accumulated data are obtained through the data processing module of the device and can be directly input into the model, and further through the calculation of the risk assessment module and the contribution ratio of the interpretable method to the visualization of the model assessment process, namely important risk factors to the outcome of the patient; finally, the risk of the early prediction of the elderly patients with poor outcome (death) and the explanation of model reasoning are obtained in two age groups.

The present invention will be described in detail with reference to fig. 1 to 19.

The electronic health record data based on high quality, large samples and rich dimensionality is developed and used for early evaluation and prediction of risks of poor outcome of two types of potential danger groups (low-age and high-age MODS elderly patients) in an ICU scene during hospitalization, and a robust, universal, understandable and credible and clinically touchable risk evaluation model is obtained through comprehensive evaluation indexes and external verification. The early assessment of disease severity of two age-stratified elderly patients with multiple organ failure is performed fully automatically, in combination with the characteristics of the elderly patients, to assist physicians in early intervention and treatment of patients at risk of deterioration. The invention utilizes rich information collected by the electronic health record, the three-dimensional data can represent the disease development track of the patient during hospitalization, and a prediction model with better grading performance compared with clinical use is obtained by mining the complex nonlinear correlation between the data and the target through a machine learning model. Meanwhile, as the data volume is very large and the data volume comes from a plurality of centers, a more universal model can be developed, which is an advantage that the traditional clinical random contrast research institute which carries out a great deal of energy and financial resources cannot have by linearly adding clinical scores. Through multi-center and external verification, subgroup analysis and the like, the model with the optimal performance and the data processing model are finally packaged, can be integrated into the conventional EHR information system, can automatically acquire analysis results and visual reasons, provides reference basis for treatment evaluation of doctors, and does not increase the workload of medical staff.

The process proposed in the present invention mainly comprises 3 models: (1) the data set construction module is used for acquiring research data sets of the low-age and old MODS elderly patients based on the 4 intensive care data sets and the inclusion and exclusion standards and research variables of the research population determined by the clinician; (2) a data processing module: and (2) cleaning, regulating, sampling and interpolating the data according to the 4 original research data sets of the two age groups obtained in the step (1). Further completing the construction of statistical characteristics according to data characteristics to obtain 6 types of research characteristic data; (3) and (3) inputting the data obtained in the step (2) into the selected machine learning model, completing the construction and parameter tuning of the model based on the selected model training set, and performing internal verification performance evaluation. And then, based on the selected external/time-series verification data set, the determined 7 evaluation indexes and 1 functional index and subgroup analysis, and the content of the included partial features, further evaluating the predictive performance of the model to obtain the generalizability, universality and robustness of the model. And then the model with the best performance and the data processing link are packaged, so that the doctor can be fully automatically helped to acquire the death risk and the risk factor ranking of the patient in the early stage, and the doctor can be assisted in disease diagnosis and treatment.

The electronic health record data set from multiple centers provided by the invention develops an ICU early death risk assessment method for the patients with low-age and old aged multiple organ failure, the prediction performance of the ICU early death risk assessment method is consistently better than that of a baseline model and clinical scores, and a more convenient and accurate assessment method can be provided for doctors to assess the patient's condition early. The risk assessment model is respectively constructed for the elderly patients with low age and the elderly patients for the first time, a multi-center and large-sample data set (36185 patients) is adopted for model training, and the performance shows good universality and robustness after multi-center and multi-country external verification (10595 patients); at the same time the method obtains a risk factor ranking associated with the risk of death for patients with lower and higher aged MODS, where the glasgow score, charlson comorbidity index, respiratory rate and whether mechanical ventilation is performed are critical for both age groups and are in the top 4 of the risk factor ranking. Wherein for the elderly patients, the Code status of the patients also has important assessment effect on the outcome of the patients; finally, the method fuses two age-layered models, parallel calculation is built in, and the risk of hospital adverse outcome (death) of the aged MODS patients can be automatically, conveniently and early evaluated.

The invention provides a method for interpretable evaluation and risk factor ranking of early death risks caused by multi-organ failure of old and young aged based on electronic health records. The specific implementation is shown in fig. 1, and comprises the following steps:

the data set construction module process in the invention is as follows:

the group involved in the study (patients who were admitted to ICU for the first time; ICU duration is 24 hours or more; measurement of heart rate, respiratory rate, mean arterial pressure, GCS, body temperature and oxygen saturation occurred within the first day of ICU) was further obtained by taking MIMIC-III, eICCU-CRD, Ams-UMC and MIMIC-IV (2014-2019) as a basal population, i.e., elderly patients (age 65 years) with at least two organ systems failing according to the SOFA score, and further according to the screening criteria shown in FIG. 2. Fig. 3-6 show a detailed screening procedure for patients in each data set. Thus, the middle aged and elderly MODS patients of MIMIC-III were 9396 (12.4% mortality) and 6338 aged and elderly MODS patients (18.4% mortality). The low aged geriatric MODS patients in eICU-CRD were 18287 (10.8% mortality) and 11211 elderly MODS patients (13.8% mortality). Ams-patients with intermediate aged senile MODS in UMC 1297 (12.6% mortality) and 608 patients with advanced senile MODS (21.4% mortality). MIMIC-IV middle aged geriatric MODS patients were 5517 (10.5% mortality) and 3173 elderly MODS patients (16.1% mortality). The data of MIMIC-III and eICU-CRD are fused for obtaining the model. FIG. 2 shows the size of the study sample set and the proportion of positive samples in the 3 combined study data sets, and the study populations from MIMIC-III and eICU-CRD, Ams-UMC, and MIMIC-IV are respectively called Cohort1, Cohort2, and Cohort 3. The patients of low and advanced age are denoted by ` CohortN-1 ` and ` CohortN-2 ` respectively. Table 1 is a baseline comparison of the population of 2 age-stratified patients from 3 study populations. Table 2 contains the study variables to BE included in the model determined by the physician, including 3 demographic (age, sex, BMI), 3 neurological and frailty indices (Glasgow score GCS, Charson's syndrome index, Code status), 9 vital signs (heart rate, respiratory rate, mean arterial pressure, systolic pressure, diastolic pressure, central venous pressure, body temperature, blood oxygen saturation, shock index), 32 laboratory tests (partial oxygen pressure (PaO2), inspired oxygen concentration (FiO2), partial carbon dioxide pressure (PaCO2), oxygenation index, albumin, alkaline phosphatase, alanine Aminotransferase (ALT), aspartate aminotransferase (BE), Base Excess (BE), Prothrombin Time (PT), Partial Thromboplastin Time (PTT), bicarbonate, bilirubin, Brain Natriuretic Peptide (BNP), Blood Urea Nitrogen (BUN), creatinine, chloride, fibrinogen, BNP, BMI, and the like, Glucose, hematocrit, hemoglobin, International Normalized Ratio (INR), lactate, lymphocytes, magnesium ions, neutrophils, platelets, potassium ions, Ph, sodium ions, troponin, and leukocytes), 1 fluid output (urine volume), and 6 treatment modalities (mechanical ventilation, continuous renal dialysis, dobutamine, dopamine, epinephrine, and norepinephrine).

TABLE 1.3 population baseline comparison of study populations in two age groups

TABLE 2 study variables incorporated by predictive model

The data processing module process in the invention is as follows:

the raw data of the 3 study groups and the determined study variables obtained by the process (one) are input into a data processing module to complete the preparation work before the model is constructed. The data cleaning method comprises the steps of determining the uniform names of all variables of 3 research populations (4 data sets), and combining multiple expression modes of the same variables in the same data set. Removing all variables based on the information of the physiological boundary range; data sampling, namely performing down-sampling processing (averaging processing) on variables (vital signs) with a plurality of value records in each hour in the first day of an ICU; thirdly, data interpolation, namely interpolating variables with the deletion ratio of less than 30% in the whole population of the variables by using the median of the population, wherein the variables with the deletion ratio of more than 30% need to be added with label columns for identifying whether the variables are actually measured (such as rotate and rotate _ flag); fourthly, constructing statistical characteristics, namely further extracting the statistical characteristics from the data of the variables mentioned in the step one in the first 24 hours of the ICU, wherein the names of the extracted statistical characteristics are shown in a table 2, and the names of the finally obtained characteristics are shown in a table 3. Wherein, the number of the personal information is 3, the number of nerves and weakness is 3, the number of vital signs is 11, the urine volume is 1, the number of the laboratory examinations is 55, and the number of the treatment information is 6. Table 3 also presents the deletion ratios of all study variables in the low and high age groups in each study population simultaneously.

TABLE 3.3 variable deletion ratios of study populations

The model construction and evaluation module process in the invention is as follows:

model construction and training are carried out, and models are respectively constructed for patients of low age and old age. Model construction with a model derived from 80%

And (3) the Cohort1-n population, wherein an ensemble learning model XGboost is selected for the research, and the research characteristics obtained in the step (II) are input into a prediction model. On this basis, 80% of the data set will be further divided into 80% for training of the model and 20% for tuning of the model parameters. The finally obtained model operating function and the hyper-parameter are set as follows:

params＝{'base_score':0.5,'booster':'gbtree','colsample_bylevel':1,'colsample_bynode':1,'colsample_bytree':1,'gamma':0,'learning_rate':0.025,'max_delta_step':0,'max_depth':7,'min_child_weight':4.0,'missing':1,'n_estimators':430,'n_jobs':-1,'nthread':None,'objective':'binary:logistic','random_state':0,'reg_alpha':0,'reg_lambda':1,'scale_pos_weight':1,'seed':None,'silent':None,'subsample':0.85,'verbosity':1}

model_use＝xgboost.XGBClassifier(**params)

explainer＝shap.TreeExplainer(model_use)

20% of the Cohort1-n population was used for internal validation of the model. For subsequent comparison model performance, we trained 3 machine learning models (logistic regression LR, random forest RF and naive bayes NB models) simultaneously. Fig. 7 and 8 present internal validation results for the low and high age models, XGBoost (our model) performance consistently better than 3 machine learning models and 3 clinical scores. Prediction result AUROC of the elderly of low age: XGboost (0.866), LR (0.844), RF (0.792), NB (0.784), APSIII (0.753), SAPS (0.742), SOFA (0.706). Prediction result AUROC of the elderly: XGBoost (0.821), LR (0.793), RF (0.742), NB (0.731), apsiiii (0.697), SAPS (0.708), SOFA (0.673); performance evaluation of the model, all patients of Cohort2 and Cohort3 are used as evaluation population. Performance evaluation was performed in 3 ways (external validation, time series validation, subgroup analysis). The selected models of the study were compared with 3 of the above mentioned machine learning models (logistic regression LR, random forest RF and naive bayes NB model) and 3 of the commonly used clinical scores (acute physiological assessment score APSIII, systemic infection associated organ failure score SOFA, simplified acute physiological score SAPS), respectively. 7 evaluation indexes and 1 function index are selected for the performance of quantitative and qualitative evaluation models and other comparison models/scores. Fig. 9 and 10 present the external validation results for both age groups, XGBoost consistently superior to other models and clinical scores. Prediction result AUROC of the elderly of low age: XGboost (0.856), LR (0.836), RF (0.795), NB (0.767), APSIII (0.775), SAPS (0.766), SOFA (0.628). Prediction result AUROC of the elderly: XGboost (0.853), LR (0.831), RF (0.796), NB (0.784), APSIII (0.732), SAPS (0.774), SOFA (0.628). Fig. 11 and 12 present the two age group timing verification results with XGBoost consistently superior to other models and clinical scores. Prediction result AUROC of the elderly of low age: XGboost (0.845), LR (0.822), RF (0.772), NB (0.772), APSIII (0.819), SAPS (0.733), SOFA (0.689). Prediction result AUROC of the elderly: XGboost (0.776), LR (0.723), RF (0.701), NB (0.697), APSIII (0.819), SAPS (0.733), SOFA (0.689). Fig. 13 presents calibration curve performance for models of two age groups for internal, external, and time series validation, with the results of the model having better proximity to the y-x curve. Fig. 14 presents the bias of the two age group prediction models in each race (caucasian, black, and hispanic), with the models differing less in performance in each race, with the black versus hispanic performing less than the caucasian. Tables 4 and 5 present detailed performance presentation of the model at 7 indices of internal validation, external validation, time series validation for both age groups. Tables 6 and 7 present the predicted performance comparisons of the model for the two age groups in detail with 3 machine learning models and 3 clinical scores; risk factor ranking of the models for predicting the low-age aged MODS and the high-age aged MODS in assessing disease risk can be obtained based on the SHAP method, and the important features of Top20 are respectively shown in fig. 15 and fig. 16. The MODS risk factors for the elderly of low age are ranked as: GCS, charsen co-morbid index, whether mechanical ventilation is performed, respiratory rate, urea nitrogen, shock index, heart rate, BMI, body temperature minimum, total urine volume during the day, body temperature maximum, age, rate of use of norepinephrine, blood oxygen saturation, blood potassium, blood glucose, hematocrit, lymphocytes, carbon dioxide partial pressure, and creatinine. The MODS risk factors for the elderly are ranked as: mechanical ventilation, GCS, respiratory rate, charsen co-morbid index, blood oxygen saturation, heart rate, body temperature nadir, shock index, total urine volume during the day, body temperature nadir, creatinine, aspartate aminotransferase, carbon dioxide partial pressure, Code status, albumin, oxygen partial pressure, BMI, lactate, rate of use of norepinephrine, and lymphocytes. Fig. 17 and 18 are analytical causes presenting disease severity assessments for two age-stratified 4 patients based on interpretable predictive models, presenting risk and protective factors and the respective weight of outcome in the assessed patients. Table 9 shows the results of the predictive performance of the top 79 (all), top 25, top20, top 15 and top 10 important feature models, respectively, based on the above feature rankings, and it can be seen that the performance is slightly reduced, but the performance of the models remains mostly better than the commonly used clinical scores in various ways of assessment. Finally, the above mentioned data processing procedure, two predictive models (elderly and elderly), interpretable functions are packaged to form a device that can automatically perform data cleaning, calculation, evaluation and give the reason for analysis, as shown in fig. 19.

TABLE 4 Young-old mortality Risk prediction model validation results

Index (95% CI)	Internal authentication	External verification, Europe	Time sequence verification, USA
				AUROC	0.866(0.849-0.881)	0.856(0.82-0.888)	0.845(0.828-0.862)
Sensitivity of the composition	0.816(0.781-0.848)	0.847(0.786-0.906)	0.821(0.786-0.856)
				Specificity of	0.742(0.727-0.754)	0.718(0.688-0.749)	0.702(0.686-0.715)
Accuracy of	0.748(0.736-0.761)	0.733(0.706-0.761)	0.713(0.7-0.726)
				F1 value	0.425(0.397-0.452)	0.444(0.384-0.5)	0.375(0.348-0.401)
Accuracy of	0.287(0.263-0.31)	0.301(0.252-0.349)	0.243(0.223-0.264)
				AUPRC	0.521(0.473-0.569)	0.498(0.415-0.597)	0.416(0.373-0.465)

TABLE 5 verification results of Old-Old mortality prediction model

TABLE 6 comparison of Young-old prediction model with machine learning model and clinical scores

TABLE 7 comparison of Old-Old prediction model with machine learning model and clinical scores

TABLE 8 Young-Old and Old-Old prediction models characteristic ranking based on SHAP method

TABLE 9 incorporation of Young-Old and Old-Old prediction models into partial feature model Performance

The invention has the advantages that:

(1) aiming at two high-risk groups (low-age and high-age MODS elderly patients) in the ICU, the occurrence probability of hospital bad outcomes and the contribution degree of risk factors can be predicted early, and then doctors are assisted to perform early intervention and treatment on the patients;

(2) after training of a large sample and a multi-center data set, external and time sequence verification, comparison of a calibration curve and subgroup analysis related to ethnicity, the performance of the model is evaluated by adopting 7 evaluation indexes and 1 functional index, the model has good performance and universality, and the performance is consistently better than that of a baseline model and the clinical existing score;

(3) important factors and ranking associated with poor outcome for the elderly patients of low age and old age can be provided respectively to help doctors understand the development process of diseases;

(4) the number of input data can be selected to be 10-79 according to actual application scenes, and the prediction evaluation performance meeting clinical requirements can be obtained;

(5) the risk prediction device can fully automatically output the risk assessment result and the visual risk reasoning process of the early-stage occurrence of the hospital adverse outcome (death) of the patient, can be conveniently deployed in a hospital information system, and is convenient for the operation and use of doctors.

Unless defined otherwise, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The materials, methods, and examples set forth in this application are illustrative only and not intended to be limiting.

Although the present invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the teachings of this application and yet remain within the scope of this application.

Claims

1. An interpretable hierarchical early-age MODS mortality risk assessment model, comprising an assessment module; the evaluation module is based on an XGboost model fused with a SHAP method, and comprises a plurality of characteristics;

2. The interpretable hierarchical model for early mortality risk in old MODS according to claim 1, wherein:

the plurality of features are from class 6 data; the 6 types of data are respectively:

personal information, comprising: age, gender, BMI index;

a liquid output comprising: the urine volume;

3. The interpretable hierarchical model for early mortality risk in old MODS according to claim 2, wherein:

for the elderly-aged assessment sub-module, the input features are the first 10, 15, 20 or 79 features of the plurality of features from high to low according to the importance;

4. The interpretable hierarchical model for early mortality risk in old MODS according to claim 1, wherein: comprises a data processing module;

5. An interpretable hierarchical old-aged MODS early-mortality risk assessment apparatus comprising a computing unit for executing the interpretable hierarchical old-aged MODS early-mortality risk assessment model of any one of claims 1 to 4; the model adopts a SHAP method fused with the model to obtain the assessment of the contribution degree of the risk factors of the individual patients; the first color is used for representing that the factor is in the abnormal state at present and has harmful influence on the outcome of the patient, the second color is used for representing that the factor is in the normal state at present and has no harmful influence on the outcome of the patient, and the influence degree on the outcome is larger when the SHAP value is larger.

6. A method of establishing an interpretable model for assessing risk of early mortality in elderly MODS, comprising:

7. The method for establishing the interpretable model for evaluating the risk of early death in elderly MODS according to claim 6, wherein:

in model construction and evaluation, the evaluation model is trained, evaluated and optimized for MODS patients aged 65 years or more and less than 80 years old and MODS patients aged 80 years or more, respectively.

8. The method for establishing the interpretable model for evaluating the risk of early death in elderly MODS according to claim 6, wherein:

the plurality of intensive care data sets comprises: MIMIC-III, eICU-CRD, AmsterdamUMCdb, and MIMIC-IV; the study population was screened for patients based on the SOFA score and the established inclusion protocol to obtain data sets for patients with MODS aged 65 years or older but less than 80 years old and patients with MODS aged 80 years or older, respectively.

9. The method for establishing the interpretable model for evaluating the risk of early death in elderly MODS according to claim 8, wherein:

when training the model, fusing the data from MIMIC-III and eICU-CRD as a large sample, multi-center training set, wherein 80% of the patient data is used for training the model and adjusting the hyper-parameters of the prediction model by adopting cross validation, and the rest 20% of the patient data is used for internal validation of the model performance;

10. The method for establishing the interpretable model for evaluating the risk of early death in elderly MODS according to claim 8, wherein:

internal validation was evaluated using 20% of the patient data from MIMIC-III and eICU-CRD in the data set consistent with the training set; external validation is carried out by adopting all patient data of AmsterdamUMCdb distributed in a inconsistent way with training data for evaluation; the timing verification uses all patient data of the MIMIC-IV updated over the training set time span for evaluation; subgroup analysis verified the groups as caucasian, african and hispanic, respectively; so as to comprehensively evaluate whether bias exists or not, and the universality and the robustness of the model.