KR102509550B1 - Apparatus and method for predicting recurrence - Google Patents

Abstract

The present disclosure relates to an apparatus and method for predicting recurrence, and may provide an apparatus and method for predicting a recurrence probability by selecting variables influencing cancer recurrence from patient information and inputting them to each of a plurality of algorithms. In particular, a recurrence predicting apparatus and method capable of more accurately predicting cancer recurrence may be provided by comparing the performance of the predictive models for each generated algorithm to determine a final predictive model.

Description

Apparatus and method for predicting recurrence {APPARATUS AND METHOD FOR PREDICTING RECURRENCE}

The present embodiments provide a recurrence prediction apparatus and method.

Today, the number of cancer patients is increasing due to the westernization of dietary life, and accordingly, various methods for cancer treatment are required. In addition, with the development of medical technology, the number of people who survive after cancer treatment is increasing. Therefore, as the number of cancer survivors gradually increases, quality of life after treatment and lifestyle management are drawing attention. However, in reality, cancer survivors have the highest recurrence frequency within 1 to 2 years after cancer surgery, and recurrence and metastasis may occur even after up to 10 to 15 years, so regular examinations and steady management are required. In particular, there is a problem that most patients live with vague anxiety about cancer recurrence or death because they do not manage properly other than checking for recurrence after treatment. Therefore, in maximizing the therapeutic effect of cancer, there is a need for a technology capable of predicting the recurrence of cancer in advance depending on the patient.

Recently, with the development of information digitization and data storage technology, a large amount of data has been accumulated, and artificial intelligence technology has been introduced and utilized in various fields. In particular, machine learning, a type of artificial intelligence technology, analyzes input data and probabilistically classifies objects or predicts values within a specific range, and is gradually being used in the medical field. Therefore, if the recurrence of cancer is predicted in advance using such machine learning, it will help determine the patient's personalized treatment strategy.

Against this background, the present embodiments may provide a recurrence prediction apparatus and method for predicting the probability of cancer recurrence from variables affecting cancer recurrence using artificial intelligence.

In order to achieve the above object, in one aspect, the present embodiment, in the device for predicting recurrence, obtains patient-specific patient information including body information, clinical information, and recurrence information, and determines the patient to relapse based on the recurrence information. A basic variable setting unit that extracts the primary variable showing the difference between the group and the non-recurrence group, and sets the basic variable from the secondary variable selected from among the primary variables, and patient information for each patient is set as a preset standard A learning data generation unit that selects and generates learning data, generates a model that generates a prediction model for each algorithm that predicts the probability of recurrence by inputting learning data corresponding to a basic variable or a combination variable set from the basic variables into each of a plurality of algorithms. It provides a recurrence prediction device comprising a model determining unit for determining a final predictive model by comparing the performance of the predictive model for each unit and algorithm.

In another aspect, in the present embodiment, in the recurrence prediction method, patient-specific patient information including body information, clinical information, and recurrence information is obtained, and based on the recurrence information, the patient is divided into a recurrence group and a non-recurrence group, Basic variable setting step of extracting the primary variable that shows the difference between the classified groups and setting the basic variable from the secondary variable selected from the primary variables, learning to generate learning data by selecting patient information for each patient based on preset criteria Data generation step, model generation step of generating a predictive model for each algorithm that predicts the probability of recurrence by inputting learning data corresponding to the basic variable or combination variable set from the basic variables into each of a plurality of algorithms, and the performance of the predictive model for each algorithm It provides a recurrence prediction method comprising a model determination step of comparing and determining a final prediction model.

According to the present embodiments, it is possible to provide a recurrence prediction apparatus and method for predicting the probability of cancer recurrence from variables affecting cancer recurrence using artificial intelligence.

1 is a diagram exemplarily illustrating a system configuration to which the present disclosure may be applied.
2 is a diagram showing the configuration of a recurrence prediction device according to an embodiment of the present disclosure.
3 is a diagram illustrating an example for explaining an operation of extracting a variable from patient information in a recurrence prediction apparatus according to an embodiment of the present disclosure.
4 is a diagram illustrating an example for explaining an operation of generating input learning data in a recurrence prediction apparatus according to an embodiment of the present disclosure.
5 is a diagram illustrating an example for explaining learning data generated by correcting patient information in an apparatus for predicting recurrence according to an embodiment of the present disclosure.
6 is a diagram illustrating an example for explaining an operation of determining a final predictive model in the apparatus for predicting recurrence according to an embodiment of the present disclosure.
7 is a diagram illustrating an example for explaining an operation of determining a smoothing parameter value of a CNB algorithm in an apparatus for predicting recurrence according to an embodiment of the present disclosure.
8 is a diagram illustrating an example for explaining an operation of determining a combination variable and a final predictive model in a recurrence prediction apparatus according to another embodiment of the present disclosure.
9 is a diagram illustrating an example for explaining an operation of comparing performance of predictive models for each algorithm in the apparatus for predicting recurrence according to an embodiment of the present disclosure.
10 is a diagram illustrating an example for explaining an operation of comparing performance of predictive models for each algorithm generated according to a combination variable in the apparatus for predicting recurrence according to an embodiment of the present disclosure.
11 is a flowchart of a recurrence prediction method according to an embodiment of the present disclosure.

The present disclosure relates to an apparatus and method for predicting recurrence.

Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. In adding reference numerals to components of each drawing, the same components may have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing the present embodiments, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present technical idea, the detailed description may be omitted. When "comprises", "has", "consists of", etc. mentioned in this specification is used, other parts may be added unless "only" is used. In the case where a component is expressed in the singular, it may include the case of including the plural unless otherwise explicitly stated.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present disclosure. These terms are only used to distinguish the component from other components, and the nature, sequence, order, or number of the corresponding component is not limited by the term.

In the description of the positional relationship of components, when it is described that two or more components are "connected", "coupled" or "connected", the two or more components are directly "connected", "coupled" or "connected". ", but it will be understood that two or more components and other components may be further "interposed" and "connected", "coupled" or "connected". Here, other components may be included in one or more of two or more components that are “connected”, “coupled” or “connected” to each other.

In the description of the temporal flow relationship related to components, operation methods, production methods, etc., for example, "after", "continued to", "after", "before", etc. Alternatively, when a flow sequence relationship is described, it may also include non-continuous cases unless “immediately” or “directly” is used.

On the other hand, when a numerical value or corresponding information (eg, level, etc.) for a component is mentioned, even if there is no separate explicit description, the numerical value or its corresponding information is not indicated by various factors (eg, process factors, internal or external shocks, noise, etc.) may be interpreted as including an error range that may occur.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 is a diagram exemplarily illustrating a system configuration to which the present disclosure may be applied.

Referring to FIG. 1 , the present disclosure relates to a system for providing a recurrence prediction method, and may be implemented in a recurrence prediction device 110 and a server 100 .

The recurrence prediction device 110 may include a general PC such as a general desktop or laptop computer, and may include a mobile terminal such as a smart phone, a tablet PC, a PDA (Personal Digital Assistants), and a mobile communication terminal, and is limited thereto. It should be interpreted broadly as any electronic device capable of communicating with the server 100.

The server 100 has the same configuration as a conventional web server (Web Server), web application server (Web Application Server), or web server (WAP Server) in terms of hardware. However, in terms of software, as will be described in detail below, it includes program modules that are implemented through any language such as C, C++, Java, PHP, .Net, Python, Ruby, and perform various functions. can do.

In addition, the server 100 may be connected to an unspecified number of clients (including the device 110) and/or other servers through a network. Accordingly, the server 100 receives requests from clients or other servers to perform tasks, and It may mean a computer system that derives and provides work results for it, or computer software (server program) installed for such a computer system.

In addition, the server 100 is understood as a broad concept including, in addition to the above-described server program, a series of application programs that operate on the server 100 and, in some cases, various databases built inside or outside. It should be.

Here, the database may refer to an aggregate of data in which data such as information or data is structured and managed for use by a server or other device, and may also refer to a storage medium for storing such an aggregate of data.

In addition, such a database may include a plurality of databases classified according to a data structure method, management method, type, and the like. In some cases, the database may include a database management system (DBMS), which is software that allows information or data to be added, corrected, or deleted.

In addition, the server 100 may store and manage contents and various types of information and data in a database. Here, the database may be implemented inside or outside the server 100 .

In addition, the server 100 uses server programs that are provided in various ways according to operating systems such as DOS, Windows, Linux, UNIX, and Macintosh in general server hardware It can be implemented, and as a representative example, a website, IIS (Internet Information Server) used in a Windows environment, and Apache, Nginx, Light HTTP, etc. used in a Unix environment can be used.

On the other hand, the network 120 is a network that connects the server 100 and the recurrence prediction device 110, and may be a closed network 120 such as a local area network (LAN) and a wide area network (WAN) However, it may be an open network 120 such as the Internet. Here, the Internet refers to the TCP/IP protocol and various services existing in its upper layer, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), It refers to a worldwide open computer network structure that provides Simple Network Management Protocol (SNMP), Network File Service (NFS), and Network Information Service (NIS).

The apparatus and method for predicting recurrence according to an embodiment of the present disclosure briefly described above will be described in more detail below.

2 is a diagram showing the configuration of a recurrence prediction device according to an embodiment of the present disclosure.

Referring to FIG. 2 , the apparatus for predicting recurrence 110 according to an embodiment of the present disclosure obtains patient-specific patient information including body information, clinical information, and recurrence information, and assigns the patient to a recurrence group based on the recurrence information. A basic variable setting unit 210 that extracts a primary variable showing a difference between the groups and a non-recurrence group, and sets a basic variable from a secondary variable selected from among the extracted primary variables, patient information for each patient The learning data generation unit 220 selects based on a preset criterion to generate learning data, and each algorithm predicts the probability of recurrence by inputting learning data corresponding to a basic variable or a combination variable set from the basic variables into each of a plurality of algorithms. The recurrence prediction device 110 includes a model generator 230 that generates a predictive model and a model determiner 240 that compares performance of the predictive model for each algorithm to determine a final predictive model.

The basic variable setting unit 210 may obtain patient information for each patient including body information, clinical information, and recurrence information, and classify patients into a recurrence group and a non-recurrence group based on the recurrence information. For example, the basic variable setting unit 210 may obtain a data set for analysis from a web-based large-scale renal cell carcinoma (RCC) cohort database (DB). For example, the body information included in the patient information may be information such as gender, age at the time of surgery, body mass index, smoking status, and the like. In addition, clinical information included in patient information may be information such as pathologic T stage, histopathological type, necrosis, lymphovascular invasion, capsular invasion, and Fuhrman nuclear grade. Further, the recurrence information relates to recurrence of renal cancer within a certain period of time, and may be information regarding recurrence within 5 years or recurrence within 10 years. However, the period of 5 years or 10 years is an example, but is not limited thereto. In addition, renal cancer is an example, but is not limited thereto.

As another example, the basic variable setting unit 210 may classify patients who have relapsed within a certain period of time as a recurrence group based on recurrence information in patient information for each patient, and classify patients who have not recurred as a non-recurrence group. For example, the basic variable setting unit 210 divides patient information for each patient including physical information and clinical information, which are independent variables of the prediction model, into a recurrence group and a non-recurrence group according to recurrence information, which is a dependent variable of the prediction model. can be distinguished.

The basic variable setting unit 210 may extract a primary variable showing a difference between groups from patient information. For example, patient information stored in the RCC (Renal Cell Carcinoma) cohort DB (database) includes a total of 205 variables. For example, the basic variable setting unit 210 may calculate a significance probability value (p-value) using a T-test and a Chi square test from stored patient information. The basic variable setting unit 210 may extract a primary variable significant to recurrence from variables included in the stored patient information based on the calculated significance probability value.

Also, the basic variable setting unit 210 may select a secondary variable from among the extracted primary variables and set the basic variable from the selected secondary variables. For example, the basic variable setting unit 210 may remove variables having a missing rate greater than or equal to a specific value from the extracted primary variables, and select secondary variables from the remaining variables based on clinical results. Also, the basic variable setting unit 210 may set a basic variable among selected secondary variables. Details on setting the basic variables will be described later with reference to FIG. 3 .

The learning data generation unit 220 may generate learning data by selecting the obtained patient information for each patient based on a preset criterion. For example, the learning data generation unit 220 may generate learning data used in a predictive model by selecting patient information for each patient based on previously set whether or not the patient had undergone surgery, a certain period of recurrence, a missing value, and the like.

The learning data generation unit 220 may generate learning data by correcting patient information according to a ratio of patients between divided groups. For example, if the ratio of patients between the recurrence group and the non-recurrence group is equal to or greater than a specific ratio, the learning data generation unit 220 corrects patient information so that the ratio of patients between the two groups is constant using SMOTE (synthetic minority oversampling technique) to obtain learning data. can create For example, if data imbalance exists when the patient ratio between the recurrence group and the non-recurrence group exceeds a specific ratio, the learning data generation unit 220 may solve the data imbalance by oversampling. Details on generating training data by resolving data imbalance will be described later with reference to FIG. 5 .

The model generating unit 230 may generate a predictive model for each algorithm that predicts a recurrence probability by inputting learning data corresponding to a basic variable or a combination variable set from the basic variables to each of a plurality of algorithms. For example, the model generation unit 230 sets a combination variable by adding at least one or more of the secondary variables to the basic variable, and inputs learning data corresponding to the combination variable to each of a plurality of algorithms to predict the probability of recurrence. A predictive model for each algorithm can be created. Accordingly, the model generation unit 230 may regenerate a prediction model for each algorithm by setting a combination variable to compare algorithm performance according to variables while adding variables one by one in the prediction model for each algorithm generated based on the basic variables.

The model determiner 240 may determine a final prediction model by comparing performances of the prediction models for each generated algorithm. For example, the model determiner 240 calculates at least one of accuracy, sensitivity, specificity, and ROC curve (Receiver Operating Characteristics) from the generated prediction model for each algorithm, and displays the calculation result. It is possible to determine a final prediction model among prediction models for each algorithm. In addition, when a predictive model for each algorithm is generated according to the combination variables set from the basic variables, the model determiner 240 may compare performance of the generated prediction model for each algorithm to determine a final combination variable. Accordingly, the model determination unit 240 may determine a final predictive model based on the determined final combination variable.

As another example, when the final prediction model uses a complement naive bayes (CNB) algorithm, the model determiner 240 may determine a smoothing parameter value of the CNB algorithm implemented in the final prediction model. For example, the model determiner 240 may determine a smoothing parameter value according to a performance comparison result obtained by substituting smoothing parameter values within a certain range into a final predictive model.

3 is a diagram illustrating an example for explaining an operation of extracting a variable from patient information in a recurrence prediction apparatus according to an embodiment of the present disclosure.

Referring to FIG. 3 , the basic parameter setting unit 210 of the apparatus for predicting recurrence according to an embodiment of the present disclosure may obtain patient information for each patient including body information, clinical information, and recurrence information (S310). For example, the basic variable setting unit 210 may obtain patient information for each patient from a total of 6849 patient data of medical institutions in a KOrean Renal Cell Carcinoma (KORCC) DB (database).

The basic variable setting unit 210 may extract a primary variable from patient information for each patient (S320). For example, the basic variable setting unit 210 may extract a primary variable in order to search for a variable that affects recurrence within a certain period of time from a total of 205 variables in the KOrean Renal Cell Carcinoma (KORCC) DB (database). there is. In addition, the basic variable setting unit 210 calculates a significance probability value (p-value) using a T-test and a Chi square test from patient information, and the calculated significance probability value A variable with a value of 0.05 or less can be extracted as a significant primary variable for recurrence. Here, the T-test may be used to test whether the information for each patient can represent the population through the obtained information for each patient. And the chi-square test can be used to test the degree of association between variables. However, the significance probability value serving as a standard is 0.05, but this value may be changed and used within a range of 0.5 or less, for example.

The basic variable setting unit 210 may select a secondary variable from among the extracted primary variables (S330). For example, the basic variable setting unit 210 may remove variables having a missing rate greater than or equal to a specific value from among the extracted primary variables, and may select secondary variables from the remaining variables based on clinical results. For example, the basic variable setting unit 210 may remove a variable with a missing rate of 25% or more, which reduces the sample size of observations and reduces statistical significance, because the amount of missing data is large among the extracted primary variables. In addition, the basic variable setting unit 210 may select a variable having a clinically significant effect on recurrence among the remaining variables as a secondary variable. For example, the selected secondary variables are four variables related to gender, age at the time of surgery, body mass index, and smoking status corresponding to body information, and pathologic T stage, pathologic type, necrosis, lymphovascular invasion, and clinical information corresponding to clinical information. It can be selected as 6 parameters related to capsular invasion and Fuhrman nuclear grade.

The basic variable setting unit 210 may set the basic variable among the selected secondary variables (S340). For example, the basic variable setting unit 210 may set basic variables to generate a predictive model for each algorithm for predicting the probability of recurrence. For example, the basic variables are 4 variables related to gender, age at the time of surgery, body mass index, and smoking status corresponding to body information, and 3 variables related to pathologic T stage, histopathological type, and necrosis corresponding to clinical information. can be set.

The basic variable setting unit 210 may set a combination variable by adding a variable based on the basic variable (S350). For example, the basic variable setting unit 210 may set a combination variable by adding at least one or more selected secondary variables to the basic variable. For a specific example, the variable to be added may be at least one variable among lymphovascular invasion, capsular invasion, and Fuhrman nuclear grade. Therefore, the recurrence prediction device generates a predictive model for each algorithm that predicts the probability of recurrence using a plurality of algorithms from basic variables, and compares the performance of the regenerated predictive model for each algorithm while adding variables one by one to the final combination variable and A final predictive model can be determined.

4 is a diagram illustrating an example for explaining an operation of generating input learning data in a recurrence prediction apparatus according to an embodiment of the present disclosure.

Referring to FIG. 4, the basic parameter setting unit 210 of the apparatus for predicting recurrence according to an embodiment of the present disclosure is a patient from a web-based large-scale DB built by hospitals participating in a KORCC (KOrean Renal Cell Carcinoma) study group. Individual patient information may be acquired (S410). For example, the basic variable setting unit 210 includes a total of 6849 patient data in the KORean Cell Carcinoma (KORCC) DB (database) and patient-specific patient information including a total of 205 variables that significantly affect renal cancer recurrence. can obtain

The basic variable setting unit 210 may set the variable through a variable search process (S420). For example, the basic variable setting unit 210 may select 31 variables showing differences between the recurrence group and the non-recurrence group using a T-test and a Chi square test. In addition, the basic variable setting unit 210 may finally select 10 clinically significant variables among the selected 31 variables as secondary variables. Therefore, the basic variable can be set among the selected secondary variables.

The learning data generation unit 220 may generate learning data by selecting patient information for each patient based on a preset criterion (S430). For example, the learning data generation unit 220 may generate learning data with data of only 5281 patients who received surgical treatment, excluding patients who did not undergo surgical treatment among a total of 6849 patient data stored in the DB. In addition, the learning data generating unit 220 may generate selected learning data by excluding patient data that relapses after a certain period of time and patient data corresponding to follow-up loss. Also, the learning data generating unit 220 may generate selected learning data by excluding data having a missing value in the selected secondary variable. For example, the learning data generation unit 220 may generate learning data excluding data of 13 patients who relapsed after 10 years and data of 1079 patients corresponding to follow-up loss. Further, the learning data generating unit 220 may generate learning data excluding 1375 patient data having missing values in the selected 10 variables.

The learning data generation unit 220 may generate learning data corresponding to basic variables or combination variables to be input to a plurality of algorithms (S440). For example, the learning data generation unit 220 may select 10 clinically significant variables among variables showing differences between the recurrence group and the non-recurrence group, and generate data corresponding thereto as learning data. In addition, the learning data generation unit 220 may generate patient data of 2814 patients selected from patient information for each patient as learning data.

However, although described based on each numerical value in this specification, it is not limited thereto.

5 is a diagram illustrating an example for explaining learning data generated by correcting patient information in an apparatus for predicting recurrence according to an embodiment of the present disclosure.

Referring to FIG. 5 , if the learning data generating unit 220 of the recurrence prediction device according to an embodiment of the present disclosure has a patient ratio between a recurrence group and a non-recurrence group classified according to whether or not recurrence within a certain period of time is greater than or equal to a specific ratio, SMOTE An example of generating learning data by correcting patient information using a synthetic minority oversampling technique will be described. For example, the learning data generation unit 220 may check the patient ratio 510 of the recurrence group and the non-recurrence group, classified according to recurrence within 5 years. Specifically, the ratio of patients between groups is 1:10, and it can be confirmed that the number of patients in the recurrence group is 210 and the number of patients in the non-recurrence group is 2022. Therefore, the learning data generation unit 220 generates learning data by correcting patient information so that the patient ratio between groups is 1:1 and the number of patients in each group is 2022 in a method of generating new data using SMOTE. can do. In this way, by resolving the imbalance between the data, it is possible to provide the effect of properly learning the model.

For example, the learning data generation unit 220 may generate learning data by correcting patient information so that a patient ratio between groups is constant using an oversampling method among synthetic minority oversampling techniques (SMOTE). Here, the oversampling method may be a method of creating a new sample made by adding a random value from a sample of a group having a small number of data and adding it to the data. Specifically, the learning data generation unit 220 may generate new data using synthetic samples of data of a low ratio group by utilizing a bootstrapping or K-Nearest Neighbor (KNN) model technique. can That is, the learning data generation unit 220 calculates the difference between a specific vector and the nearest neighbor among the decimal data, multiplies the difference by a number between 0 and 1, adds it to the target vector, and obtains a line segment between the two vectors. You can create new data by selecting an arbitrary point along the way.

As another example, the learning data generation unit 220 may check the patient ratio 520 of the recurrence group and the non-recurrence group, classified according to recurrence within 10 years. Specifically, the patient ratio between the groups is 1:10, and it can be confirmed that the number of patients in the recurrence group is 226 and the number of patients in the non-recurrence group is 2025. Although the number of patients differed between groups according to a certain period of time as a criterion for recurrence, it can be confirmed that the imbalance in the proportion of patients between groups is maintained. Therefore, the learning data generation unit 220 generates learning data by correcting patient information so that the patient ratio between groups is 1:1 and the number of patients in each group is 2025 in a manner of generating new data using SMOTE. can do.

6 is a diagram illustrating an example for explaining an operation of determining a final predictive model in the apparatus for predicting recurrence according to an embodiment of the present disclosure.

Referring to FIG. 6 , the model generation unit 230 of the apparatus for predicting recurrence according to an embodiment of the present disclosure inputs the learning data corresponding to a basic variable or a combination variable set from the basic variables to each of a plurality of algorithms, thereby recurring recurrence. A predictive model for each algorithm predicting probability may be generated (S610). The model generator 230 may use representative machine learning algorithms as a plurality of algorithms in the predictive model to learn the configured training data. For example, the plurality of algorithms may be algorithms such as Kernel Support Vector Machine (SVM), Logistic Regression, DecisionTree, K-Nearest Neighbor (KNN), Complement Naive Bayes, RandomForest, AdaBoost, GradientBoost, Xgboost, and the like. However, the algorithm that can be used in the model generating unit 230 is not limited thereto and may be various machine learning algorithms that can be used now or in the future.

The model determination unit 240 may compare performance of prediction models for each algorithm generated using each of a plurality of algorithms (S620). For example, the model determination unit 240 calculates at least one of accuracy, sensitivity, specificity, and ROC curve (Reciever Operating Characteristics) from the predictive model for each algorithm, and uses the calculation result The performance of prediction models for each algorithm can be compared. Accordingly, the model determination unit 240 may determine a final prediction model among prediction models for each algorithm. For example, accuracy means the ratio of the number in which the actual value and the predicted value match among the total number, and sensitivity means the ratio in which the predicted value and the actual value agree among the positive (recurrence) actual values, and the specific Degree may mean the ratio of coincidence between the predicted value and the actual value among those whose actual value is negative (non-recurrence). The model determination unit 240 may calculate accuracy, sensitivity, and specificity, and may evaluate that the performance of the model is good as the calculation result has a larger value. In addition, the model determiner 240 generates an ROC curve with 1-specificity as the X-axis and sensitivity as the Y-axis, and the wider the AUC (Area Under the Curve) of the generated ROC curve, the better the model. Performance can be evaluated as good. For example, when the model determiner 240 comprehensively compares the ROC curve, accuracy, sensitivity, and specificity in the prediction model for each algorithm for predicting the probability of recurrence, the prediction model using the CNB algorithm has the best performance. It can be confirmed that

The model determination unit 240 may determine a smoothing parameter value of the CNB algorithm when the final prediction model determined from among prediction models for each algorithm uses a complement naive bayes (CNB) algorithm (S620). For example, the model determiner 240 may determine a smoothing parameter value according to a performance comparison result obtained by substituting a smoothing parameter value in the range of 0 to 200 in order to increase the accuracy of the CNB algorithm. Here, the smoothing parameter is set to exclude the case where the probability is 0, and can be used for value correction.

The model determination unit 240 may determine a final predictive model based on the determined smoothing parameter value. For example, the model determiner 240 may determine a prediction model in which a complement naive bayes (CNB) algorithm is used and a smoothing parameter value is optimized as the final prediction model.

7 is a diagram illustrating an example for explaining an operation of determining a smoothing parameter value of a CNB algorithm in an apparatus for predicting recurrence according to an embodiment of the present disclosure.

Referring to FIG. 7 , when the model determination unit 240 of the apparatus for predicting recurrence according to an embodiment of the present disclosure determines the final prediction model as a prediction model using a complement naive bayes (CNB) algorithm, the CNB algorithm An example of determining the value of the smoothing parameter (α) will be described. For example, the model determiner 240 may utilize a Laplace smoothing technique to determine a smoothing parameter value. This is to prevent the probability from becoming 0 by adding a smoothing parameter value to the denominator numerator of the probability. However, the model determiner 240 may determine the smoothing parameter value according to the performance comparison result obtained by substituting the smoothing parameter value in the range of 0 to 200, respectively. As a specific example, in the case of a predictive model predicting recurrence within 5 years, the model determiner 240 may determine that the performance of the predictive model is the best when the smoothing parameter value of the CNB algorithm is 10. Accordingly, the model determiner 240 may determine a smoothing parameter value of 10 for the final predictive model for predicting recurrence within 5 years implemented by the CNB algorithm. On the other hand, in the case of a predictive model predicting recurrence within 10 years, the model determiner 240 may determine that the performance of the predictive model is the best when the smoothing parameter value of the CNB algorithm is 100. Accordingly, the model determiner 240 may determine a smoothing parameter value of 100 for the final prediction model for predicting recurrence within 10 implemented by the CNB algorithm. However, the set recurrence period and the determined smoothing parameter value are examples, and are not limited thereto.

8 is a diagram illustrating an example for explaining an operation of determining a combination variable and a final predictive model in a recurrence prediction apparatus according to another embodiment of the present disclosure.

Referring to FIG. 8 , the model generating unit 230 of the apparatus for predicting recurrence according to an embodiment of the present disclosure may set combination variables from basic variables (S810). For example, the model generator 230 may set a combination variable by adding at least one or more secondary variables to the basic variable. For example, the model generating unit 230 may use at least one of lymphovascular invasion, capsular invasion, and Fuhrman nuclear grade in basic variables including gender, age at the time of surgery, body mass index, smoking status, pathologic T stage, histopathological type, and necrosis. Combination variables can be set by adding the above variables.

The model generating unit 230 may generate a predictive model for each algorithm for predicting a recurrence probability by inputting learning data corresponding to a basic variable or a combination variable to each of a plurality of algorithms (S820). For example, the model generation unit 230 may generate a prediction model for each algorithm by inputting learning data corresponding to a basic variable to each of a plurality of algorithms. In addition, the model generation unit 230 adds variables included in the secondary variables to the basic variables one by one and inputs learning data corresponding to the set combination variable to each of a plurality of algorithms to generate a predictive model for each algorithm according to the combination variable. there is.

The model determination unit 240 may compare performance of prediction models for each algorithm generated according to basic variables or combination variables (S830). For example, the model determiner 240 may compare performance by calculating at least one of accuracy, sensitivity, specificity, and ROC curve from a predictive model for each algorithm generated according to the changed variable.

The model determination unit 240 may determine a final combination variable according to a performance comparison result of prediction models for each algorithm generated according to basic variables or combination variables (S830). For example, the model determination unit 240 may determine a variable with good performance as a final combination variable according to a performance comparison result of prediction models for each algorithm. As a specific example, the model determination unit 240 may confirm that the performance of the predictive model for each algorithm generated using all 10 variables selected as secondary variables is the best. Accordingly, the model determining unit 240 may determine the final combination variables as 10 variables.

The model determination unit 240 may determine a final prediction model from prediction models for each algorithm generated according to the determined final combination variable (S850). For example, the model determination unit 240 may determine a final prediction model by comparing performance of prediction models for each algorithm generated by inputting training data corresponding to the same final combination variable to each of a plurality of algorithms. As a specific example, the model determination unit 240 may input 10 variables selected as secondary variables to the CNB algorithm to determine a final predictive model as a model for predicting a recurrence probability.

9 is a diagram illustrating an example for explaining an operation of comparing performance of predictive models for each algorithm in the apparatus for predicting recurrence according to an embodiment of the present disclosure.

Referring to FIG. 9 , the model determination unit 240 of the recurrence prediction apparatus according to an embodiment of the present disclosure compares the performance of prediction models for each algorithm generated using basic variables and uses a complement naive bayes (CNB) algorithm. An example of determining the prediction model as the final prediction model will be described. For example, the model determiner 240 may calculate at least one of accuracy, sensitivity, specificity, and ROC curve from the prediction model for each algorithm, and compare the performance of the prediction model for each algorithm using the calculation result. Referring to FIG. 9 , the model determiner 240 can confirm that the performance of a prediction model using a complement naive bayes (CNB) algorithm is the highest among prediction models for each algorithm. Accordingly, the model determination unit 240 may determine a model using the CNB algorithm as a final prediction model.

10 is a diagram illustrating an example for explaining an operation of comparing performance of predictive models for each algorithm generated according to a combination variable in the apparatus for predicting recurrence according to an embodiment of the present disclosure.

Referring to FIG. 10, the model determination unit 240 of the recurrence prediction apparatus according to an embodiment of the present disclosure compares performance of prediction models for each algorithm generated using 10 combination variables to determine a final prediction model. Explain an example. For example, the model determiner 240 calculates at least one of accuracy, sensitivity, specificity, and ROC curve from a predictive model for each algorithm generated according to a combination variable set from basic variables, and uses the calculation result for each algorithm. You can compare the performance of predictive models. Referring to FIG. 10 , the model determiner 240 may confirm that the performance of the predictive model with 10 combination variables is higher than that of the basic variables or other combination variables. Here, the 10 combined variables can be set to gender, age at surgery, body mass index, smoking status, Pathologic T stage, histopathological type, necrosis, lymphovascular invasion, capsular invasion, and Fuhrman nuclear grade. In addition, the model determiner 240 can confirm that the performance of the prediction model using the CNB algorithm is the highest among the prediction models for each algorithm using 10 combination variables. Accordingly, the model determination unit 240 may determine a model for predicting a recurrence probability by inputting 10 combination variables to the CNB algorithm as a final prediction model.

Hereinafter, a recurrence prediction method that can be performed by the apparatus for predicting recurrence described with reference to FIGS. 1 to 10 will be described.

11 is a flowchart of a recurrence prediction method according to another embodiment of the present disclosure.

Referring to FIG. 11 , the recurrence prediction method of the present disclosure may include a basic variable setting step of setting basic variables (S1110). For example, the recurrence prediction device may obtain patient information for each patient including physical information, clinical information, and recurrence information, and classify patients into a recurrence group and a non-recurrence group based on the recurrence information. For example, the apparatus for predicting recurrence may acquire a data set for analysis by constructing a large-scale web-based renal cell carcinoma (RCC) cohort DB (database). For example, the body information included in the patient information may be information such as gender, age at the time of surgery, body mass index, smoking status, and the like. In addition, clinical information included in patient information may be information such as pathologic T stage, histopathological type, necrosis, lymphovascular invasion, capsular invasion, and Fuhrman nuclear grade. In addition, the recurrence information relates to recurrence within a certain period of time, and may be information regarding recurrence within 5 years or recurrence within 10 years.

As another example, the apparatus for predicting recurrence may classify patients who have relapsed within a certain period of time as a recurrence group based on recurrence information in patient information for each patient, and classify patients who have not recurred as a non-recurrence group. For example, the apparatus for predicting recurrence may classify patient information for each patient including body information and clinical information as independent variables of the prediction model into a recurrence group and a non-recurrence group according to recurrence information as a dependent variable of the prediction model.

For example, the recurrence prediction device may extract a primary variable showing a difference between the groups from patient information, and may set a basic variable from a secondary variable selected from among the extracted primary variables. For example, the recurrence prediction device calculates a significance probability value (p-value) using a T-test and a Chi square test from stored patient information, and calculates a significance probability value As a criterion, the primary variable significant in recurrence can be extracted. Also, the apparatus for predicting recurrence may remove variables having a missing rate greater than or equal to a specific value from the extracted primary variables, and select secondary variables from the remaining variables based on clinical results. Also, the recurrence prediction device may set a basic variable among selected secondary variables.

The recurrence prediction method of the present disclosure may include a learning data generation step of generating learning data (S1120). For example, the apparatus for predicting recurrence may generate learning data by selecting acquired patient information for each patient based on a preset criterion. For example, the apparatus for predicting recurrence may generate learning data used in a predictive model by selecting patient information for each patient based on previously set whether or not the patient had undergone surgery, a certain period of recurrence, a missing value, and the like.

As another example, the recurrence prediction device may generate learning data by correcting patient information according to a patient ratio between divided groups. For example, if the ratio of patients between the recurrence group and the non-recurrence group exceeds a certain ratio, the recurrence prediction device corrects patient information so that the ratio of patients between the two groups is constant using SMOTE (synthetic minority oversampling technique) to generate learning data. can

The recurrence prediction method may include a model generation step of generating a predictive model for each algorithm (S1130). For example, the recurrence prediction apparatus may generate a predictive model for each algorithm that predicts a recurrence probability by inputting learning data corresponding to a basic variable or a combination variable set from the basic variables to each of a plurality of algorithms. For example, the recurrence prediction device sets a combination variable by adding at least one or more of the secondary variables to the basic variable, and inputs learning data corresponding to the combination variable to each of a plurality of algorithms to predict the probability of recurrence. A predictive model can be created. Through this, the recurrence predicting device can compare the performance of the predictive model for each algorithm while adding variables one by one in the predictive model for each algorithm generated based on the basic variables.

The recurrence prediction method may include a model determination step of determining a final prediction model (S1140). For example, the recurrence predicting device may determine a final predictive model by comparing the performance of the generated predictive model for each algorithm. For example, the recurrence prediction device calculates at least one of accuracy, sensitivity, specificity, and ROC curve (Receiver Operating Characteristics) from the generated predictive model for each algorithm, and uses the calculation result A final prediction model may be determined among prediction models for each algorithm. In addition, when a predictive model for each algorithm is generated according to the combination variables set from the basic variables, the recurrence prediction apparatus may compare performance of the generated prediction model for each algorithm to determine a final combination variable. Accordingly, the recurrence prediction device may determine a final predictive model based on the determined final combination variable.

As another example, when the final prediction model uses a complement naive bayes (CNB) algorithm, the recurrence prediction device may determine a smoothing parameter value of the CNB algorithm implemented in the final prediction model. In addition, the recurrence prediction apparatus may determine a final predictive model by determining a smoothing parameter value according to a performance comparison result of the final predictive model derived by substituting smoothing parameter values within a certain range.

In the above, the recurrence prediction method according to an embodiment of the present disclosure has been described as being performed by the same procedure as in FIG. Accordingly, the procedure for performing each step may be changed, two or more steps may be integrated, or one step may be separated into two or more steps.

The above description is merely illustrative of the technical idea of the present disclosure, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the technical idea. In addition, the present embodiments are not intended to limit the technical idea of the present disclosure, but rather to explain the scope of the present technical idea by these embodiments. The scope of protection of the present disclosure should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of rights of the present disclosure.

Claims (16)

Acquiring patient-specific patient information including physical information, clinical information, and recurrence information, classifying patients into a recurrence group and a non-recurrence group based on the recurrence information, extracting a primary variable showing a difference between the classified groups, , a basic variable setting unit for setting a basic variable from a secondary variable selected from among the primary variables;
a learning data generation unit configured to generate learning data by selecting the patient information for each patient based on a preset criterion;
a model generating unit generating a predictive model for each algorithm for predicting a recurrence probability by inputting the learning data corresponding to the basic variable or a combination variable set from the basic variables to each of a plurality of algorithms; and
Recurrence prediction device comprising a; model determination unit for determining the final prediction model by comparing the performance of the prediction model for each algorithm. According to claim 1,
The basic variable setting unit,
Calculating a P-value from the patient information using a T-test and a Chi square test, and extracting the primary variable based on the significance probability value Recurrence prediction device, characterized in that. According to claim 2,
The basic variable setting unit,
A recurrence predicting device, characterized in that for removing variables whose missing rate is equal to or greater than a specific value from the primary variables, and selecting the secondary variables based on clinical results from the remaining variables. According to claim 1,
The learning data generating unit,
If the patient ratio between the groups is greater than or equal to a specific ratio, a synthetic minority oversampling technique (SMOTE) is used to correct the patient information so that the ratio of patients between the groups is constant to generate learning data. According to claim 1,
The model generator,
Recurrence prediction apparatus, characterized in that for setting a combination variable by adding at least one or more of the secondary variables to the basic variable, and generating a predictive model for each algorithm according to the combination variable. According to claim 5,
The model determining unit,
Recurrence predicting device, characterized in that for determining the final combination variable by comparing the performance of the predictive model for each algorithm generated according to the combination variable, and determining the final prediction model based on the final combination variable. According to claim 1,
The model determining unit,
At least one of Accuracy, Sensitivity, Specificity, and ROC Curve (Reciever operating characteristics) is calculated from the prediction model for each algorithm, and the final prediction is made among the prediction models for each algorithm using the calculation result. Recurrence prediction device, characterized in that for determining the model. According to claim 1,
The model determining unit,
When the final prediction model uses a Complement naive bayes (CNB) algorithm, a smoothing parameter value of the CNB algorithm is determined, and performance derived by substituting each of the smoothing parameter values within a certain range into the final prediction model Recurrence prediction device, characterized in that for determining the smoothing parameter value according to the comparison result. In the method for predicting recurrence by a recurrence prediction device,
The basic variable setting unit of the recurrence prediction device acquires patient-specific patient information including body information, clinical information, and recurrence information, classifies patients into a recurrence group and a non-recurrence group based on the recurrence information, and A basic variable setting step of extracting a primary variable showing a difference and setting a basic variable from a secondary variable selected from among the primary variables;
a learning data generation step in which the learning data generation unit of the recurrence prediction device selects the patient information for each patient based on a preset criterion and generates learning data;
A model generating step of generating a predictive model for each algorithm for predicting a recurrence probability by inputting the learning data corresponding to the basic variable or a combination variable set from the basic variables to each of a plurality of algorithms, by the model generating unit of the recurrence predicting device; and
A model determination step of determining a final prediction model by comparing the performance of the prediction model for each algorithm by the model determining unit of the recurrence prediction device; Recurrence prediction method comprising a. According to claim 9,
The basic variable setting step,
Calculating a P-value from the patient information using a T-test and a Chi square test, and extracting the primary variable based on the significance probability value Recurrence prediction method, characterized in that. According to claim 10,
The basic variable setting step,
A method for predicting recurrence, characterized in that, from the primary variable, variables having a missing rate greater than or equal to a specific value are removed, and the secondary variable is selected from the remaining variables based on clinical results. According to claim 9,
The learning data generation step,
If the patient ratio between the groups is more than a specific ratio, using SMOTE (Synthetic minority oversampling technique), the patient information is corrected so that the ratio of patients between the groups is constant to generate learning data. Recurrence prediction method. According to claim 9,
The model creation step,
A method for predicting recurrence, characterized in that a combination variable is set by adding at least one or more of the secondary variables to the basic variable, and a predictive model for each algorithm is generated according to the combination variable. According to claim 13,
The model determination step,
Recurrence prediction method, characterized in that the final combination variable is determined by comparing the performance of the prediction model for each algorithm generated according to the combination variable, and the final prediction model is determined based on the final combination variable. According to claim 9,
The model determination step,
At least one of Accuracy, Sensitivity, Specificity, and ROC Curve (Reciever operating characteristics) is calculated from the prediction model for each algorithm, and the final prediction is made among the prediction models for each algorithm using the calculation result. Recurrence prediction method characterized by determining the model. According to claim 9,
The model determination step,
When the final prediction model uses a Complement naive bayes (CNB) algorithm, a smoothing parameter value of the CNB algorithm is determined, and performance derived by substituting each of the smoothing parameter values within a certain range into the final prediction model Recurrence prediction method, characterized in that for determining the smoothing parameter value according to the comparison result.

Images