KR20220064243A - Artificial intelligence-based abnormal reaction monitoring method and system therefor - Google Patents

Abstract

The present specification relates to a method for monitoring an abnormal response based on artificial intelligence and an apparatus therefor. More specifically, the method for monitoring an abnormal response disclosed in the present invention includes the steps of: extracting input data for learning a machine learning model from a database storing medical data related to medical records of patients, based on the medical data; inputting the extracted input data to the machine learning model, and learning the machine learning model to assign feature importance to each of all elements included in the input data, in which the feature importance is calculated based on a degree of correlation between a specific element and the abnormal response determined through the machine learning model; and determining at least one suspicious element related to the occurrence of the abnormal response among the all elements based on the feature importance. Thus, the number of reaction-causing test objects is reduced.

Description

Artificial intelligence-based abnormal reaction monitoring method and system therefor

The present invention relates to an artificial intelligence-based abnormal reaction monitoring method and system, and more particularly, to a method and system for monitoring an abnormal reaction based on feature importance obtained in a machine learning model learning process.

In the midst of increasing interest in and importance of monitoring for adverse drug reactions in recent years, the Food and Drug Administration, medical institutions that prescribe and manufacture drugs, and pharmaceutical companies that produce drugs, etc. Efforts are being made

Currently, active monitoring of adverse reactions through statistical modeling is the mainstay, and according to world-renowned academic journals, statistical modeling is specialized in deriving inferences about the population, and machine learning is specialized in finding generalizable patterns. More specifically, a data mining technique called tree scan based statistic has been used for active monitoring of adverse reactions. In the tree scan-based analysis, analysis can be performed only on an analysis target having a hierarchical structure, only the presence or absence of diagnosis or prescription is used for analysis, and quantitative information such as the number of diagnoses or prescriptions is not reflected. has the limitations of In addition, in the tree scan-based analysis, there is a limitation in analysis ability in that interactions between drugs of different layers are not considered.

In addition to the tree scan-based analysis, another machine learning-based adverse reaction study includes an analysis based on natural language processing. In analysis based on natural language processing, adverse reactions can be found indirectly by using literature search. However, the adverse reactions inferred through the natural language processing-based analysis have limitations in that natural language processing has inaccuracies and that the inferred adverse reactions are not based on direct study of prescription and diagnostic data. In addition, in research through conventional statistical modeling, the method of finding the optimal solution in closed form for terabytes of data is used. However, the amount of computing power and computation required to find the optimal solution in closed form for terabytes of data is very high. There is a problem that it is not easy to apply big data because it is a cursor.

An object of the present specification is to provide an artificial intelligence-based abnormal reaction monitoring method and a system therefor.

Another object of the present specification is to provide a method and a system for detecting a suspicious element causing an adverse reaction by using the feature importance of a machine learning model.

The technical problems to be achieved in this specification are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

The present specification provides an artificial intelligence-based abnormal reaction monitoring method and a system therefor.

More specifically, the present specification provides a method comprising: extracting input data for learning a machine learning model from a database storing medical data related to medical records of patients based on the medical data; inputting the extracted input data into a machine learning model, and training the machine learning model so that feature importance is assigned to each of all elements included in the input data, wherein the feature importance is determined through learning the machine learning model calculated based on the degree of correlation between the determined adverse reaction and a specific factor; and determining at least one suspicious factor related to induction of an adverse reaction from among the total factors based on the feature importance.

Also, in the present specification, the step of extracting the input data may further include the step of extracting, for each patient, data at the time of occurrence of the adverse event related to the time at which the adverse event occurred in the patients, respectively.

Also, in the present specification, the extracting of the input data includes, based on the data on occurrence of an adverse event extracted for each patient, a medical record of the specific patient for a specific period before the time when the adverse event occurs in the specific patient and It may be characterized in that it further comprises the step of extracting the related event (event) interval data.

In addition, in the present specification, the step of extracting the input data includes extracting control section data related to the medical record of the specific patient during the specific period before any time other than the time when the adverse event occurs in the specific patient. It may be characterized in that it further comprises the step of

In addition, in the present specification, the medical record included in each of the event section data and the control section data for the patients is at least one of a record related to diagnosis of a specific symptom, a record related to drug prescription, or a record related to vaccination It may be characterized in that it includes.

In addition, the present specification may be characterized in that a time section of a specific interval is inserted between the time section from which the event section data is extracted and the time section from which the control section data is extracted.

In addition, the present specification may be characterized in that the time interval from which the event interval data is extracted and the time interval from which the control interval data is extracted based on the inserted time interval of the specific interval are temporally spaced apart without overlapping.

In addition, in the present specification, the machine learning model may be characterized in that it is learned to distinguish between the event section data and the control section data by giving the feature importance to each of the entire elements.

In addition, the present specification further comprises the step of determining whether the abnormal reaction occurs by inputting specific input data that is one of the event section data or the control section data to the machine learning model on which the machine learning model learning is completed can be characterized as

In addition, the present specification may be characterized in that the feature importance is given as a high value to a factor determined to have a relatively high degree of correlation.

In addition, the present specification may be characterized in that the at least one suspicious element is composed of elements whose feature importance is greater than a specific threshold value among all the elements.

In addition, the present specification may be characterized in that the specific threshold value is set to a value that is twice the overall average value of the feature importance given to each of the all elements.

In addition, the adverse reaction monitoring system provided herein includes a database (data base) for storing medical data related to patients' medical records; machine learning models; and a control unit, wherein the control unit extracts input data for learning a machine learning model from the database based on the medical data, and inputs the extracted input data to the machine learning model. , the machine learning model is trained so that feature importance is assigned to each of the elements included in the input data, and the feature importance is the degree of correlation between the specific element and the abnormal response determined through the machine learning model learning is calculated based on , and at least one suspect factor related to induction of an adverse reaction is determined from among all the factors based on the feature importance.

In addition, in the present specification, the controller may extract, for each patient, data at the time of occurrence of the adverse event related to the time at which the adverse event occurred in the patients, respectively, in order to extract the input data.

In addition, in the present specification, in order to extract the input data, the control unit may control the specific patient's It may be characterized in that the event period data related to the medical record is extracted.

In addition, in the present specification, in order to extract the input data, the controller is a control section related to the medical record of the specific patient during the specific period before any time other than the time when the abnormal reaction occurs in the specific patient. It may be characterized by extracting data.

In addition, in the present specification, the medical record included in each of the event section data and the control section data for the patients is at least one of a record related to diagnosis of a specific symptom, a record related to drug prescription, or a record related to vaccination It may be characterized by including.

In addition, the present specification may be characterized in that a time section of a specific interval is inserted between the time section from which the event section data is extracted and the time section from which the control section data is extracted.

In addition, the present specification may be characterized in that the time interval from which the event interval data is extracted and the time interval from which the control interval data is extracted based on the inserted time interval of the specific interval are temporally spaced apart without overlapping.

In addition, in the present specification, the machine learning model may be characterized in that it is learned to distinguish between the event section data and the control section data by giving the feature importance to each of the entire elements.

The present specification has the effect of monitoring abnormal reactions based on artificial intelligence.

In addition, the present specification has the effect of detecting a suspicious element causing an adverse reaction by utilizing the feature importance of the machine learning model.

In addition, the present specification has the effect of reducing the number of substances to be tested for inducing an adverse reaction by detecting a suspicious element that induces an adverse reaction.

Effects that can be obtained in this specification are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to help the understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical features of the present invention.
1 is a block diagram of an artificial intelligence-based abnormal reaction monitoring system according to an embodiment of the present invention.
2 is a flowchart of an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
3 is a diagram illustrating an example of extracting input data for machine learning model learning in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
4 is a diagram illustrating an example for helping the understanding of a learning process of a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
5 is a diagram illustrating an example for helping the understanding of a learning process of a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
6 is a diagram illustrating an example of a process of learning a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
7 is a diagram illustrating a result of monitoring a suspected drug causing an adverse reaction based on an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.
8 is a flowchart illustrating an example of an operation implemented in a control unit for performing an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description for better understanding of the present invention, provide embodiments of the present invention, and together with the detailed description, explain the technical features of the present invention. Throughout the specification, like reference numerals refer to like elements in principle. In addition, if it is determined that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

Hereinafter, the method and apparatus related to the present invention will be described in more detail with reference to the drawings. In addition, the general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context before and after, and should not be interpreted in an excessively reduced meaning. Also, as used herein, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or various steps described in the specification, some of which components or some steps are It should be construed that it may not include, or may further include additional components or steps. The suffixes "unit", "module" and "unit" for components used in the following description are given or mixed in consideration of ease of writing the specification, and do not have a meaning or role distinct from each other by themselves. . Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms.

Hereinafter, an embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an artificial intelligence-based abnormal response monitoring system according to an embodiment of the present invention, and FIG. 2 is a flowchart of an AI-based abnormal response monitoring method according to an embodiment of the present invention.

1 and 2 , according to an embodiment of the present invention, the controller 130 may extract data for learning a machine learning model from the database 110 ( S210 ). More specifically, the controller 130 first extracts data used for data extraction for learning the machine learning model 120 from the database 110 , and learns the machine learning model based on the previously extracted data. data can be extracted for A process in which the controller 130 extracts data for learning the machine learning model based on the previously extracted data may be referred to as a 'data preprocessing' process. Here, the database 110 may store medical data related to medical records of patients.

Thereafter, the controller 130 may train the machine learning model 120 by using the data for learning the machine learning model extracted in step S210 ( S220 ). More specifically, the extracted data for machine learning model learning may be configured in the form of a specific input value (X) and an output value (Y) for the specific input value (X). Here, the process of training the machine learning model 120 may be understood as a process of inferring a relationship between an arbitrary input value (X) and an output value (Y) for the arbitrary input value (X). That is, the trained machine learning model 120 can know the relationship between the input value and the output value, and when an arbitrary input value (X) is input to the machine learning model 120 that has been trained, the machine learning model 120 ) can output the output value Y based on the inferred correlation. In the present invention, the input value (X) may be a suspected drug, vaccination, etc., and the output value (Y) may be whether or not an abnormal reaction occurs. Hereinafter, the meaning of learning the machine learning model 120 in the present specification may be understood based on the above-described content.

Next, the controller 130 may determine a suspicious factor that caused an abnormal reaction based on the result of training the machine learning model 120 ( S230 ). More specifically, in the present invention, determining a suspect element based on the result of training the machine learning model 120 means the input value (X) and the output value (Y) inferred by the machine learning model 120 as a result of learning It can be understood as using the relationship itself of That is, in the present invention, when a specific input value (X) is input in order to determine a suspicious factor that induced an abnormal reaction, the relationship between the inferred input value and the output value is used to determine the output value (Y) corresponding to the specific input value. Rather than using the ability of the machine learning model 120 to obtain

Hereinafter, each of the above-described steps S210 to S230 will be described in more detail.

How to extract input data for training machine learning models

3 is a diagram illustrating an example of extracting input data for machine learning model learning in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.

Hereinafter, with reference to FIG. 3 , a method in which the control unit 130 extracts input data for machine learning learning from the database 110 storing medical data related to medical records of patients based on the medical data will be described in detail. Let's take a look at

Referring to FIG. 3 , first, the controller 130 may extract medical data related to patients' medical records from the database 110 . The medical data may be extracted for each patient ( 301 to 303 ). The medical data extraction process may be mutually performed through each communication unit of the controller 130 and the database 110 .

Referring back to FIG. 3 , the controller 130 analyzes the extracted medical data in order to extract the input data for machine learning learning, and extracts data related to the time when an adverse reaction occurs in the patients for each patient. It can be done (FIGS. 310 to 330). Here, the data related to the time point at which the adverse event occurred in the patients may be referred to as data at the time point at which the adverse event occurred, and of course, it may be expressed in various ways within the same and similarly interpreted range. In FIG. 3 , it can be seen that an adverse reaction occurred in patient A and patient C 310 and 330 , but no adverse reaction occurred in patient B 120 .

Referring back to FIG. 3 , in order to extract the input data for machine learning learning, the control unit 130, based on the abnormal reaction occurrence data extracted for each patient, a specific period before the time point when the abnormal reaction occurred in a specific patient Data related to the medical record of the specific patient during the period may be extracted (312, 331, and 333). Here, the data related to the medical record of the specific patient for a specific period before the time when the adverse reaction occurred in the specific patient may be referred to as event interval data, and may be expressed in various ways within the same interpretation range. of course there is The specific period may be set including a time point at which an adverse reaction occurs. In addition, the length of the specific period from which the event period data is extracted may be flexibly set according to the characteristics of the target adverse reaction. For example, when the target adverse event occurs after a short period of time from the time when there was a suspicious factor (ie, drug prescription, vaccination, etc.) related to the target adverse event, the specific period may be set to be short. Conversely, when the target adverse event occurs after a long period of time from the point in time when there was a suspicious factor (ie, drug prescription, vaccination, etc.) related to the target adverse event, the specific period may be set to be long.

Referring back to FIG. 3 , in order to extract the input data for machine learning learning, the controller 130 , the medical record of the specific patient for the specific period before any point other than the time when the abnormal reaction occurred in the specific patient. Data related to can be extracted (311, 313, 321, 322, 323 and 332). Data related to the medical record of the specific patient for the specific period before any time other than the time when the adverse event occurred in the specific patient may be referred to as control interval data, and may vary within the same similarly interpreted range. It is, of course, possible to express

Each of the extracted event section data and control section data may include information such as diagnosis, drug prescription, and vaccination of patients. In addition, the diagnosis, the drug prescription, the vaccination, and the like may be information on a plurality of different types of diagnosis, drug prescription, vaccination, and the like, respectively. Additionally, each of the extracted event section data and control section data may further include patient information such as gender and age of the patients. However, in the present specification, among the information included in each extracted event section data and control section data, information related to drug prescription and vaccination may be more preferably used to monitor suspicious factors that cause adverse reactions. .

Additionally, the number of extracted total event period data and the total number of control period data may be configured at a constant ratio. For example, the ratio may consist of a ratio of the total number of event section data: the total number of control section data = 1:2.

Referring back to FIG. 3 , in order to extract the input data for machine learning learning, the control unit 130 provides a specific interval between a time interval in which the event interval data is extracted and a time interval in which the control interval data is extracted. A time interval of can be inserted. Of course, the time period of the specific interval may be expressed as a wash out period, and may be expressed in various ways within the same and similarly interpreted range. Based on the inserted time section of the specific interval, the time section from which the event section data is extracted and the time section from which the control section data is extracted may be temporally spaced apart from each other without overlapping. When the event section data and the control section data are not separated and overlapped, the learning of the machine learning model 120 based on the input data may be performed inaccurately. Therefore, by configuring so that there is no overlap between the time period in which the event period data is extracted and the time period in which the control period data is extracted, the influence of the event period data and the control period data on each other can be removed, and the machine learning model The learning of (120) can be accurately performed.

In this method, there is an effect that quantitative information such as the number of times of drug prescription and vaccination, etc. can be taken into account, rather than simply considering input data by considering only the presence or absence of prescription and vaccination. In addition, since quantitative information can be considered, there is an effect of obtaining a learning result of the machine learning model 120 in which the interaction between different suspicious elements is considered.

How to train a machine learning model

Hereinafter, a method in which the control unit 130 inputs the extracted input data to the machine learning model 120 and trains the machine learning model will be described in detail. It goes without saying that the input data may also be referred to as learning data and the like, and may be expressed in various ways within a range that can be interpreted similarly to this.

The input data may be composed of sample data obtained by randomly sampling a specific number of data from among the total event section data and control section data extracted by the controller 130 for each patient from the database at a specific number of times. For example, the controller 130 extracts event section data and control section data for each of 10 different patients (patients 1 to 10), and one event section data and two control sections for each patient When data exists, the input data is obtained by randomly sampling a certain number of data multiple times from the entire event interval data and control interval (or control interval) data (10 event interval data and 20 control interval data). It may consist of a plurality of sampling data. That is, if 5 data are randomly sampled 10 times from the entire event interval data and control interval data (10 event interval data and 20 control interval data), 10 consisting of 5 event interval data and/or control interval data The number of sample data may constitute input data. However, configuring the input data in this way is only an example, and the present invention is not limited thereto.

As described above, each event section data and control section data extracted by the controller 130 for each patient from the database may include information such as diagnosis, drug prescription, vaccination, etc. for specific symptoms of patients. Additionally, each of the extracted event section data and control section data may further include patient information such as gender and age of the patients. Here, information such as diagnosis of patients' specific symptoms, drug prescription, vaccination, etc. included in the input data may be collectively referred to as 'total factor'. That is, as a result of the control section 130 analyzing the entire event section data and control section data extracted from the database for each patient, drugs A, B, C and vaccines 1, 2, 3 are included in the total event section data and control section data If so, drugs A, B, C and vaccines 1,2,3 could be 'whole elements'.

In this method, the controller 130 may train the machine learning model 120 so that feature importance is assigned to each of all elements included in the input data. That is, as above, as a result of the control section 130 analyzing the entire event section data and control section data extracted from the database for each patient, drugs A, B, C and vaccines 1, 2, 3 are the entire event section data and control section If included in the data, when the machine learning model 120 has completed learning, the feature importance may be assigned to each of drugs A, B, C and vaccines 1, 2, and 3 . In this case, the feature importance may be respectively calculated based on the degree of correlation between the abnormal response determined through the machine learning model learning and a specific factor. That is, the feature importance may be assigned (or assigned) as a high value to an element determined to have a relatively high degree of correlation among all elements included in the input data. More specifically, as a result of the learning of the machine learning model 120, when it is determined that a specific element included in all elements has a large causal relationship with the occurrence of an abnormal reaction, a relatively high feature importance value will be given to the specific element. can Conversely, as a result of learning the machine learning model 120 , when it is determined that a specific factor included in all factors has a small causal relationship with the occurrence of an abnormal reaction, a relatively low feature importance value may be assigned to the specific factor. . The feature importance may be assigned to each of the entire elements. It goes without saying that the feature importance may also be referred to as feature importance or the like, and may be expressed in various ways within a range that can be interpreted in the same way.

Hereinafter, a process in which feature importance is calculated and assigned to each of all elements included in input data will be described in more detail with reference to FIGS. 4 and 5 .

4 is a diagram illustrating an example for helping the understanding of a learning process of a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.

More specifically, FIG. 4 is a diagram illustrating an example for explaining a problem of an abnormal reaction monitoring method based on a tree scan method. The abnormal reaction monitoring method based on the tree scan method may be performed on data to be analyzed having a hierarchical structure. Here, the analysis target data may include a plurality of sample data. Referring to FIG. 4 , analysis target data (or source data) has a hierarchical structure such as 410 (layer 1), 421 and 422 (layer 2), and 431, 432, 433, and 434 (layer 3). Able to know. More specifically, the layer 2 may be configured by branching the analysis target data of the layer 1, and the layer 3 may be configured by branching the data of the layer 2, respectively.

Referring to layer 1 of FIG. 4 , the original data 410 consists of 4 sample data in which an abnormal reaction occurred and 8 sample data in which an abnormal reaction did not occur. In this case, the sample group incidence rate of the original data 410 is 33 It becomes (4/12 * 100)%.

Next, referring to layer 2 of FIG. 4 , the original data of layer 1 may be divided into data 421 for a case where A is prescribed and data 422 for a case where B is prescribed. Here, the data 421 for the case where A was prescribed consists of 3 sample data in which an adverse reaction occurred and 6 sample data in which the adverse reaction did not occur, and at this time, the data 421 for the case where A was prescribed. The sample population incidence is 33 (3/9 * 100)%. In addition, the data 422 for the case of receiving a prescription for B is composed of two sample data in which an abnormal reaction occurred and four sample data in which an abnormal reaction did not occur. The sample population incidence is 33 (2/6 * 100)%. At this time, since the sample data included in the original data 410 may include sample data corresponding to the case where both A and B are prescribed, the sample data included in the data for the case where A of the layer 2 is prescribed. The sum of the number and the number of sample data included in data for a case in which B is prescribed may be greater than the number of sample data included in original data. Here, each of A and B may be a drug group unit. As a result, since the sample population incidence rate in each of the data 421 for the case of prescription A and the data 422 for the case where B was prescribed is the same, in this case, the data obtained from tier 2 are It cannot be regarded as a meaningful result for determining the suspicious factors related to

Next, referring to the layer 3 of FIG. 4 , the data 421 for the case where A of the layer 2 is prescribed is data 431 for the case where A01 is prescribed and the data 432 for the case where A02 is prescribed. can be divided. Also, the data 422 for the case where B is prescribed may be divided into data 433 for the case where B01 is prescribed and data 434 for the case where B02 is prescribed. Here, the data 431 for the case where A01 was prescribed consists of two sample data in which an adverse reaction occurred and four sample data in which an adverse reaction did not occur. The sample population incidence is 33 (2/6 * 100)%. In addition, the data 431 for the case where A02 was prescribed consists of one sample data in which an adverse reaction occurred and two sample data in which an adverse reaction did not occur. The sample population incidence is 33 (1/3 * 100)%.

The data 433 for the case where B01 was prescribed consists of one sample data in which an adverse reaction occurred and two sample data in which an adverse reaction did not occur. The incidence rate becomes 33 (1/2 * 100)%. In addition, the data 434 for the case where B02 was prescribed consists of one sample data in which an adverse reaction occurred and two sample data in which an adverse reaction did not occur. The sample population incidence is 33 (1/3 * 100)%.

Here, A01 and A02 may be sub-drugs belonging to drug group A, and each of B01 and B02 may be sub-drugs belonging to drug group B. As a result, data on the case where A01 was prescribed (431), data about the case where A02 was prescribed (432), data about the case where B01 was prescribed (433), and data about the case where B02 was prescribed (434) Since the incidence rates of the sample population in each are the same, in this case, the data obtained from stratum 3 cannot be regarded as a meaningful result for determining a suspect factor related to the occurrence of an adverse event.

As such, in the case of the abnormal reaction monitoring method based on the tree scan method, since the analysis is performed only within the hierarchy, there may be cases where a suspicious element related to the occurrence of an adverse reaction cannot be detected depending on the composition of the sample data included in the analysis target data. exist. That is, the present method may have a limitation that the interaction between drugs of different layers (ie, analysis of the presence or absence of prescription A on the premise of prescription B) cannot be considered.

5 is a diagram illustrating an example for helping the understanding of a learning process of a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention. More specifically, FIG. 5 is a diagram to help the understanding of feature importance calculation based on a random forest method.

The input data for learning the machine learning model based on the random forest method may be composed of sample data obtained by randomly sampling a specific number of data from the entire event section data and control section data extracted for each patient.

In the random forest method, the concept of impurity is used to calculate the feature importance of each of all elements included in the input data for machine learning model training, and the importance of each of the elements based on the impurity is used. gain: IG), and feature importance may be calculated based on the importance gain of each of the total elements. The above process may be performed on each of the sample data included in the input data.

A process in which feature importance is calculated based on impurity and importance gain in one sample data will be described in more detail with reference to FIG. 5 .

5, it is assumed that the controller 130 analyzes the entire event section data and control section data extracted from the database 110 for each patient, and it is assumed that all elements are drugs A and B, in FIG. The sampling data used consists of 4 event section data (abnormal reaction occurrence) and 6 control section data (no abnormal reaction occurrence). The sampling data may be sampled from the entire event section data and control section data extracted by the controller 130 for each patient from the database 110 .

First, the process of calculating the importance benefit of drug B is described. In order to calculate the importance gain of drug B, first, the impurity of the original data 510 sampled from the entire data is calculated. Above, the impurity is calculated as 1 - (the number of sample data in which an adverse reaction occurred / the total number of sample data)^2 - (the number of sample data in which an adverse reaction did not occur / the total number of sample data)^2 can be Accordingly, the impurity of the original data 510 becomes 1-(4/12)^2 - (8/12)^2 = 4/9.

Next, calculate the impurity of B. The original data 510 is divided into data 521 for a case where drug B is not prescribed and data 522 for a case where drug B is prescribed, and data 521 for a case where drug B is not prescribed and First, the impurity for each of the data 522 for the case of receiving drug B is calculated. By the method of calculating the impurity, the impurity of the data 521 for the case where drug B is not prescribed becomes 1-(2/6)^2 - (4/6)^2 = 4/9. In addition, the impurity of the data 522 for the case where drug B is prescribed is 1-(2/6)^2 - (4/6)^2 = 4/9. Next, the total impurity of the drug B is calculated by considering the weights of the data 521 for the case where the drug B is not prescribed and the data 522 for the case where the drug B is prescribed. In this case, the weight may be determined based on a ratio of the number of sample data included in each data divided from the original data. That is, the number of sample data included in the original data 510 in FIG. 5 was 12, and the data 521 for the case where the drug B was not prescribed and the data 522 for the case where the drug B was prescribed are 6 Since the sample data is included, the weights of the data 521 for the case where the drug B is not prescribed and the data 522 for the case where the drug B is prescribed can be calculated as 6/12 and 6/12, respectively. . If we calculate the total impurity of drug B based on the calculated weight, it can be calculated as 1/2*4/9 + 1/2*4/9 = 4/9.

Finally, the importance gain of drug B is calculated based on the calculated total impurity of drug B. It can be calculated as importance gain = (impurity of the original data) - (total impurity of the partitioned data). Thus, the (total) importance gain of drug B = (4/9) - (4/9) = 0. These results may mean that drug B has no correlation with the occurrence of adverse events.

Next, the importance benefit of drug A can also be calculated through the same process as the above-described method. 5 shows that the data 522 for the case where B is prescribed is divided into the data 531 for the case where A is prescribed and the data 532 for the case where A is not prescribed, but only when B is prescribed. The data 521 for the case of not receiving may also be divided into data for the case where A is prescribed and data for the case where A is not prescribed. At this time, the data 522 for the case where B is prescribed and the data 521 for the case where B is not prescribed are the original data for each of the divided data for the case where A is prescribed and the case where the prescription is not received. can be In this case, the process of calculating the importance gain of drug A is (i) data divided based on whether drug A is prescribed from data 521 for the case where B is not prescribed in FIG. 5 and (ii) B An importance gain is calculated for each of the divided data 531 and 532 based on whether drug A is prescribed from the data 522 for the case of prescription, and the average value of the calculated importance gain is calculated as the overall importance gain of drug A do.

Finally, the feature importance of drug A and drug B is calculated based on the sample data in FIG. 5 . It can be calculated as feature importance of drug A = total importance gain of drug A/(total importance gain of drug A + overall importance gain of drug B). It can also be calculated as feature importance of drug B = total importance gain of drug B/(total importance gain of drug A + total importance gain of drug B).

Through the above-described process, feature importance for specific sample data sampled from all data may be calculated.

Data that is divided by a specific division (or branching) method in specific sample data and the division is completed may be understood as one data tree. That is, even in one sample data, a plurality of data trees may be configured according to a division method. In conclusion, the process of calculating the final feature importance for all elements is to calculate the feature importance for all elements in each of a plurality of data trees constructed based on a data sample and a segmentation method, respectively, and each of the plurality of data trees It can be performed as a process of taking the average value of the feature importance calculated in . In this case, the number of times division occurs in one data tree, that is, the number of layers may be appropriately set according to the total number of elements. In addition, the number of data included in the sample data and the number of data sampling may be appropriately set by the machine learning model 120 user.

The feature importance calculation method based on the random forest method described above is only an example for helping the understanding of the feature importance calculation process, and the feature importance calculation method of the present invention is not limited thereto. In addition to random forests, SVM models and the like can be used to calculate feature importance. In particular, in the case of the SVM model, a reference line for determining the feature importance of each element on a specific plane is set, the position of each element on the specific plane is set by a specific method, and the reference line and the entire set A feature importance of each of the entire elements may be determined based on wide/narrowness of a margin between positions of each of the elements. More specifically, the wider the interval, the higher the feature importance may be.

6 is a diagram illustrating an example of a process of learning a machine learning model in an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.

In this method, the control unit 130 trains the machine learning model 120 so that feature importance is assigned to each of all elements included in the input data, meaning that the machine learning model 120 controls the event interval data and It may have the same/similar meaning as learning to distinguish interval data. Therefore, apart from the operation of the controller 130 to determine the suspicious factor related to inducing an abnormal reaction based on the feature importance given to each of the elements, which will be described below, the machine learning model 120 that has been trained is It is possible to distinguish the event section data and the control section data based on the learning result. That is, the machine learning model 120 on which the learning is completed extracts the feature importance of each of the elements (drug prescription, vaccination, etc.) included in the specific data input as the input data, and based on the feature importance of each of the extracted elements Thus, it can be determined whether the input data is event section data or control section data. Additionally, the machine learning model 120 may determine whether an abnormal reaction occurs according to a result of determining whether specific input data is event section data or control section data. For example, when it is determined that the input specific data is event section data, it may be determined that the pattern of elements reflected in the input specific data may cause an abnormal reaction. Conversely, when it is determined that the input specific data is control section data, it may be determined that the pattern of elements reflected in the input specific data does not cause an abnormal reaction.

How to Determine Suspects

Hereinafter, a detailed description will be given of a method in which the controller 130 determines a suspicious factor related to inducing an abnormal reaction from among all factors included in the input data based on the feature importance. In the present method, there may be at least one suspect factor related to the induction of an adverse reaction. In addition, each of the at least one suspicious element may be composed of a specific drug group.

In this method, the control unit 130 determines a suspicious factor related to the induction of an adverse reaction among all the factors based on the characteristic importance, wherein the characteristic importance of the factors determined to be the suspicious factor is a value greater than a predetermined specific threshold value. can have As an example, the specific threshold value may be set to a value that is twice the overall average value of the feature importance assigned to each of the elements. This is only an example to help the description, and the present invention is not limited thereto. As another example, the control unit 130 may determine, among all the elements, a specific upper-order element having a large feature importance value as the suspect element.

As described above with reference to FIGS. 1 and 2 , in the present invention, determining a suspect element based on the result of training the machine learning model 120 means an input value inferred by the machine learning model 120 as a result of learning. It can be understood as using the relationship between (X) and the output value (Y) itself. That is, in the present invention, when a specific input value (specific input data) is input in order to determine the suspicious factor that caused the abnormal reaction, the relationship between the inferred input value and the output value (the importance of each feature assigned to all factors) ), rather than using the ability of the machine learning model 120 to obtain an output value (whether or not an adverse reaction occurs) corresponding to a specific input value, the relationship between the inferred input value and the output value (feature importance) itself By using it, it is possible to determine factors having a feature importance greater than or equal to a specific threshold as a suspect factor that induced an abnormal reaction.

7 is a diagram illustrating a result of monitoring a suspected drug causing an adverse reaction based on an artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention. The adverse reaction in FIG. 7 may be anaphylaxis. More specifically, FIG. 7 corresponds to a case in which the top 20 drug groups are determined as a factor suspected of the occurrence of an adverse reaction. As a result of the analysis, it was confirmed that 95% of drug groups except for the M09 drug group were registered in the drug side effect database. In addition, as a result of examining the M09 group, which is not registered in the drug side effect database, it was confirmed that the M09 drug group could cause adverse reactions. The drug side effect database may be SIDER (Protect collaborative and the Side Effect Resouce).

8 is a flowchart illustrating an example of an operation implemented by the control unit 130 for performing the artificial intelligence-based abnormal reaction monitoring method according to an embodiment of the present invention.

More specifically, the controller 130 extracts input data for learning a machine learning model from a database storing medical data related to medical records of patients based on the medical data ( S810).

Next, the controller 130 inputs the extracted input data to the machine learning model, and trains the machine learning model so that feature importance is assigned to each of all elements included in the input data (S820). Here, the feature importance is calculated based on the degree of correlation between the abnormal response and a specific element determined through the machine learning model learning.

Finally, the control unit 130 determines at least one suspicious factor related to induction of an abnormal reaction from among all the factors based on the feature importance ( S830 ).

It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

The embodiments described above are those in which elements and features of the present invention are combined in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, it is also possible to configure an embodiment of the present invention by combining some elements and/or features. The order of operations described in the embodiments of the present invention may be changed. Some features or features of one embodiment may be included in another embodiment, or may be replaced with corresponding features or features of another embodiment. It is obvious that claims that are not explicitly cited in the claims can be combined to form an embodiment or included as a new claim by amendment after filing.

Embodiments according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention provides one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs ( field programmable gate arrays), a processor, a controller, a microcontroller, a microprocessor, and the like.

In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of modules, procedures, functions, etc. that perform the functions or operations described above. The software code may be stored in the memory and driven by the processor. The memory may be located inside or outside the processor, and may transmit/receive data to and from the processor by various well-known means.

It is apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are included in the scope of the present invention.

100: adverse reaction monitoring system
110: database
120: machine learning model
130: control unit

Claims (20)

extracting input data for learning a machine learning model from a database storing medical data related to medical records of patients based on the medical data;
inputting the extracted input data into a machine learning model, and training the machine learning model so that feature importance is assigned to each of all elements included in the input data;
the feature importance is calculated based on a degree of correlation between the abnormal response and a specific factor determined through the machine learning model learning; and
and determining at least one suspect factor related to induction of the adverse event from among the total factors based on the feature importance. The method of claim 1,
The step of extracting the input data,
The method further comprising the step of extracting, for each patient, data on the occurrence time of the adverse event related to the time point at which the adverse event occurred in the patients, respectively. 3. The method of claim 2,
The step of extracting the input data is,
The method further comprising the step of extracting event section data related to the medical record of the specific patient for a specific period prior to the time when the adverse event occurred in the specific patient, based on the abnormal reaction occurrence data extracted for each patient Adverse Event Monitoring Methods. 4. The method of claim 3,
The step of extracting the input data is,
The method further comprising the step of extracting control interval data related to the medical record of the specific patient for the specific period before any time other than the time when the adverse event occurred in the specific patient. 5. The method of claim 4,
Medical records included in each of the event section data and the control section data for the patients include at least one of a record related to diagnosis of a specific symptom, a record related to drug prescription, or a record related to vaccination. Way. 5. The method of claim 4,
An abnormal reaction monitoring method in which a time section of a specific interval is inserted between a time section from which the event section data is extracted and a time section from which the control section data is extracted. 7. The method of claim 6,
An abnormal reaction monitoring method in which a time section from which the event section data is extracted and a time section from which the control section data is extracted based on the inserted time section of the specific interval are temporally spaced apart without overlap. 5. The method of claim 4,
The machine learning model assigns the feature importance to each of the entire elements, and the abnormal reaction monitoring method is trained to distinguish the event section data from the control section data. 9. The method of claim 8,
Abnormal reaction monitoring method further comprising the step of determining whether the abnormal reaction occurs by inputting specific input data that is one of the event section data or the control section data to the machine learning model on which the machine learning model learning has been completed. The method of claim 1,
The feature importance is assigned a high value to a factor determined to have a relatively high degree of correlation. 11. The method of claim 10,
and wherein the at least one suspicious element includes elements in which the feature importance is greater than a specific threshold value among all elements. 12. The method of claim 11,
and the specific threshold value is set to a value that is twice a total average value of the feature importance assigned to each of the all elements. a database for storing medical data related to medical records of patients;
machine learning models; and
control unit; including;
The control unit is
Extracting input data for learning a machine learning model from the database based on the medical data,
By inputting the extracted input data to a machine learning model, the machine learning model is trained so that feature importance is assigned to each of all elements included in the input data,
The feature importance is calculated based on the degree of correlation between the abnormal response and a specific element determined through the machine learning model learning,
The adverse reaction monitoring system is configured to determine at least one suspect element related to induction of the adverse reaction from among the entire elements based on the feature importance. 14. The method of claim 13,
The control unit, to extract the input data,
An adverse reaction monitoring system for extracting, for each patient, data at the time of occurrence of the adverse event related to the time at which the adverse event occurred in the patients. 15. The method of claim 14,
The control unit, to extract the input data,
An adverse reaction monitoring system for extracting event section data related to the medical record of the specific patient for a specific period prior to the time when the adverse event occurred in the specific patient, based on the data on occurrence of the abnormal event extracted for each patient. 16. The method of claim 15,
The control unit, to extract the input data,
An adverse reaction monitoring system for extracting control interval data related to the medical record of the specific patient for the specific period before any time other than the time when the adverse event occurred in the specific patient. 17. The method of claim 16,
Medical records included in each of the event section data and the control section data for the patients include at least one of a record related to diagnosis of a specific symptom, a record related to drug prescription, or a record related to vaccination. system. 17. The method of claim 16,
An abnormal reaction monitoring system in which a time section of a specific interval is inserted between a time section from which the event section data is extracted and a time section from which the control section data is extracted. 19. The method of claim 18,
An abnormal reaction monitoring system in which a time section from which the event section data is extracted and a time section from which the control section data is extracted based on the inserted time section of the specific interval are temporally spaced apart from each other without overlap. 17. The method of claim 16,
The machine learning model is an abnormal reaction monitoring system that is trained to distinguish between the event section data and the control section data by assigning the feature importance to each of the entire elements.

Images