WO2020242239A1

WO2020242239A1 - Artificial intelligence-based diagnosis support system using ensemble learning algorithm

Info

Publication number: WO2020242239A1
Application number: PCT/KR2020/006971
Authority: WO
Inventors: 김원태; 강신욱; 이명재; 김동민; 남동연
Original assignee: (주)제이엘케이
Priority date: 2019-05-29
Filing date: 2020-05-29
Publication date: 2020-12-03
Also published as: KR102100698B1

Abstract

According to the present invention, an integrated learning device for lesions can be provided. The integrated learning device for lesions may include: an image-based training unit which trains at least one image-based learning model that receives a medical image and outputs an image-based lesion prediction result; at least one clinical data-based training unit which trains a clinical data-based learning model that receives clinical data and outputs a clinical data-based lesion prediction result; and an integrated training unit which causes an integrated learning model to perform ensemble learning, the integrated learning model receiving the image-based lesion prediction result and the clinical data-based lesion prediction result, and outputting a final lesion prediction result.

Description

AI-based diagnostic assistance system using ensemble learning algorithm

The present disclosure relates to a deep learning model learning technology, and more specifically, using a method and apparatus for learning about lesions based on medical images and clinical data, and a learning model built on the basis of medical images and clinical data. It is a method and apparatus for diagnosing lesions.

Deep learning is the learning of a very large amount of data, and when new data is input, the highest probability is selected based on the learning result. Such deep learning can operate adaptively according to an image, and since feature factors are automatically found in the process of learning a model based on data, there is a trend of increasing attempts to utilize this in the field of artificial intelligence.

Since lesions are expressed through symptoms, signs, or changes in the body occurring in the body, doctors generally determine the lesions by checking the symptoms or signs or changes in the body of the patient.

Certain lesions may show signs in a specific area of the body, but some specific lesions may appear complexly in various areas of the body, and changes in the body may also appear complex. Therefore, it is difficult to detect a disease or lesion simply by considering only symptoms or signs appearing in a specific area of the patient. For example, a disease such as systemic leukoplakia, which is a kind of rheumatic disease, may exhibit simultaneous and multiple symptoms throughout the body.

An object of the present disclosure is to provide a method and apparatus for learning lesion severity by comprehensively considering an image of a body and a biological change occurring in the body.

Another technical problem of the present disclosure is to provide a method and apparatus for diagnosing a lesion that predicts the severity of a lesion by comprehensively reflecting an image of a body and a biological change occurring in the body.

Another technical task of the present disclosure is to provide a method and apparatus for complexly learning the progress of the disease, the relationship between diseases, the metastasis state, etc. by comprehensively reflecting and learning various symptoms or signs expressed in the body. will be.

Another technical problem of the present disclosure is to provide a diagnostic method and apparatus capable of comprehensively predicting the progress of the disease, the relationship between the diseases, the metastasis state, etc. by comprehensively reflecting the symptoms or signs that are variously expressed in the body. will be.

The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present disclosure belongs from the following description. I will be able to.

According to an aspect of the present disclosure, an apparatus for learning lesion integration may be provided. The device includes an image-based learning unit that receives a medical image and learns at least one image-based learning model that outputs an image-based lesion prediction result, and a clinical data-based learning unit that receives clinical data and outputs a clinical data-based lesion prediction result. Ensemble learning of at least one clinical data-based learning unit that learns the learning model of, and an integrated learning model that receives the image-based lesion prediction result and the clinical data-based lesion prediction result, and outputs the final lesion prediction result. ) May include an integrated learning unit.

According to another aspect of the present disclosure, a method for integrating lesion learning may be provided. The method includes a process of learning at least one image-based learning model that receives a medical image and outputs an image-based lesion prediction result, receives a bio-signal, and outputs a bio-signal-based lesion prediction result corresponding to the bio-signal. At least one process of learning at least one bio-signal-based learning model, receiving at least one clinical information obtained through a clinical trial, and outputting a clinical data-based lesion prediction result corresponding to the at least one clinical information. Ensemble an integrated learning model that receives the process of learning one clinical information-based learning model, the image-based lesion prediction result, the biosignal-based lesion prediction result, and the clinical data-based lesion prediction result, and outputs the final lesion prediction result. It may include a process of ensemble learning.

According to another aspect of the present disclosure, an apparatus for diagnosing a lesion may be provided. The apparatus is an apparatus for diagnosing a lesion using a learning model obtained by learning the lesion, the apparatus comprising: an image-based prediction unit that outputs an image-based lesion prediction result corresponding to an input medical image using at least one image-based learning model; , Using a clinical data-based learning model, at least one clinical data-based prediction unit for outputting each clinical data-based lesion prediction result corresponding to the input clinical data, and the image-based lesion prediction result and the clinical data-based lesion It may include an integrated diagnosis unit that inputs the prediction result into the integrated learning model and checks the final lesion prediction result output through the integrated learning model.

According to another aspect of the present disclosure, a method for diagnosing a lesion may be provided. The method is a method for diagnosing a lesion using a learning model that has learned the lesion, the process of outputting an image-based lesion prediction result corresponding to an input medical image using an image-based learning model, and a biosignal-based learning Using the model, the process of confirming the result of physiological signal-based lesion prediction corresponding to the input of the physiological signal, and using a clinical information-based learning model, based on clinical data corresponding to at least one clinical information acquired through a clinical trial The process of confirming the lesion prediction result, the image-based lesion prediction result, the biosignal-based lesion prediction result, and the clinical data-based lesion prediction result are input into an integrated learning model, and the final lesion prediction result output through the integrated learning model. It may include a process of verifying.

Features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the detailed description of the present disclosure described below, and do not limit the scope of the present disclosure.

According to the present disclosure, a method and an apparatus for learning a lesion severity by comprehensively considering an image of a body and a biological change occurring in the body may be provided.

Further, according to the present disclosure, a method and apparatus for predicting a lesion severity by comprehensively considering an image of a body and a biological change occurring in the body may be provided.

In addition, according to the present disclosure, a method and an apparatus for complexly learning the progress of a disease, a relationship between diseases, a metastasis state, etc. can be provided by learning by comprehensively reflecting and learning various symptoms or signs expressed in the body. have.

In addition, according to the present disclosure, a method and apparatus capable of comprehensively predicting the progress of a disease, a relationship between diseases, a metastasis state, etc. may be provided by comprehensively reflecting various symptoms or symptoms expressed in the body. .

In addition, according to the present disclosure, the diagnosis result is predicted through a complex learning model of disease progression, relationship between diseases, and metastasis based on a large amount of data on symptoms or signs that are variously expressed in the body. By doing so, it is possible to derive a diagnosis result with relatively high reliability compared to the result of diagnosis or determination based on the experience value.

The effects obtainable in the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.

1 is a block diagram showing the configuration of a lesion integrated learning apparatus according to an embodiment of the present disclosure.

2 is a diagram illustrating a learning operation of a learning model provided in the apparatus for learning lesion integration according to an embodiment of the present disclosure.

3 is a block diagram showing a detailed configuration of an image-based learning unit included in the apparatus for learning lesion integration according to an embodiment of the present disclosure.

4 is a block diagram illustrating a detailed configuration of a first clinical data-based learning unit provided in the apparatus for learning lesion integration according to an embodiment of the present disclosure.

5 is a block diagram illustrating a detailed configuration of a second clinical data-based learning unit provided in the apparatus for learning lesion integration according to an embodiment of the present disclosure.

6 is a block diagram showing the configuration of a lesion diagnosis apparatus according to an embodiment of the present disclosure.

7 is a block diagram illustrating a detailed configuration of an image-based detection unit included in a lesion diagnosis apparatus according to an embodiment of the present disclosure.

8 is a block diagram illustrating a detailed configuration of a first clinical data-based detection unit provided in a lesion diagnosis apparatus according to an embodiment of the present disclosure.

9 is a block diagram illustrating a detailed configuration of a second clinical data-based learning unit provided in a lesion diagnosis apparatus according to an embodiment of the present disclosure.

10 is a flowchart illustrating a procedure of a method for learning lesion integration according to an embodiment of the present disclosure.

11 is a flowchart illustrating a procedure of a method for diagnosing a lesion according to an embodiment of the present disclosure.

12 is a block diagram illustrating a method and apparatus for integrated learning of a lesion and a computing system for executing the method and apparatus for diagnosing a lesion according to an embodiment of the present disclosure.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the embodiments. However, the present disclosure may be implemented in various different forms, and is not limited to the embodiments described herein.

In describing an embodiment of the present disclosure, when it is determined that a detailed description of a known configuration or function may obscure the subject matter of the present disclosure, a detailed description thereof will be omitted. In addition, parts not related to the description of the present disclosure in the drawings are omitted, and similar reference numerals are attached to similar parts.

In the present disclosure, when a component is "connected", "coupled" or "connected" with another component, this is not only a direct connection relationship, but an indirect connection relationship in which another component exists in the middle. It can also include. In addition, when a component "includes" or "has" other components, it means that other components may be further included, rather than excluding other components unless otherwise stated. .

In the present disclosure, terms such as first and second are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise stated. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment is a first component in another embodiment. It can also be called.

In the present disclosure, components that are distinguished from each other are intended to clearly describe each characteristic, and do not necessarily mean that the components are separated. That is, a plurality of components may be integrated into one hardware or software unit, or one component may be distributed to form a plurality of hardware or software units. Therefore, even if not stated otherwise, such integrated or distributed embodiments are also included in the scope of the present disclosure.

In the present disclosure, components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, an embodiment consisting of a subset of components described in an embodiment is also included in the scope of the present disclosure. In addition, embodiments including other components in addition to components described in the various embodiments are included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

Referring to FIG. 1, the integrated lesion learning apparatus 10 may include an image-based learning unit 11, at least one clinical data-based learning unit 13, and an integrated learning unit 15.

The image-based learning unit 11 may learn an image-based learning model for receiving a medical image and outputting a lesion prediction result. In particular, the image-based learning unit 11 may include a plurality of image-based learning models, and a probability of developing a specific disease in a specific region included in the medical image may be set as a target variable of the image-based learning model. The detailed configuration and operation of the image-based learning unit 11 will be described in detail with reference to FIG. 3 attached below.

Furthermore, medical images are images of the entire body or a specific diagnostic area through various imaging techniques, such as T2 (T2-weighted) images, ADC (apparent diffusion coefficients) images, STIR images, T1 images, and T1 with agents images. And parametric MRI, X-ray images, CT images, such as FLAIR, and the like.

The clinical data-based learning unit 13 may learn a clinical data-based learning model that receives clinical data and outputs lesion prediction results. Here, the clinical data may include data detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change or a body fluid generated by the user's body, urine, biopsy, and the like. Based on this, the clinical data-based learning model receives data detected from biological signals (e.g., ECG, PPG, EMG, etc.) or body fluids, urine, biopsy, etc. generated by the user's body, and the aforementioned clinical data Corresponding to, the probability of developing a specific disease may be set as an objective variable of a clinical data-based learning model. Accordingly, the clinical data-based learning model may be trained to output a lesion prediction result corresponding to the above-described clinical data.

At least one of the aforementioned clinical data-based learning units 13 may be provided in the lesion integrated learning unit 10, and each clinical data-based learning unit 13 may include a clinical data-based learning model.

Clinical data-based learning models include logistic regression, multi-layer perceptron, stochastic gradient descent, bagging, random forest, decision tree, support vector machine, k-nearest neighbor, multinomial logistic regression, multi-layer perceptron, stochastic gradient descent, random Learning may be performed based on at least one method among forest, decision tree, linear regression, bayesian regression, and kernel ridge regression. In an embodiment of the present disclosure, a machine learning algorithm used for a clinical data-based learning model is illustrated, but the present disclosure is not limited thereto, and various types of machine learning algorithms may be used in addition to the illustrated algorithm.

Furthermore, the clinical data-based learning unit 13 may be configured to be classified according to the type of input data. For example, the clinical data-based learning unit 13 includes a first clinical data-based learning unit 13-1 that learns biological signals (eg, ECG, PPG, EMG, etc.), and detected from body fluids, urine, biopsy, etc. A second clinical data-based learning unit 13-2 for learning data may be included.

Based on the above, the clinical data-based learning unit 13 checks the type of input data, selects the learning units 13-1 and 13-2 corresponding to the identified type, and provides the clinical data. Can be configured to For example, when the input clinical data is confirmed as a bio-signal (eg, ECG, PPG, EMG, etc.), the corresponding clinical data can be input to the first clinical data-based learning unit 13-1, and the input clinical data is In the case of data detected from body fluids, urine, and biopsy, the corresponding clinical data may be input to the second clinical data-based learning unit 13-2.

As another example, the clinical data-based learning unit 13 inputs bio-signals (eg, ECG, PPG, EMG, etc.) to the first clinical data-based learning unit 13-1, and from bodily fluids, urine, biopsy, etc. A user interface may be provided so that the detected data can be input to the second clinical data-based learning unit 13-2. For example, the clinical data-based learning unit 13 sets clinical data suitable for the first clinical data-based learning unit 13-1 and the second clinical data-based learning unit 13-2 in the design stage, and the user An environment in which corresponding clinical data is input to the first and second clinical data-based learning units 13-1 and 13-2 may be provided through the interface.

The detailed structure and operation of the clinical data-based learning unit 13 will be described in detail with reference to FIGS. 4 and 5 below.

On the other hand, the integrated learning unit 15 can receive the lesion prediction result output from the image-based learning unit 11 and the clinical data-based learning unit 13, and as an output corresponding thereto, the integrated learning unit 15 learns the final lesion prediction result. It can contain a learning model. In particular, the integrated learning unit 15 performs ensemble learning on a plurality of lesion prediction results provided from the image-based learning unit 11 and the clinical data-based learning unit 13 to obtain the final lesion prediction results. Configurable.

Specifically, the lesion prediction results output from the image-based learning unit 11 and the clinical data-based learning unit 13 are set as input data of the integrated learning model, and are set when learning the image-based learning model and the clinical data-based learning model. The same objective variable as the one used can be set as the output data of the integrated learning model. Accordingly, the integrated learning model can build an ensemble model by learning weights between the outputs of the image-based learning model and the clinical data-based learning model.

* In FIG. 2, in the integrated lesion learning apparatus according to an embodiment of the present disclosure, an image-based learning model 21, a first clinical data-based learning model 22, a second clinical data-based learning model 23, and It illustrates that the integrated learning model 25 is included.

In order to learn a learning model provided in the integrated learning device, the integrated learning data set 200 may be configured, and data obtained by combining a plurality of integrated learning data sets 200 may be configured as integrated learning data.

The integrated learning data set 200 may be configured to include medical image data 210, first clinical data 220, and second clinical data 230. In addition, the integrated learning data set 200 includes an image reading result 215 including the lesion and the severity of the lesion for a specific area in the medical image, the first clinical reading including the lesion and the severity of the lesion for the first clinical data It may be configured to include a result 225, and a second clinical reading result 235 including the lesion and the severity of the lesion for the second clinical data, and the image reading result 215 and the first clinical reading result 225 ), and the second clinical reading result 235 may be configured to correspond to the medical image data 210, the first clinical data 220, and the second clinical data 230, respectively. In addition, the integrated learning data set 200 includes the final reading result data including the lesion and the severity of the lesion determined based on the medical image data 210, the first clinical data 220, and the second clinical data 230 ( 250) can be configured to include.

The image-based learning model 21 may perform learning by receiving medical image data 210 and receiving an image reading result 215 as a target variable. In addition, the first clinical data-based learning model 22 may receive the first clinical data 220 and receive the first clinical reading result 225 as a target variable to perform learning. Likewise, the second clinical data-based learning model 23 may perform learning by receiving the second clinical data 230 and receiving the second clinical reading result 235 as a target variable.

In addition, the integrated learning model 25 receives the image reading result 215, the first clinical reading result 225, the second clinical reading result 235, etc., and provides the final reading result data 250 as a target variable. Take it, you can perform learning. For example, the integrated learning model 25 generates each of the image readout result 215, the first clinical readout result 225, and the second clinical readout result 235 as one feature vector, and targets each of the feature vectors. You can create an ensemble model that trains a class for a variable.

The image-based learning model (21), the first clinical data-based learning model (22), the second clinical data-based learning model (23), and the integrated learning model (25) are a combination of a plurality of integrated learning data sets (200). The learning model can be built by performing learning on the integrated learning data.

In an embodiment of the present disclosure, it has been illustrated that learning of the above-described learning model is performed using the integrated learning data set 200, but the present disclosure is not limited thereto. As another example, medical image data 210, first clinical data 220, second clinical data 230, image reading result 215, first clinical reading result 225, second clinical reading result 235 ), etc., and includes the image reading result 215, the first clinical reading result 225, the second clinical reading result 235, and the final reading result data 250 A second data set 270 may be configured. Furthermore, the second data set 270 may be a validation set used for verification, or a new data set having information on the same characteristics and target variables as the first data set 260.

Further, in the above-described embodiment, using the integrated learning data set 200, the image-based learning model 21, the first clinical data-based learning model 22, the second clinical data-based learning model 23, and Although it is illustrated that learning of the integrated learning model 25 is simultaneously performed, the present disclosure is not limited thereto. For example, learning about the image-based learning model 21, the first clinical data-based learning model 22, and the second clinical data-based learning model 23 is performed using the first data set 260, and By performing the learning of the integrated learning model 25 using the two data set 270, the learning of the integrated learning model 25 may be separately performed.

Referring to FIG. 3, the image-based learning unit 30 includes a lesion area detection unit 31, an image-based severity learning unit 32a, 32b, and 32c, and a plurality of image-based severity learning units 32a, 32b, and 32c. It may include an image-based integrated learning unit 35 for ensemble-learning each output image-based lesion prediction result. Here, it is preferable that the plurality of image-based severity learning units 32a, 32b, and 32c each have a different learning structure.

Based on this, the lesion area detection unit 31 can receive a medical image (hereinafter referred to as'original medical image') photographing the user's body, and a medical image extracted from the original medical image (hereinafter,'' Lesion area image) can be detected. In addition, the lesion area detection unit 31 may detect the lesion area image and provide the detection of the lesion area image as an input of the image-based severity learning units 32a, 32b, and 32c.

The operation of extracting the lesion area image from the original medical image by the lesion area detection unit 31 may be performed based on a convolutional neural network (CNN) technique or a pooling technique. For example, the lesion region detection unit 31 may include a lesion region detection learning model 310 through learning that takes an original medical image as an input and outputs the lesion region image. In addition, as the original medical image is input, the lesion region detection unit 31 inputs the original medical image into the lesion region detection learning model 310 and generates a lesion region image corresponding thereto through the lesion region detection learning model 310. Can be detected.

Furthermore, medical images may be selectively used based on the characteristics of each body organ or diagnosis region, or lesions present in the body organ or diagnosis region.

For example, when a body organ or diagnosis region is a prostate region, the image-based learning unit 30 may use a T2-weighted (T2) image, an apparent diffusion coefficients (ADC) image, and the like as an input of the lesion region detection unit 31. . As another example, when a body organ or diagnosis region is a liver region, the image-based learning unit 30 may use a STIR image, a T1 image, a T1 with Agents image, and a T2 image as an input of the lesion region detection unit 31. . As another example, when a body organ or a diagnosis region is a brain region, the image-based learning unit 30 may use a T1 image, a T2 image, or a FLAIR as an input to the lesion region detection unit 31.

On the other hand, the image-based severity learning unit (32a, 32b, 32c) can learn the image-based severity learning model (320a, 320b, 320c), can receive a lesion region image as an analysis target image for performing the learning. , Learning of the image-based severity learning models 320a, 320b, and 320c may be performed by labeling a specific object included in the lesion area image or a severity level for a specific area.

Furthermore, the image-based severity learning unit (32a, 32b, 32c) is based on a convolutional neural network (CNN) technique or a pooling technique, for the image-based severity learning model (320a, 320b, 320c). Learning can be carried out.

For example, the image-based severity learning models 320a, 320b, and 320c may analyze an input image to extract features of an image. The feature may be a local feature for each area of the image. The image-based severity learning models 320a, 320b, and 320c may extract features of an input image using a general convolutional neural network (CNN) technique or a pooling technique. The pooling technique may include at least one of a max pooling technique and an average pooling technique. However, the pooling technique referred to in the present disclosure is not limited to the max pooling technique or the average pooling technique, and includes any technique for obtaining a representative value of an image region of a predetermined size. For example, the representative value used in the pooling technique may be at least one of a variance value, a standard deviation value, a mean value, a most frequent value, a minimum value, and a weighted average value, in addition to the maximum value and the average value.

The convolutional neural network of the present disclosure may be used to extract "features" such as a border, a line color, and the like from input data (images), and may include a plurality of layers. Each layer may receive input data and process the input data of the layer to generate output data. The convolutional neural network may output an input image or a feature map generated by convolving an input feature map with filter kernels as output data. The initial layers of the convolutional neural network can be operated to extract low-level features such as edges or gradients from the input. The next layers of the neural network can gradually extract more complex features such as eyes, nose, etc.

The convolutional neural network may include a pooling layer in which a pooling operation is performed in addition to a convolutional layer in which a convolution operation is performed. The pooling technique is a technique used to reduce the spatial size of data in the pooling layer. Specifically, the pooling technique includes a max pooling technique that selects the maximum value in a corresponding area and an average pooling technique that selects an average value of the area. In the image recognition field, the max pooling technique is generally used. do. In the pooling technique, generally, the pooling window size and interval (stride) are set to the same value. Here, the stride refers to an interval to be moved when a filter is applied to input data, that is, an interval to which the filter is moved, and stride may also be used to adjust the size of output data.

In particular, the image-based severity learning models (320a, 320b, 320c) may be configured to build learning models of different structures, and the image-based lesion prediction results (321a, 320c) based on the learning models (320a, 320b, 320c), respectively. 321b, 321c) may be constructed to be output. Further, the image-based lesion prediction results 321a, 321b, and 321c provided by the learning models 320a, 320b, and 320c, respectively, may be configured to be provided as an input of the image-based integrated learning unit 35.

In response, the image-based integrated learning unit 35 receives the image-based lesion prediction results 321a, 321b, 321c, and learns the image-based integrated lesion prediction result as an output corresponding thereto. 350 may be included. In particular, the image-based integrated learning unit 35 targets a plurality of image-based lesion prediction results (321a, 321b, 321c) provided from the image-based severity learning models (320a, 320b, 320c) ensemble learning (Ensemble learning). Can be performed to construct an image-based integrated lesion prediction result.

Specifically, by setting the image-based lesion prediction results 321a, 321b, 321c as input data of the image-based integrated learning model 350, and setting a target variable corresponding to the lesion area image, the image-based integrated learning model 350 ), it is possible to build an ensemble model by learning weights for a plurality of image-based lesion prediction results 321a, 321b, 321c.

Hereinafter, the configuration and operation of the clinical data-based learning unit will be described in more detail with reference to FIGS. 4 and 5.

First, in an embodiment of the present disclosure, clinical data is detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change or a body fluid, urine, biopsy, etc. generated by the user's body. Although illustrated as data, the present disclosure is not limited thereto, and types of clinical data may be variously changed. In addition, the configuration of the clinical data-based learning unit is exemplified based on the type of clinical data described above, but the configuration of the clinical data-based learning unit may be variously changed according to the type of clinical data.

Hereinafter, in an embodiment of the present disclosure, the first clinical data-based learning unit exemplifies a configuration unit that receives a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change and performs learning of a learning model, The second clinical data-based learning unit exemplifies a component that performs learning of a learning model by receiving data detected from a body fluid, urine, biopsy, etc. generated by a user's body.

Referring to FIG. 4, the first clinical data-based learning unit 40 includes a noise filter unit 41 that removes noise from an input biosignal (eg, ECG, PPG, EMG, etc.), and a noise-removed biosignal. It may include a diagnostic section extraction unit 42 for extracting a diagnostic section to be used for learning or detection.

In addition, the first clinical data-based learning unit 40 may include a lesion signal learning unit 43. The lesion signal learning unit 43 is a first clinical data-based learning model (430-1, 430-2, ...430) that learns by using the biosignal of the diagnosis section as input data and using the lesion severity as a target variable. -n) may be included.

Since the first clinical data may be data configured in a sequential form, the first clinical data-based learning model (430-1, 430-2, ...430-n) is a recurrent neural network (RNN). ), it is possible to perform learning on the first clinical data.

The second clinical data-based learning unit 50 may include a data normalization unit 51 and a lesion data learning unit 52.

The second clinical data detected from bodily fluids, urine, biopsy, etc. generated by the user's body may be configured in various forms, and each may represent different values. Accordingly, the data normalization unit 51 may normalize the second clinical data configured in various forms.

The lesion data learning unit 52 may include a second clinical data-based learning model 520 for learning by using the normalized second clinical data as input data and using the lesion severity as a target variable. Preferably, the second clinical data-based learning model 520 may perform learning on the second clinical data based on a Feed-Foward Neural Nework (FFNN) method.

Referring to FIG. 6, the lesion diagnosis apparatus 60 may include an image-based detection unit 61, at least one clinical data-based detection unit 63, and an integrated diagnosis unit 65.

The image-based detection unit 61 may include at least one image-based learning model 610 that receives a medical image and outputs an image-based lesion prediction result corresponding thereto, and includes at least one image-based learning model 610. Through this, the probability of developing a specific disease in a specific area included in the medical image can be output as a lesion prediction result.

Further, medical images are images of the entire body or a specific diagnostic area through various imaging techniques, such as T2 (T2-weighted) images, ADC (apparent diffusion coefficients) images, STIR images, T1 images, and T1 with agents images. , And may include parametric MRI, X-ray images, CT images, such as FLAIR, and the like.

The clinical data-based detection unit 63 may include clinical data-based learning models 630-1 and 630-2 for receiving clinical data and outputting lesion prediction results. Here, the clinical data may include data detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change or a body fluid generated by the user's body, urine, biopsy, and the like. Based on this, the clinical data-based learning models 630-1 and 630-2 are detected from biosignals (eg, ECG, PPG, EMG, etc.) or body fluids, urine, biopsy, etc. generated by the user's body. The model may be trained to receive data and output a probability of developing a specific disease in response to the above-described clinical data as a lesion prediction result.

At least one of the above-described clinical data-based detection unit 63 may be provided in the lesion diagnosis device 60, and each clinical data-based detection unit 63 includes clinical data-based learning models 630-1 and 630-2. Each can be included.

Furthermore, the clinical data-based detection unit 63 may be configured to be classified according to the type of input data. For example, the clinical data-based detection unit 63 includes a first clinical data-based detection unit 63-1 that inputs a biological signal (eg, ECG, PPG, EMG, etc.), and data detected from body fluids, urine, biopsy, etc. A second clinical data-based detection unit 63-2 may be included as an input.

The clinical data-based detection unit 63 may be configured to check the type of input data, select the detection units 63-1 and 63-2 corresponding to the identified type, and provide the corresponding clinical data. As another example, biological signals (e.g., ECG, PPG, EMG, etc.) are input to the first clinical data-based detection unit 63-1, and data detected from bodily fluids, urine, biopsy, etc. are the second clinical data-based detection unit. It is also possible to provide a user interface so that the input can be entered as (63-2).

Meanwhile, the integrated diagnosis unit 65 may receive a lesion prediction result output from the image-based detection unit 61 and the clinical data-based detection unit 63, and confirm the final lesion prediction result as an output corresponding thereto. In particular, the integrated diagnosis unit 65 receives a plurality of lesion prediction results provided from the image-based detection unit 61 and the clinical data-based detection unit 63, and ensemble learning to provide a final lesion prediction result corresponding thereto. ) May include an integrated learning model 650.

The image-based learning model 610, clinical data-based learning models 630-1 and 630-2, and the integrated learning model 650 are learning models constructed by the lesion integrated learning device 10 of FIG. I can.

Referring to FIG. 7, the image-based detection unit 70 is output from the lesion area detection unit 71, the image-based severity detection units 72a, 72b, and 72c, and a plurality of image-based severity detection units 72a, 72b, and 72c, respectively. It may include an image-based integrated detection unit 75 that receives the image-based lesion prediction result and outputs an image-based integrated disease prediction result corresponding thereto. Here, it is preferable that the plurality of image-based severity detection units 72a, 72b, and 72c each have a different learning structure.

Medical images may be selectively used based on the characteristics of each body organ or diagnosis region or lesions present in the body organ or diagnosis region. For example, when the body organ or the diagnosis region is the prostate region, the image-based detection unit 70 may use a T2-weighted (T2) image, an apparent diffusion coefficients (ADC) image, or the like as an input of the lesion region detection unit 71. As another example, when a body organ or diagnosis region is a liver region, the image-based detection unit 70 may use a STIR image, a T1 image, a T1 with Agents image, and a T2 image as an input of the lesion region detection unit 71. As another example, when the body organ or the diagnosis region is a brain region, the image-based detection unit 70 may use a T1 image, a T2 image, or a FLAIR as an input to the lesion region detection unit 71.

Based on this, the lesion area detection unit 71 may include a lesion area detection learning model 710, and the lesion area detection learning model 710 is a medical image (hereinafter, referred to as' An original medical image') is received, and a medical image obtained by extracting a lesion area from the original medical image (hereinafter, referred to as a'lesion area image') may be detected. In addition, the lesion region detection unit 71 may detect the lesion region image and provide the detection of the lesion region image as an input of the image-based severity detection units 72a, 72b, and 72c.

The operation of detecting the lesion area image from the original medical image by the lesion area detection unit 71 may be performed based on a convolutional neural network (CNN) technique or a pooling technique. For example, the lesion region detection unit 71 may include a lesion region detection learning model 710 that receives an original medical image as an input and outputs the lesion region image.

The image-based severity detection units 72a, 72b, 72c may include image-based severity learning models 720a, 720b, and 720c, and the image-based severity learning models 720a, 720b, and 720c are convolutional neural networks. , CNN) technique or a pooling technique.

In particular, the image-based severity learning models 720a, 720b, and 720c may be composed of learning models having different structures, and even when the same lesion area image is input, different image-based lesion prediction results 721a are respectively used by different learning models. , 721b, 721c) can be output. The image-based lesion prediction results 721a, 721b, and 721c output as described above may be provided as an input of the image-based integrated detection unit 75.

The image-based integrated detection unit 75 may include an image-based integrated learning model 750 that receives image-based lesion prediction results 721a, 721b, and 721c and outputs an image-based integrated lesion prediction result.

For example, the lesion area detection learning model 710, the image-based severity learning models 720a, 720b, and 720c, the image-based integrated learning model 750, etc., are to be constructed by the image-based learning unit 30 of FIG. I can.

Referring to FIG. 8, the first clinical data-based detection unit 80 includes a noise filter unit 81 that removes noise from an input bio-signal (eg, ECG, PPG, EMG, etc.), and a lesion from the noise-removed bio-signal. It may include a diagnostic section extraction unit 82 for extracting a diagnostic section to be used for detection.

In addition, the first clinical data-based detection unit 80 may include a lesion signal detection unit 83. The lesion signal detection unit 83 may include a first clinical data-based learning model 830-1, 830-2, ... 830-n that receives a biosignal of a diagnosis section and outputs a lesion severity.

Since the first clinical data may be data composed in a sequential form, the first clinical data-based learning model (830-1, 830-2, ... 830-n) is a recurrent neural network (RNN). ) May be a model learned based on the method.

Furthermore, the first clinical data-based learning models 830-1, 830-2, ... 830-n may be models constructed by the clinical data-based learning unit of FIG. 4 described above.

9 is a block diagram showing a detailed configuration of a second clinical data-based learning unit provided in a lesion diagnosis apparatus according to an embodiment of the present disclosure.

The second clinical data-based learning unit 90 may include a data normalization unit 91 and a lesion data detection unit 92.

The second clinical data detected from bodily fluids, urine, biopsy, etc. generated by the user's body are composed of various types, and different values may be displayed according to each type. Accordingly, the data normalization unit 91 may normalize the second clinical data configured in various forms.

The lesion data detection unit 92 may include a second clinical data-based learning model 920 that receives normalized second clinical data and outputs a lesion severity. Preferably, the second clinical data-based learning model 920 may be a model built based on the FFNN (Feed-Foward Neural Nework) method, for example, built by the clinical data-based learning unit of FIG. It can be a model.

The method for learning integrated lesions according to an embodiment of the present disclosure may be performed by the apparatus for learning integrated lesions according to an embodiment of the present disclosure described above.

First, in step S1010, the integrated lesion learning apparatus may learn an image-based learning model that receives a medical image and outputs a lesion prediction result. In particular, the integrated lesion learning apparatus may include a plurality of image-based learning models, and a probability of developing a specific disease in a specific region included in a medical image may be set as a target variable of the image-based learning model. Here, the medical image is an image taken of the entire body or a specific diagnostic area through various imaging techniques, and is a T2 (T2-weighted) image, an ADC (apparent diffusion coefficients) image, a STIR image, a T1 image, and a T1 with agents image. And parametric MRI, X-ray images, CT images, such as FLAIR, and the like.

Specifically, the integrated lesion learning apparatus may receive a medical image (hereinafter referred to as'original medical image') photographing a user's body, and a medical image obtained by extracting a diagnosis area from the original medical image (hereinafter, referred to as'lesion area image). ') can be detected. The operation of extracting the lesion region image from the original medical image by the integrated lesion learning apparatus may be performed based on a convolutional neural network (CNN) technique or a pooling technique. For example, the integrated lesion learning apparatus may include a lesion region detection learning model through learning that takes an original medical image as an input and outputs a lesion region image. In addition, as the original medical image is input, the lesion integrated learning apparatus may input the original medical image into the lesion region detection learning model and detect a lesion region image corresponding thereto through the lesion region detection learning model.

For example, when a body organ or a diagnosis region is a prostate region, the integrated lesion learning apparatus may use a T2-weighted (T2) image, an apparent diffusion coefficients (ADC) image, or the like as an input of the lesion region detection learning model. As another example, when a body organ or a diagnosis region is a liver region, the apparatus for integrated lesion learning may use an STIR image, a T1 image, a T1 with Agents image, and a T2 image as an input of the lesion region detection learning model. As another example, when a body organ or a diagnosis region is a brain region, the integrated lesion learning apparatus may use a T1 image, a T2 image, or a FLAIR as an input of a lesion region detection learning model.

On the other hand, the integrated lesion learning device can learn an image-based severity learning model, which can receive a lesion region image as an image to be analyzed for performing the learning, and label the severity of a specific object or specific region included in the lesion region image. By (Labeling), learning of an image-based severity learning model can be performed.

Here, the image-based severity learning model may be learned based on a convolutional neural network (CNN) technique or a pooling technique. Specifically, the image-based severity learning model may extract features of an image by analyzing an input image. The feature may be a local feature for each area of the image. The image-based severity learning model can extract features of an input image using a general convolutional neural network (CNN) technique or a pooling technique. The pooling technique may include at least one of a max pooling technique and an average pooling technique. However, the pooling technique referred to in the present disclosure is not limited to the max pooling technique or the average pooling technique, and includes any technique for obtaining a representative value of an image region of a predetermined size. For example, the representative value used in the pooling technique may be at least one of a variance value, a standard deviation value, a mean value, a most frequent value, a minimum value, and a weighted average value, in addition to the maximum value and the average value.

The convolutional neural network of the present disclosure may be used to extract “features” such as a border, a line color, and the like from input data (image), and may include a plurality of layers. Each layer may receive input data and may generate output data by processing the input data of the layer. The convolutional neural network may output an input image or a feature map generated by convolving an input feature map with filter kernels as output data. The initial layers of the convolutional neural network can be operated to extract low-level features such as edges or gradients from the input. The next layers of the neural network can gradually extract more complex features such as eyes, nose, etc.

The convolutional neural network may include a convolutional layer in which a convolution operation is performed, as well as a pooling layer in which a pooling operation is performed. The pooling technique is a technique used to reduce the spatial size of data in the pooling layer. Specifically, the pooling technique includes a max pooling technique that selects a maximum value in a corresponding domain and an average pooling technique that selects an average value of the domain. In the image recognition field, the max pooling technique is generally used. do. In the pooling technique, in general, the window size and interval (stride) of the pooling are set to the same value. Here, the stride refers to an interval to be moved when a filter is applied to the input data, that is, an interval to which the filter is moved, and the stride may also be used to adjust the size of the output data.

In particular, at least one image-based severity learning model may be configured such that learning models having different structures are constructed. And, the integrated lesion learning device builds a learning model that constructs the image-based integrated lesion prediction result by performing ensemble learning on at least one image-based lesion prediction result provided from the image-based severity learning model. can do.

Specifically, by setting the image-based lesion prediction result as input data of the image-based integrated learning model, and by setting the objective variable corresponding to the lesion area image to perform learning of the image-based integrated learning model, multiple image-based lesion prediction An ensemble model can be built by learning the weights for the results.

Meanwhile, in step S1020, the integrated lesion learning apparatus may learn at least one clinical data-based learning model that receives clinical data and outputs a lesion prediction result. Here, the clinical data may include data detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change or a body fluid generated by the user's body, urine, biopsy, and the like. Based on this, the clinical data-based learning model receives data detected from biological signals (e.g., ECG, PPG, EMG, etc.) or body fluids, urine, biopsy, etc. generated by the user's body, and the above-described clinical data Corresponding to, the probability of developing a specific disease may be set as an objective variable of a clinical data-based learning model. Accordingly, the clinical data-based learning model may be trained to output a lesion prediction result corresponding to the above-described clinical data.

In one embodiment of the present disclosure, clinical data is data detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change, or a body fluid generated by the user's body, urine, biopsy, etc. However, the present disclosure is not limited thereto, and the type of clinical data may be variously changed. In addition, the configuration of the clinical data-based learning unit is exemplified based on the type of clinical data described above, but the configuration of the clinical data-based learning unit may be variously changed according to the type of clinical data.

Hereinafter, in an embodiment of the present disclosure, the first clinical data is an example of a biological signal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change, and the second clinical data is a bodily fluid generated by the user's body. , Urine, biopsy, and the like, and the operation of learning at least one clinical data-based learning model based on this is illustrated in more detail.

First, the integrated lesion learning apparatus may build a learning model based on the first clinical data that learns the first clinical data. To this end, the integrated lesion learning device can remove noise from the first clinical data, that is, bio-signals (e.g., ECG, PPG, EMG, etc.), and extract a diagnostic section to be used for learning or detection from the noise-removed bio-signals. have.

In addition, the integrated lesion learning apparatus may construct a first clinical data-based learning model that learns by using the biosignal of the diagnosis section as input data and the lesion severity as a target variable. Since the first clinical data may be data configured in a sequential form, the first clinical data-based learning model can perform learning on the first clinical data based on the RNN (Recurrent Neural Network) method. have.

In addition, the integrated lesion learning apparatus may build a second clinical data-based learning model that performs learning on the second clinical data.

The second clinical data, that is, the second clinical data detected from the body fluid generated by the user's body, urine, biopsy, etc., are configured in various forms, and may each represent different values. Accordingly, the integrated lesion learning apparatus may normalize the second clinical data configured in various forms.

Thereafter, the integrated lesion learning apparatus may perform learning on a learning model based on the second clinical data that learns by using the normalized second clinical data as input data and using the lesion severity as a target variable. Preferably, the second clinical data-based learning model may be learned based on a Feed-Foward Neural Nework (FFNN) method.

Meanwhile, in step S1030, the integrated lesion learning apparatus may receive the lesion prediction result provided in steps S1010 and S1020, and as an output corresponding thereto, an integrated learning model for learning the final lesion prediction result may be constructed. In particular, the integrated learning model uses a plurality of lesion prediction results provided in steps S1010 and S1020 as input data, and outputs the same target variables as those set when learning the image-based learning model and the clinical data-based learning model. Can be set to Accordingly, the integrated learning model can build an ensemble model by learning weights between the outputs of the image-based learning model and the clinical data-based learning model.

A method for diagnosing a lesion according to an embodiment of the present disclosure may be performed by the apparatus for diagnosing a lesion.

Referring to FIG. 11, in step S1110, the apparatus for diagnosing a lesion may receive a medical image and output an image-based lesion prediction result corresponding thereto. In this case, the apparatus for diagnosing lesions may output a probability of developing a specific disease in a specific region included in the medical image as an image-based lesion prediction result using at least one image-based learning model. Here, the medical image is an image taken of the entire body or a specific diagnostic area through various methods of imaging, and includes a T2 (T2-weighted) image, an ADC (apparent diffusion coefficients) image, a STIR image, a T1 image, and a T1 with agents image. , And may include parametric MRI, X-ray images, CT images, such as FLAIR, and the like.

Looking at the operation of step S1110 in more detail, first of all, the lesion diagnosis apparatus may have a lesion area detection learning model, and the lesion area detection learning model is a medical image (hereinafter, referred to as' An original medical image') is received, and a medical image obtained by extracting a lesion area from the original medical image (hereinafter, referred to as a'lesion area image') may be detected.

Further, a medical image may be selectively used based on the characteristics of a body organ or a diagnosis region, or a lesion existing in a body organ or diagnosis region. For example, when the body organ or the diagnosis region is the prostate region, the lesion diagnosis apparatus may select a T2-weighted (T2) image, an apparent diffusion coefficients (ADC) image, or the like as an input of at least one image-based learning model. As another example, when the body organ or the diagnosis region is the liver region, the lesion diagnosis apparatus may select a STIR image, a T1 image, a T1 with Agents image, and a T2 image as inputs of at least one image-based learning model. As another example, when a body organ or a diagnosis region is a brain region, the lesion diagnosis apparatus may select a T1 image, a T2 image, or a FLAIR as an input of at least one image-based learning model.

The operation of detecting the lesion region image from the original medical image by the lesion diagnosis apparatus may be performed based on a convolutional neural network (CNN) technique or a pooling technique. For example, the lesion diagnosis apparatus may include an image-based severity learning model, and the image-based severity learning model is a learning model built based on a convolutional neural network (CNN) technique or a pooling technique. I can.

In particular, the image-based severity learning model may consist of learning models of different structures, and even when the same lesion area image is input, different image-based lesion prediction results can be output by different learning models.

The lesion diagnosis apparatus may further include an image-based integrated learning model that receives the image-based lesion prediction result and outputs the image-based lesion prediction result, and provides the image-based integrated lesion prediction result through the image-based integrated learning model. Can be calculated.

In step S1120, the apparatus for diagnosing lesions may receive at least one clinical data and output at least one lesion prediction result corresponding thereto. Here, the clinical data may include data detected from a biosignal (eg, ECG, PPG, EMG, etc.) that measures a user's biological change or a body fluid generated by the user's body, urine, biopsy, and the like. Based on this, the lesion diagnosis apparatus receives data detected from a biological signal (e.g., ECG, PPG, EMG, etc.) or a body fluid generated by the user's body, urine, biopsy, etc., and responds to the aforementioned clinical data. Thus, a model trained to output the probability of developing a specific disease as a lesion prediction result may be included.

The clinical data-based learning model may be configured to be classified according to the type of input data, and the lesion diagnosis apparatus may operate to classify the type of input data and provide it to a clinical data-based learning model.

Specifically, the lesion diagnosis apparatus may remove noise from a biological signal (eg, ECG, PPG, EMG, etc.) input as first clinical data, and then extract a diagnostic section to be used for lesion detection from the noise-removed biological signal. In addition, the lesion diagnosis apparatus may include a first clinical data-based learning model (830-1, 830-2, ... 830-n) that receives a biosignal of the extracted diagnosis section and outputs a lesion severity. , Through this, a lesion prediction result based on the first clinical data can be output.

Since the first clinical data may be data configured in a sequential form, the first clinical data-based learning model may be a model trained based on a recurrent neural network (RNN) method.

Meanwhile, the second clinical data detected from bodily fluids, urine, biopsy, etc. generated by the user's body may be configured in various forms, and different values may be represented according to each form. Accordingly, the lesion diagnosis apparatus may normalize data input as second clinical data, that is, data detected from a body fluid generated by a user's body, urine, biopsy, and the like. Thereafter, the lesion diagnosis apparatus may output a lesion prediction result based on the second clinical data through a second clinical data-based learning model that receives the normalized second clinical data and outputs a lesion severity. In this case, the second clinical data-based learning model may be a model constructed based on a feed-forward neural network (FFNN) method.

In step S1130, the lesion diagnosis apparatus may calculate a final lesion prediction result by combining an image-based lesion prediction result, a lesion prediction result based on the first clinical data, and a lesion prediction result based on the second clinical data. The final lesion prediction result may be calculated through an integrated learning model built through ensemble learning. The integrated learning model may be a learning model constructed by the lesion integrated learning apparatus 10 of FIG. 1 described above.

Referring to FIG. 12, the computing system 1000 includes at least one processor 1100 connected through a bus 1200, a memory 1300, a user interface input device 1400, a user interface output device 1500, and a storage device. (1600), and a network interface (1700).

The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and/or the storage 1600. The memory 1300 and the storage 1600 may include various types of volatile or nonvolatile storage media. For example, the memory 1300 may include read only memory (ROM) and random access memory (RAM).

Accordingly, the steps of the method or algorithm described in connection with the embodiments disclosed herein may be directly implemented in hardware executed by the processor 1100, a software module, or a combination of the two. Software modules reside in storage media (i.e., memory 1300 and/or storage 1600) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM. You may. An exemplary storage medium is coupled to the processor 1100, which is capable of reading information from and writing information to the storage medium. Alternatively, the storage medium may be integral with the processor 1100. The processor and storage media may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. Alternatively, the processor and storage medium may reside as separate components within the user terminal.

The exemplary methods of the present disclosure are expressed as a series of operations for clarity of description, but this is not intended to limit the order in which steps are performed, and each step may be performed simultaneously or in a different order if necessary. In order to implement the method according to the present disclosure, the exemplary steps may include additional steps, other steps may be included excluding some steps, or may include additional other steps excluding some steps.

Various embodiments of the present disclosure are not intended to list all possible combinations, but to describe representative aspects of the present disclosure, and matters described in the various embodiments may be applied independently or may be applied in combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. For implementation by hardware, one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general purpose It may be implemented by a processor (general processor), a controller, a microcontroller, a microprocessor, or the like.

The scope of the present disclosure is software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause an operation according to the method of various embodiments to be executed on a device or computer, and such software or It includes a non-transitory computer-readable medium (non-transitory computer-readable medium) which stores instructions and the like and is executable on a device or a computer.

Claims

In the device for learning a learning model for lesion detection,

An image-based learning unit for learning at least one image-based learning model that receives a medical image and outputs an image-based lesion prediction result;

At least one clinical data-based learning unit for learning a clinical data-based learning model that receives clinical data and outputs a clinical data-based lesion prediction result;

And an integrated learning unit configured to ensemble learning an integrated learning model for receiving the image-based lesion prediction result and the clinical data-based lesion prediction result and outputting a final lesion prediction result.
According to claim 1

The image-based learning unit,

Receiving the medical image and outputting a lesion prediction result, a plurality of image-based learning models having different learning structures,

A lesion comprising an image-based integrated learning model for receiving the plurality of lesion prediction results output through the plurality of image-based learning models and ensemble learning the image-based lesion prediction results corresponding thereto. Integrated learning device.
According to claim 1

The image-based learning unit,

Further comprising a diagnosis region detection unit including a lesion region detection model for detecting and outputting the image of the lesion region in response to the input of the medical image,

The lesion area detection unit,

The integrated lesion learning apparatus, characterized in that providing the image of the lesion region as an input of the plurality of image-based learning models.
According to claim 1

The clinical data-based learning unit,

And at least one bio-signal-based learning model that receives a bio-signal as an input and outputs a result of digitizing the severity of the lesion.
According to claim 1

The clinical data-based learning unit,

A noise filter for removing noise from the bio-signal,

And a diagnostic section detection unit that detects a diagnostic section to be used for learning from the biological signal.
The method of claim 1,

The at least one bio-signal-based learning model,

Lesion integrated learning device, characterized in that the learning model based on the RNN (Recurrent Neural Network) method.
According to claim 1

The clinical data-based learning unit,

And at least one clinical information-based learning model that receives at least one clinical information acquired through a clinical trial as an input and outputs a result of quantifying the severity of the lesion.
According to claim 7

The clinical data-based learning unit,

And a data normalization unit that normalizes and provides at least one clinical information obtained through the clinical trial.
According to claim 1

The at least one clinical information-based learning model, Lesion integration learning apparatus, characterized in that the learning model based on the FFNN (Feed-Foward Neural Nework) method.
In the method of learning a learning model for lesion detection,

A process of learning at least one image-based learning model that receives medical images and outputs image-based lesion prediction results; and

A process of learning at least one bio-signal-based learning model for receiving a bio-signal and outputting a bio-signal-based lesion prediction result corresponding to the bio-signal;

A process of learning at least one clinical information-based learning model for receiving at least one clinical information acquired through a clinical trial and outputting a clinical data-based lesion prediction result corresponding to the at least one clinical information,

And ensemble learning an integrated learning model for receiving the image-based lesion prediction result, the biosignal-based lesion prediction result, and the clinical data-based lesion prediction result, and outputting the final lesion prediction result. Integrative learning method for lesions.
In the device for diagnosing a lesion using a learning model that has learned the lesion,

An image-based prediction unit that outputs an image-based lesion prediction result corresponding to an input medical image using at least one image-based learning model;

At least one clinical data-based prediction unit for outputting a clinical data-based lesion prediction result corresponding to the input clinical data, using a clinical data-based learning model,

And an integrated diagnosis unit configured to input the image-based lesion prediction result and the clinical data-based lesion prediction result into an integrated learning model, and check a final lesion prediction result output through the integrated learning model.
The method of claim 11

The image-based prediction unit,

Receiving the medical image and outputting a lesion prediction result, a plurality of image-based learning models having different learning structures,

And an image-based integrated learning model for receiving the plurality of lesion prediction results output through the plurality of image-based learning models and outputting the image-based lesion prediction results.
The method of claim 11

The image-based prediction unit,

Further comprising a lesion region detection unit including a lesion region detection model for detecting and outputting the image of the lesion region in response to the input of the medical image,

The lesion area detection unit,

And providing the image of the lesion region as an input of the plurality of image-based learning models.
The method of claim 11

The clinical data-based prediction unit,

A lesion diagnosis apparatus comprising: at least one bio-signal-based learning model that receives a bio-signal as an input and outputs a result of quantifying the severity of the lesion.
The method of claim 11

The clinical data-based prediction unit,

A noise filter to remove noise from the bio-signal,

And a diagnostic section detector for detecting a diagnostic section to be used for learning from the biological signal.
The method of claim 11

The at least one bio-signal-based learning model,

A lesion diagnosis apparatus, characterized in that it is a learning model based on a recurrent neural network (RNN) method.
The method of claim 11

The clinical data-based prediction unit,

A lesion diagnosis apparatus comprising: at least one clinical information-based learning model for inputting at least one clinical information acquired through a clinical trial and outputting a result of digitizing the severity of the lesion.
The method of claim 17

The clinical data-based prediction unit,

And a data normalization unit that normalizes and provides at least one clinical information acquired through the clinical trial.
The method of claim 11

The at least one clinical information-based learning model,

Lesion integrated learning device, characterized in that the learning model based on the FFNN (Feed-Foward Neural Nework) method.
In a method for diagnosing a lesion using a learning model that has learned the lesion,

The process of outputting an image-based lesion prediction result corresponding to an input medical image using an image-based learning model, and

The process of confirming the result of the biosignal-based lesion prediction corresponding to the input of the biosignal using the biosignal-based learning model, and

The process of confirming the clinical data-based lesion prediction result corresponding to at least one clinical information acquired through a clinical trial by using a clinical information-based learning model, and

And inputting the image-based lesion prediction result, the biosignal-based lesion prediction result, and the clinical data-based lesion prediction result into an integrated learning model, and confirming a final lesion prediction result output through the integrated learning model. How to diagnose lesions by using.