KR102530010B1 - Apparatus and method for determining disease severity based on medical image - Google Patents

Abstract

An apparatus and method for determining disease severity based on medical images according to an embodiment of the present invention are provided. An apparatus for determining severity of a disease based on medical images according to an embodiment of the present invention includes a communication unit configured to transmit and receive data; and a control unit configured to be connected to the communication unit, wherein the control unit obtains a plurality of medical images obtained by photographing a target part of the subject in different directions from a photographing device through the communication unit, and obtains a plurality of medical images obtained from the plurality of medical images Obtain result data obtained by predicting the target region from each medical image using a prediction model trained to predict the region corresponding to the target region, and learn to classify disease severity of the target region based on the result data. and determine the severity of the disease using a classified model.

Description

Medical image-based disease severity determination apparatus and method {APPARATUS AND METHOD FOR DETERMINING DISEASE SEVERITY BASED ON MEDICAL IMAGE}

The present invention relates to an apparatus and method for determining the severity of a disease based on medical images.

In general, hospital treatment is performed in a manner in which a patient visits a specialist to receive medical treatment, receives a test for diagnosing a disease, and receives a result thereof from a specialist.

For example, in order to diagnose sinusitis occurring in the sinuses, an x-ray examination is performed on the patient, and a specialist uses the x-ray image to determine the level of opacity, air, and/or liquid in the sinuses. Sinusitis can be diagnosed.

However, since it is difficult for specialists to distinguish sinusitis with the naked eye when using X-ray images, there may be deviations in image interpretation depending on specialists, and accordingly, it may be difficult for specialists to quickly and accurately diagnose diseases.

Therefore, there is a need for an apparatus and method for determining the severity of a disease based on medical images for a specialist to promptly and accurately diagnose a disease in a target region of a patient.

Accordingly, an object to be solved by the present invention is to provide an apparatus and method for determining the severity of a disease based on medical images.

Specifically, an object to be solved by the present invention is to provide an apparatus and method for determining the severity of a disease based on medical images for promptly and accurately diagnosing a disease in a target region.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, an apparatus and method for determining disease severity based on medical images are provided.

An apparatus for determining severity of a disease based on medical images according to an embodiment of the present invention includes a communication unit configured to transmit and receive data; and a control unit configured to be connected to the communication unit, wherein the control unit obtains a plurality of medical images obtained by photographing a target part of the subject in different directions from a photographing device through the communication unit, and obtains a plurality of medical images obtained from the plurality of medical images Obtain result data obtained by predicting the target region from each medical image using a prediction model trained to predict the region corresponding to the target region, and learn to classify disease severity of the target region based on the result data. and determine the severity of the disease using a classified model.

A method for determining the severity of a disease based on medical images performed by a control unit of an apparatus for determining severity of a disease based on medical images according to an embodiment of the present invention includes obtaining a plurality of medical images of a target part of an examinee from different directions from a photographing device. step; obtaining data as a result of predicting a region corresponding to the target region from each medical image using a prediction model learned to predict a region corresponding to the target region based on the plurality of medical images; and determining the severity of the disease by using a classification model learned to classify the severity of the disease in the target region based on the resulting data.

Other embodiment specifics are included in the detailed description and drawings.

The present invention determines the severity of a disease using an artificial neural network based on a plurality of medical images taken from multiple viewpoints of a subject's target area, so that a medical expert such as a specialist can more quickly and accurately diagnose a subject's disease, Time and resources required for diagnosing a subject's disease can be minimized.

In addition, the present invention can improve the accuracy and reliability of diagnosis results by minimizing misdiagnosis of diseases by using an artificial neural network for disease diagnosis.

In addition, the present invention can diagnose a disease of a subject's target site at an early stage, enabling rapid and appropriate response to the disease.

Effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is schematic diagrams for explaining a disease severity determination system according to an embodiment of the present invention.
2 is a schematic block diagram of an electronic device according to an embodiment of the present invention.
3 is an exemplary diagram for explaining a method of detecting a target region using a plurality of artificial neural network models and determining the severity of a disease of the target region according to an embodiment of the present invention.
4 is an exemplary diagram for explaining the operation of a predictive model used to detect a target region in a plurality of medical images according to an embodiment of the present invention.
5 is an exemplary diagram for explaining the operation of a classification model used to classify the severity of a disease of a target site according to an embodiment of the present invention.
6 is a flowchart illustrating a method for determining severity of disease based on medical images in an electronic device according to an embodiment of the present invention.
7 are diagrams showing evaluation results for a plurality of artificial neural network models used in an embodiment of the present invention.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and will be implemented in various forms different from each other, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like elements.

In this document, expressions such as "has," "may have," "includes," or "may include" indicate the existence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this document, expressions such as “A or B,” “at least one of A and/and B,” or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B” (1) includes at least one A, (2) includes at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as “first,” “second,” “first,” or “second,” used in this document may modify various elements, regardless of order and/or importance, and refer to one element as It is used only to distinguish it from other components and does not limit the corresponding components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, without departing from the scope of rights described in this document, a first element may be named a second element, and similarly, a second element may also be renamed to a first element.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component). On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, third components) do not exist between the other components.

As used in this document, the expression "configured to" means "suitable for," "having the capacity to," depending on the circumstances. ," "designed to," "adapted to," "made to," or "capable of." The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware. Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (e.g., embedded processor) to perform those operations, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe a specific embodiment, and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted as having the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, an ideal or excessively formal meaning. not be interpreted as In some cases, even terms defined in this document cannot be interpreted to exclude the embodiments of this document.

Each feature of the various embodiments of the present invention can be partially or entirely combined or combined with each other, and as those skilled in the art can fully understand, various interlocking and driving operations are possible, and each embodiment can be implemented independently of each other. It may be possible to implement together in an association relationship.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is schematic diagrams for explaining a disease severity determination system according to an embodiment of the present invention.

Referring to FIG. 1 , a disease severity determination system 100 is a system for determining disease severity by using a plurality of medical images taken from different directions of a target part of a subject. It includes a photographing device 110 and an electronic device 120 that determines the severity of a disease by using a plurality of medical images provided from the photographing device 110 .

Here, the medical image is an image of a target part of the subject, and may be an X-ray image, but is not limited thereto. The target site is a specific body part of the subject for which the presence and/or severity of the disease is to be predicted, such as the paranasal sinus, spinal cord, kidney, mouth, lips, throat, oral cavity, nasal cavity, small intestine, colon, parathyroid gland, gallbladder, head and neck, and breast. , bone, bile duct, cervix, heart, hypopharyngeal gland, lung, bronchus, liver, skin, ureter, urethra, testicles, vagina, anus, laryngeal gland, ovary, thyroid, esophagus, nasopharyngeal gland, pituitary gland, salivary gland, prostate, pancreas , adrenal glands, lymph nodes, spleen, brain, varicose veins, and/or musculoskeletal system, but is not limited thereto, and may be various parts that can be obtained as images by an X-ray imaging device or the like. In the presented embodiment, the target site is described as a paranasal sinus.

First, the photographing apparatus 110 may be an X-ray apparatus for providing a plurality of medical images obtained by photographing a target part of an examinee in different directions. For example, the different directions may be multi-view directions for capturing the front, side, and/or back of the target part, but are not limited thereto.

Next, the electronic device 120 obtains a plurality of medical images from the photographing device 110, and based on the plurality of medical images, the electronic device 120 uses at least one of a tablet PC, a laptop computer, and/or a PC for determining the severity of a disease of a target part. can be one

Specifically, the electronic device 120 may detect a region corresponding to the target region (hereinafter, referred to as 'target region') based on a plurality of medical images, and determine the severity of the disease based on the detected target region. there is.

In this way, the electronic device 120 may use an artificial neural network model to detect the target region based on the plurality of medical images and determine the severity of the disease based on the detected target region. Specifically, the electronic device 120 determines the disease severity based on a first artificial neural network model learned to detect a target region based on a plurality of medical images and the target region detected through the first artificial neural network model. A second artificial neural network model learned to do so may be used.

In the presented embodiment, the case where the photographing device 110 and the electronic device 120 are implemented as separate devices has been described, but is not limited thereto, and the photographing device 110 and the electronic device 120 may be implemented as one device. may be

In the following, the electronic device 120 will be described in more detail with reference to FIG. 2 .

2 is a schematic block diagram of an electronic device according to an embodiment of the present invention.

Referring to FIG. 2 , the electronic device 200 includes a communication unit 210, a storage unit 220, a display unit 230, and a control unit 240. In various embodiments, the display unit 230 may be selectively provided. In the presented embodiment, the electronic device 200 may mean the electronic device 120 of FIG. 1 .

The communication unit 210 connects the electronic device 200 to enable communication with an external device. The communication unit 210 may be connected to the photographing device 110 using wired/wireless communication to transmit/receive various data for determining the severity of a disease. Specifically, the communication unit 210 may receive a plurality of medical images from the photographing device 110 .

The storage unit 220 may store various data used to determine the severity of a disease based on a plurality of medical images.

In various embodiments, the storage unit 220 may be a flash memory type, a hard disk type, a multimedia card micro type, or a card type memory (eg SD or XD). memory, etc.), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The electronic device 200 may operate in relation to a web storage that performs the storage function of the storage unit 220 on the Internet.

The display unit 230 may display various contents to the user. For example, the display unit 230 may display a plurality of medical images or information about the determined severity of a disease.

In various embodiments, the display unit 230 may include a touch screen, and for example, a touch using an electronic pen or a part of the user's body, a gesture, a proximity, a drag, or a swipe A swipe or hovering input may be received.

The control unit 240 is operatively connected to the communication unit 210, the storage unit 220, and the display unit 230, and may execute various commands for determining the severity of a medical image-based disease.

The control unit 240 includes at least one of a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a digital signal processing unit (DSP), an arithmetic logic unit (ALU), and an artificial neural network processor (NPU). It can be configured to include.

In detail, the controller 240 acquires a plurality of medical images from the photographing device 110 through the communication unit 210, and based on the plurality of medical images, the controller 240 is trained to predict (or recognize) a target region of the subject. Data obtained as a result of predicting a target region from each medical image using a model may be obtained. The controller 240 may obtain result data obtained by classifying the disease severity by using a classification model learned to classify the disease severity of the target site based on the acquired result data.

In the following, the operation of the controller 240 described above will be described in detail with reference to FIGS. 3 to 5 .

3 is an exemplary diagram for explaining a method of detecting a target region using a plurality of artificial neural network models and determining the severity of a disease of the target region according to an embodiment of the present invention. Operations described below in the presented embodiment may be performed by the controller 240 of FIG. 2 .

Referring to FIG. 3 , the controller 240 predicts the target region from the plurality of medical images 300 by using the prediction model 305 trained to predict the target region by inputting the plurality of medical images 300 . Prediction result data 320, which is one result data, may be obtained. Here, the plurality of medical images 300 include a first medical image 302 capturing the front of the examinee's face, a second medical image 304 capturing the back side of the examinee's face, and a third medical image 304 capturing the side of the examinee's face. An image 306 may be included.

The prediction model 310 may be an artificial neural network model trained to predict a target region based on the plurality of medical images 300 .

Specifically, the prediction model 310 may be an artificial neural network model configured to learn a plurality of reference images in advance and predict a target region from a plurality of newly input medical images 300 . Here, the plurality of reference images may be medical images obtained by capturing target parts of various subjects from a plurality of different viewpoints, but are not limited thereto. For example, the reference image may be a medical image obtained by capturing a target region without disease and a target region with disease.

This predictive model 310 may be a pretrained Convolutional Neural Network (CNN), but is not limited thereto. The pretrained convolutional neural network may include one or more layers that perform convolution operations on input values, and may infer an output value by performing a convolution operation from the input values. For example, the pretrained convolutional neural network may be a RetinaNet including a backbone network and two subnetworks, but is not limited thereto.

In various embodiments, in order to learn the predictive model 310, 85% of reference images among all reference images may be used for learning and 15% of reference images may be used for verification. The hyper-parameters set to learn the predictive model include the batch size, which indicates the number of data to be trained at once, the epoch, which indicates the number of times the entire data is repeatedly learned, and the learning rate. ) and an input size indicating the size of the input image. For example, the optimal batch size is 128, the optimal epoch is 200 times, the optimal learning rate is 0.0001, and the optimal input size is It may be 256 (pixel) x 256 (pixel), but is not limited thereto, and these hyperparameter values may be set to values to improve learning performance.

In the following, the operation of the predictive model 310 will be described in detail with reference to FIG. 4 .

4 is an exemplary diagram for explaining the operation of a predictive model used to detect a target region in a plurality of medical images according to an embodiment of the present invention. In the presented embodiment, the medical image is limited to an X-ray image of the front, side, and back of the subject's face, and the target site is limited to the paranasal sinuses (two maxillary sinuses, two frontal sinuses, two ethmoid sinuses, and two sphenoid sinuses). let it do

Referring to FIG. 4 , the predictive model 400 may include a plurality of artificial neural networks trained to predict a target region by using a medical image 410 as an input. The plurality of artificial neural networks may include a first artificial neural network 420 , a second artificial neural network 430 and a third artificial neural network 440 .

The first artificial neural network 420 may be a Residual Network (ResNet) that generates intermediate feature maps for each layer by taking the medical image 410 as an input, but is not limited thereto. .

The second artificial neural network 430 is a feature pyramid network (Feature Pyramid Network) that generates feature data corresponding to each layer by merging intermediate feature data of a specific layer and intermediate feature data of the next layer in order from the upper layer to the lower layer. Pyramid Network, FPN), but is not limited thereto. Here, the layer corresponding to the feature data may mean each layer of RESNET.

The third artificial neural network 440 includes a bounding box regression subnet for adjusting a bounding box to include a target site and a classification subnet for predicting (or classifying) the type of target site included in the bounding box. subnet) may be included. These subnets can be configured in parallel.

The operation of adjusting the bounding box may be performed using an offset between a ground-truth bounding box (i.e., correct answer) representing a target region in an actual medical image and a predicted bounding box. . Here, the offset may mean the degree to which the predicted bounding box and the ground-truth bounding box corresponding to the correct answer overlap (or coincide) with each other. Through this, the performance of the predictive model can be evaluated.

The predicted result data 450 output through the predictive model 400 includes the predicted type of target region (eg, left/right maxillary sinus, left/right frontal sinus, left/right ethmoid sinus, and/or left/right sphenoid sinus) and purpose A bounding box corresponding to the region may be included. In various embodiments, the prediction result data 450 may further include scores for predicted types. For example, the prediction result data 450 includes the first prediction result image 452 including the type and bounding box of the target region predicted in the first medical image 412 and the predicted result data 452 in the second medical image 414. The second prediction result image 454 including the target region type and the bounding box and the third prediction result image 456 including the target region type and the bounding box predicted in the third medical image 416 may be included. can Here, the prediction result data 450 may mean the prediction result data 320 described in FIG. 3 .

The control unit 240 may generate crop data 460 obtained by cropping a target region in the medical image 410 based on the output prediction result data 450 . For example, the cropped data 460 is a first cropped image 462 obtained by cropping the sinus area from the first prediction result image 452 and a second cropped image obtained by cropping the sinus area from the second prediction result image 454. 464 and a third cropped image 466 obtained by cropping the sinus area in the third prediction result image 456. In various embodiments, the controller 240 converts the cropped image into a square by performing zero padding based on a larger value among rows x rows of the cropped image, and transforms the cropped image into a square. It can be resized to a specific size. Through this, the resolution of the cropped image may be improved.

Referring back to FIG. 3 , the control unit 240 classifies the disease severity in the target region by using the classification model 330 learned to classify the disease severity in the target region by inputting the obtained prediction result data 320. Classification result data 340, which is one result data, may be obtained.

The classification model 330 may be an artificial neural network model trained to classify the severity of a disease in a target region based on the prediction result data 320 .

Specifically, the classification model 330 may be an artificial neural network model configured to learn a plurality of reference images in advance and classify disease severity from newly input prediction result data 320 . Here, the plurality of reference images may be cropped images obtained by cropping a target region from various medical images, but, for example, the reference images may include a cropped image including a disease and various cropped images not including a disease. In various embodiments, cropped images including diseases may be cropped images corresponding to different severities of diseases (eg, asymptomatic, mild, severe, and/or severe). This severity includes the degree of inflammation, and the degree of inflammation may be classified according to the size, color, and/or shape of the inflammation shown in the medical image. In various embodiments, the classification model 330 may further learn not only the reference image but also classification answer data (eg, the actual disease severity of the corresponding reference image).

This classification model 330 may be a pretrained convolutional neural network, but is not limited thereto. For example, the pretrained convolutional neural network may be a Multi-view Convolutional Neural Network (MVCNN) or a Multi-Planar CNN (MPCNN), but is not limited thereto.

In the following, the operation of the classification model 330 will be described in detail with reference to FIG. 5 .

5 is an exemplary diagram for explaining the operation of a classification model used to classify the severity of a disease of a target site according to an embodiment of the present invention.

Referring to FIG. 5 , the classification model 500 is composed of a plurality of artificial neural networks trained to classify the disease severity of the target region by inputting crop data 510, which is prediction result data output through the prediction model 310. can Here, the cropped data 510 may include a first cropped image 512 , a second cropped image 514 , and a third cropped image 516 .

The plurality of artificial neural networks include a plurality of convolutional neural networks (eg, a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network) 520, a pooling layer 540, and a fourth synthesis. A product neural network 560 may be included.

The plurality of convolutional neural networks 520 may output feature data 530 by taking the crop data 510 as an input.

Specifically, the plurality of convolutional neural networks 520 take the first cropped image 512 as an input, the first convolutional neural network 522 outputting the first feature data 532, and the second cropped image 514 as inputs. The second convolutional neural network 524 outputs the second feature data 534 and the third convolutional neural network 526 outputs the third feature data 536 with the third cropped image 516 as an input. can include Here, the number of convolutional neural networks may match the number of input images.

The pooling layer 540 is a view-pooling layer that generates one integrated feature data by aggregating the first feature data 532, the second feature data 534, and the third feature data 536. layer), but is not limited thereto.

The fourth convolutional neural network 560 may be an artificial neural network that has been trained to classify the severity of a disease in a target region using integrated feature data as an input. When the target site region is a paranasal sinus, the target site region may include at least one of regions corresponding to a left maxillary sinus, a right maxillary sinus, a left anterior sinus, a right anterior sinus, a left ethmoid sinus, a right ethmoid sinus, a left sphenoid sinus, and a right sphenoid sinus.

The fourth convolutional neural network 560 assigns at least one of the left maxillary sinus, the right maxillary sinus, the left anterior sinus, the right anterior sinus, the left ethmoid sinus, the right ethmoid sinus, the left sphenoid sinus, and the right sphenoid sinus to n labels corresponding to disease severity. can be classified as (n>0). For example, if the disease of the sinuses is sinusitis, the severity of the disease may correspond to the degree of inflammation of the sinusitis.

The classification result data 570 output through the fourth convolutional neural network 560 may include the type of label in which the target region is classified. For example, a classification result if n labels contain Inflammation Degree 0 (eg asymptomatic), Inflammation Degree 1 (eg Mild), Inflammation Degree 2 (eg Severe), and Inflammation Degree 3 (eg Severe Severe). In the data 570, at least one of the left maxillary sinus, the right maxillary sinus, the left anterior sinus, the right anterior sinus, the left ethmoid sinus, the right ethmoid sinus, the left sphenoid sinus, and the right sphenoid sinus corresponding to the input crop data 510 has an inflammation degree of 0 and inflammation It may be data as a result of classification as one of FIG. 1, degree of inflammation 2, and degree of inflammation 3. Here, the classification result data 570 may mean the classification result data 340 described in FIG. 3 .

Through this, the present invention determines the diagnosis severity of the target site using an artificial neural network, so that medical experts such as specialists can diagnose the subject's disease more quickly and accurately.

In the following, a method for determining the severity of a medical image-based disease in an electronic device will be described with reference to FIG. 6 .

6 is a flowchart illustrating a method for determining severity of disease based on medical images in an electronic device according to an embodiment of the present invention. Operations described below may be performed by the controller 240 of FIG. 2 .

Referring to FIG. 6 , the controller 240 acquires a plurality of medical images taken at different viewpoints of the subject's target part (S600). Here, the plurality of medical images may be X-ray images obtained by multi-viewing a target part of the subject.

The controller 240 obtains data as a result of predicting the target region from each medical image using a prediction model learned to predict the target region based on a plurality of medical images (S610). The target site region may include at least one sub-region. For example, when the target region region is a paranasal sinus, the at least one sub-region includes a left maxillary sinus, a right maxillary sinus, a left anterior sinus, a right anterior sinus, a left ethmoid sinus, and a right An area corresponding to at least one of the ethmoid sinus, the left sphenoid sinus, and the right sphenoid sinus may be included.

Specifically, the controller 240 predicts the type of target region and a bounding box corresponding to the target region through a predictive model by taking a plurality of medical images as inputs, and predicts the predicted target region for each of the plurality of medical images. Prediction result data indicating the type of and the bounding box of the target region may be output. In this case, the bounding box of the target region may include a bounding box including at least one of a left maxillary sinus, a right maxillary sinus, a left anterior sinus, a right anterior sinus, a left ethmoid sinus, a right ethmoid sinus, a left sphenoid sinus, and a right sphenoid sinus.

The controller 240 determines the severity of the disease by using a classification model learned to classify the severity of the disease in the target region based on the obtained result data (S620).

Specifically, the control unit 240 generates a plurality of cropped images obtained by cropping a target region corresponding to a bounding box in the resulting data, and uses the generated plurality of cropped images as inputs to generate at least one sub-region through a classification model. It is possible to classify severity and output classification result data. For example, the classification result data indicates that each of at least one of the left maxillary sinus, right maxillary sinus, left anterior sinus, right anterior sinus, left ethmoid sinus, right ethmoid sinus, left sphenoid sinus, and right sphenoid sinus has an inflammation degree of 0, an inflammation degree of 1, an inflammation degree of 2, It may be the result data classified as one of inflammation degree 3.

The controller 240 provides data on the determined disease severity (S630). For example, the controller 240 may display an interface screen showing classification result data through the display unit 230 . The interface screen may include at least one graphic object representing a medical image, a target region detected in the medical image, and/or a disease severity of the target region.

In the following, evaluation results for 'a plurality of artificial neural network models used to determine the severity of diseases based on medical images' will be described with reference to FIG. 7 .

7 are diagrams showing evaluation results for a plurality of artificial neural network models used in an embodiment of the present invention.

Referring to (a) of FIG. 7 , accuracy, which is the ratio of accurately predicting true and false answers of the prediction model and classification model used to determine disease severity, among those classified as correct by the model, the actual A table showing scores for precision, which is the ratio of correct answers, and recall, which is the ratio of actual correct answers predicted by the model to be correct, is shown.

The accuracy score for determining disease severity was 0.908 ± 0.105, and the predictive model and classification model used for determining disease severity can be judged to have very high accuracy.

In addition, the precision score for disease severity determination was 0.845 ± 0.228, and the prediction model and classification model used for disease severity determination can be judged to have very high precision.

On the other hand, the recall score for disease severity determination is 0.774 ± 0.213, and the prediction model and classification model used for severity determination can be judged to have very high recall rates.

Referring to (b) of FIG. 7 , ROC showing the rate of classification as correct among actual incorrect answers (False Positive Rate, FPR) and the rate of classification as correct among actual correct answers (True Positive Rate, TPR) by the predictive model and classification model (Receiver Operating Characteristic) graph 700 is shown. For example, the x-axis can be represented by FRP and the y-axis can be represented by TRP.

The ROC graph 700 has a curve shape with a large TPR and a small FPR, and is located higher than the y=x graph 710 in the y-axis direction. In addition, AUC (Area Under Curve) representing an area value under the ROC graph 700 is 0.885. As the AUC value is closer to 1, the model may have a higher ability to classify disease severity. Thus, the predictive model and the classification model can be evaluated as having excellent performance for determining disease severity.

Through this, the present invention determines the severity of the disease of the target area using an artificial neural network based on a plurality of images taken from multi-viewpoints of the target area of the subject, thereby enabling early diagnosis of the disease and promptly coping with the disease.

In addition, according to the present invention, it is possible to diagnose and confirm the severity of diseases that are difficult to distinguish with the naked eye using medical images such as X-ray images.

In addition, the present invention can reduce the cost of diagnosis and increase the convenience of diagnosis and severity confirmation.

Devices and methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in computer readable media. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination.

Program instructions recorded on a computer readable medium may be specially designed and configured for the present invention or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - Includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media and ROM, RAM, flash memory, etc. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and may be variously modified and implemented without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. The protection scope of the present invention should be construed according to the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: disease severity determination system
110: shooting device
120, 200: electronic device
210: communication department
220: storage unit
230: display unit
240: control unit

Claims (14)

a communication unit configured to transmit and receive data; and
Including a control unit configured to connect with the communication unit,
The control unit,
Obtaining a plurality of medical images (the medical images are x-ray images) in which the front, side, and back surfaces of the face including the paranasal sinuses, which are the target part of the subject, are photographed from the photographing device through the communication unit;
Based on the plurality of medical images, a plurality of sub-target parts, left maxillary sinus, right maxillary sinus, left anterior sinus, right anterior right sinus, left ethmoid sinus, right ethmoid sinus, left sphenoid sinus, observable from the front, side, and rear surfaces of the paranasal sinus, which is the target part, and obtaining data as a result of predicting a region of a target region from each medical image using a prediction model learned to predict a region corresponding to the right sphenoid sinus;
configured to determine the severity of the disease by using a classification model learned to classify the severity of the disease for each region of the plurality of sub-target regions based on the result data;
The predictive model,
Using each of the plurality of medical images as an input, a class of the plurality of sub-target regions and a bounding box corresponding to the region of the plurality of sub-target regions are predicted, and each of the plurality of medical images Medical image-based disease severity determination apparatus configured to output prediction result data representing the predicted type and bounding box. delete delete delete The method of claim 1, wherein the control unit,
generating a cropped image obtained by cropping the bounding box from the prediction result data;
The generated cropped image is input as an input value of the classification model, a medical image-based disease severity determination device. The method of claim 5, wherein the classification model,
A plurality of artificial neural networks that output feature data by taking a plurality of cropped images as inputs, a pooling layer that outputs integrated feature data by integrating feature data output through the plurality of artificial neural networks, and the integrated feature data A medical image-based disease severity determination apparatus comprising an artificial neural network for outputting classification result data obtained by classifying the disease severity as an input. The method of claim 6, wherein the classification result data,
An apparatus for determining disease severity based on medical images, which is data obtained by classifying disease severity for at least one region of a sub-target region. In the method for determining the severity of a disease based on a medical image performed by a control unit of an apparatus for determining the severity of a disease based on a medical image,
acquiring a plurality of medical images of the front, side, and back surfaces of the subject's face including the paranasal sinuses, the medical images being X-ray images, from a photographing device;
Based on the plurality of medical images, a plurality of sub-target parts, left maxillary sinus, right maxillary sinus, left anterior sinus, right anterior right sinus, left ethmoid sinus, right ethmoid sinus, left sphenoid sinus, observable from the front, side, and rear surfaces of the paranasal sinus, which is the target part, and obtaining data as a result of predicting a region of a target region from each medical image using a prediction model learned to predict a region corresponding to the right sphenoid sinus; and
determining the severity of the disease by using a classification model learned to classify the severity of the disease for each of the plurality of sub-target regions based on the result data;
The predictive model,
Using each of the plurality of medical images as an input, a class of the plurality of sub-target regions and a bounding box corresponding to the region of the plurality of sub-target regions are predicted, and each of the plurality of medical images The medical image-based disease severity determination method configured to output prediction result data representing the predicted type and bounding box. delete delete delete According to claim 8,
Further comprising generating a cropped image obtained by cropping the bounding box from the prediction result data;
The generated cropped image is input as an input value of the classification model, the medical image-based disease severity determination method. The method of claim 12, wherein the classification model,
A plurality of artificial neural networks that take a plurality of cropped images as input and output feature data, a pooling layer that outputs integrated feature data by integrating feature data output through the plurality of artificial neural networks, and the integrated feature data A method for determining disease severity based on medical images, comprising an artificial neural network outputting classification result data obtained by classifying the severity of the disease as an input. The method of claim 13, wherein the classification result data,
A method for determining disease severity based on medical images, which is data obtained by classifying disease severity for at least one region of a sub-target region.

Images