KR20210028053A - Method and system for detecting pneumothorax - Google Patents

Abstract

A method for a computing device to detect a pneumothorax comprises the steps of: obtaining predicted pneumothorax information, predicted tube information, and predicted spine baseline for an input image from a learned pneumothorax prediction model; determining at least one representative pneumothorax position for the predicted pneumothorax information and at least one tube representative position for the predicted tube information from the predicted image in which the predicted pneumothorax information and the predicted tube information are displayed; dividing the predicted image into a first region and a second region by the predicted spine baseline; and determining a region in which the at least one representative pneumothorax representative position and the at least one tube representative position exist among the first region and the second region.

Description

Pneumothorax detection method and system {METHOD AND SYSTEM FOR DETECTING PNEUMOTHORAX}

The present invention relates to an artificial intelligence-based medical image reading technology.

Pneumothorax (pneumothorax) is a disease in which the lungs cannot be breathed normally when the air is filled in the pleural cavity surrounding the lungs, and symptoms of shortness of breath and chest pain appear. The cause of pneumothorax has not yet been accurately identified, there are no fundamental preventive measures, and even younger age groups without lung disease can develop a variety of causes.

Medical staff can diagnose a pneumothorax through X-ray images, and if the pneumothorax is large in size or symptoms are severe, a procedure to discharge air by inserting a thin tube into the chest can be performed. Pneumothorax, an emergency disease, can relieve the patient's breathing difficulties and chest pain and reduce additional damage through rapid procedures.

Recently, artificial intelligence technology has been applied to the field of medical image reading, increasing the accuracy and speed of image reading. In particular, an artificial intelligence reading system has been developed that marks areas with abnormal findings such as lung nodules through X-ray image analysis and presents the possibility as an index. Therefore, pneumothorax can also be detected to some extent through artificial intelligence image reading.

However, pneumothorax can be found in both lungs of a patient or in only one lung. In particular, it is necessary to predict the presence or absence of a pneumothorax as well as the left and right positions of the pneumothorax due to the nature of urgently proceeding with the tube insertion into the lung where the pneumothorax exists. However, there is a limit to accurately predicting whether the detected pneumothorax is located on the left or right lung side of the patient. Therefore, there is a need for an artificial intelligence image reading technology optimized for pneumothorax.

(Patent Document 1) KR10-1879207 B

(Patent Document 2) US2018-0047158 A

(Patent Document 3) US2019-0156484 A

The task to be concluded is to generate a pneumothorax prediction model that predicts a pneumothorax from an image and determines the area where the pneumothorax is located among the left and right regions of the patient, and provides a method and system for detecting pneumothorax using the learned pneumothorax prediction model.

The task to be solved is to create a pneumothorax prediction model that simultaneously discriminates the presence or absence of pneumothorax in the image, the area where the pneumothorax is located, the presence or absence of the treated tube, and the area where the tube is located, and emergency using the learned pneumothorax prediction model. It is to provide a method and system for detecting pneumothorax.

The task to be solved is to provide a method and system for generating a spine reference line, which is a criterion for discriminating the left and right regions of a patient from an image, and generating an image labeled with the left and right regions as learning data based on the spine reference line. It is to do.

According to an embodiment, a method for detecting a pneumothorax by a computing device operated by at least one processor, comprising: from a learned pneumothorax prediction model, predicted pneumothorax information, predicted tube information, and predicted spine baseline for an input image. Obtaining, in a prediction image displaying the predicted pneumothorax information and the predicted tube information, at least one representative pneumothorax location for the predicted pneumothorax information, and at least one representative tube location for the predicted tube information Determining, dividing the predicted image into a first region and a second region by the predicted spine reference line, and among the first region and the second region, the at least one representative pneumothorax location and the at least one And determining an area in which the representative tube location exists.

The method for detecting pneumothorax may further include classifying the input image as an emergency pneumothorax when the tube representative location does not exist in the first region where the at least one representative pneumothorax location exists.

The pneumothorax detection method may further include outputting information on the first area requiring tube treatment together with an alarm indicating the emergency pneumothorax.

The method for detecting pneumothorax may further include classifying the input image as a general pneumothorax when the tube representative location is present in the first region where the at least one representative pneumothorax location exists.

Each of the representative pneumothorax location and the representative tube location is a location in which the predicted pneumothorax information or the predicted tube information is displayed as a predicted value, the location where the predicted value is the maximum value, the location of the maximum value in the region where the predicted value is greater than or equal to the threshold value, or the predicted value is critical. It may be determined as at least one of the positions of the median value in the region greater than or equal to the value.

The pneumothorax detection method may further include learning the pneumothorax prediction model for at least one task. The at least one task is a task of predicting a pneumothorax in the training images, and predicting a tube in the training images based on training images in which a first area and a second area divided by a spine reference line are tagged with left and right discrimination labels. And a task of predicting the spine baseline from the training images.

According to another embodiment, a method of detecting a pneumothorax by a computing device, from a pneumothorax prediction model learned to predict a pneumothorax using learning images tagged with left and right discrimination labels, and predicted pneumothorax information and predicted spine for an input image. Obtaining a baseline, and determining a pneumothorax presence region among left and right regions of the patient using the predicted pneumothorax information and the predicted spinal baseline. The left and right discrimination labels include area information divided based on a spine reference line of each training image.

The pneumothorax detection method includes obtaining predicted tube information for the input image from the pneumothorax prediction model when the pneumothorax prediction model is trained to further predict a tube from the training images, the predicted tube information and the Determining a tube presence region among left and right regions of the patient using the predicted spine baseline, and the pneumothorax presence region is a first region among the left and right regions of the patient, and the tube presence region is a second region or the tube is present. If the region does not exist, the step of classifying the input image as an emergency pneumothorax may be further included.

According to another embodiment, a method for detecting a pneumothorax by a computing device operated by at least one processor, receiving training images, separating a spine region from each learning image, and a spine reference line representing the spine region Generating, tagging the left and right discrimination labels on the basis of the spine reference line in each of the training images, generating each training image tagged with the left and right discrimination labels as training data, and using the training data And training a pneumothorax prediction model. The training of the pneumothorax prediction model includes the pneumothorax prediction model for each of at least one training image corresponding to the training data, a task for predicting a pneumothorax, a task for predicting a tube, and a task for predicting the spine baseline. It includes the step of learning.

In the generating of the spine reference line, the spine region may be separated from each learning image to which the spine region information is annotated, or the spine region may be separated from each learning image through a machine learning model learned by a spine region detection task.

In the generating of the spine baseline, a baseline may be generated by post-processing the pixels separated into the spine region using a multiple regression algorithm.

The pneumothorax detection method includes inputting a request image into the learned pneumothorax prediction model, obtaining from the pneumothorax prediction model, predicted pneumothorax information, predicted tube information, and predicted spine baseline for the request image, Separating the predicted pneumothorax information and the predicted image on which the predicted tube information is displayed into two regions using the predicted spine baseline, and the predicted pneumothorax information and the predicted tube information exist among the two regions. It may further include the step of determining whether the request image is emergency by comparing the regions.

The step of determining whether the request image is emergency or not is, when the predicted tube information does not exist in the region where the predicted pneumothorax information exists, classifies the request image as an emergency pneumothorax, and the predicted pneumothorax information exists. When the predicted tube information exists in the region, the request image may be classified as a general pneumothorax.

A computing device according to another exemplary embodiment includes a memory and at least one processor that executes instructions of a program loaded in the memory. In the program, inputting a request image as a pneumothorax prediction model, acquiring predicted pneumothorax information, predicted tube information, and predicted spine baseline for the request image from the pneumothorax prediction model, the predicted pneumothorax information And dividing the predicted image on which the predicted tube information is displayed into two regions using the predicted spine baseline, and comparing the predicted pneumothorax information and the region in which the predicted tube information exists among the two regions, And instructions described to execute the step of determining whether the request image is emergency.

The program may include the pneumothorax prediction model learned about a task of predicting a pneumothorax from an input image, a task of predicting a tube from the input image, and a task of predicting the spine baseline from the input image.

Determining whether the request image is emergency, in the prediction image, determining at least one representative pneumothorax location for the predicted pneumothorax information and at least one representative tube location for the predicted tube information, the Determining an area in which the at least one representative pneumothorax location and the at least one tube representative location exist among the two areas of the predicted image separated using the predicted spine baseline, and the at least one representative pneumothorax location is In the case where the representative tube position does not exist in the present region, instructions described to execute the step of classifying the request image as an emergency pneumothorax may be included.

According to the embodiment, not only the pneumothorax is predicted from the image through the pneumothorax prediction model, but also the presence or absence of the treated tube and the baseline of the patient's spine, which is the criterion for discrimination of the left and right regions, can be predicted, through which emergency pneumothorax is required. Can be quickly and accurately judged.

According to the embodiment, since the left and right regions for each patient can be determined without error using the patient's spine baseline predicted from the image, pneumothorax can be predicted without being affected by the patient's photographing posture and body characteristics.

1 is an exemplary diagram of a left and right asymmetric patient image.
2 and 3 are each a structural diagram of a pneumothorax detection system according to an embodiment.
4 is a diagram illustrating a method of generating a learning image tagged with left and right discrimination labels according to an exemplary embodiment.
5 is a diagram exemplarily illustrating an emergency pneumothorax classification method according to an embodiment.
6 is a flowchart of a method of generating training data for a pneumothorax detection model according to an embodiment.
7 is a flowchart of a pneumothorax detection method using a pneumothorax detection model according to an embodiment.
8 is a hardware configuration diagram of a computing device according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

In the specification, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit", "... group", and "module" described in the specification mean a unit that processes at least one function or operation, which can be implemented by hardware or software or a combination of hardware and software. have.

In the specification, a task refers to a task to be solved through machine learning or a task to be performed through machine learning. For example, when recognizing, classifying, and predicting anomaly from a medical image, each of anomaly recognition, anomaly classification, and anomaly prediction may correspond to individual tasks. The pneumothorax detection model of the present invention is a machine learning model for learning at least one task, and may be implemented as software executed on a computing device. A program including instructions described to perform the operation of the present invention may be downloaded through a network or sold as a product.

It is preferable that the left and right regions finally provided to the medical staff according to the present invention are provided as a left region and a right region of the patient divided based on the patient's spine. In this case, the computing device may map the left side of the image to the right side of the patient and map the right side of the image to the left side of the patient, and then determine whether a pneumothorax exists in the left area of the patient reference or the right area. Alternatively, the computing device may be implemented to determine whether a pneumothorax exists in the left or right area of the image reference in the intermediate step, and then change the left and right according to the patient in the output step. Alternatively, the computing device may be implemented to determine whether a pneumothorax exists in the first area or the second area of the image without distinction between the left and right directions in the intermediate stage, and then change the left area and the right area according to the patient in the output stage. have.

1 is an exemplary diagram of a left and right asymmetric image.

Referring to FIG. 1, pneumothorax is a disease in which the lungs cannot be breathed normally when the air is filled in the pleural cavity surrounding the lungs, so the medical staff quickly inserts a tube into the pleural cavity where the pneumothorax is formed to expel air. You have to do the procedure that tells you. Since pneumothorax patients need emergency procedures, damage to emergency patients can be reduced if the image reading system accurately and quickly predicts pneumothorax from images.

However, since most of the image reading systems up to now predict abnormal signs in the entire image, the predicted pneumothorax location is the patient's left area (left lung location), even though it is possible to predict the pneumothorax and display the predicted pneumothorax location on the image. It is difficult to determine whether it is located in the right area (the area where the right lung is located). If an image is always captured in a reference posture required for photographing, the image reading system can be designed to determine the region where the pneumothorax is located by dividing the left and right regions based on the center of the image. However, such a simple segmentation method has a problem in that an error occurs in the case of a patient who is difficult to photograph in a reference posture due to physical characteristics or a special situation in which the patient cannot properly control the posture due to shortness of breath and chest pain. For example, as shown in FIG. 1, when an image in which the patient's posture is not photographed symmetrically is input, there is a big problem that the left and right regions in which the pneumothorax is located may be erroneously identified.

On the other hand, a medical staff can detect a pneumothorax on an X-ray image and at the same time determine whether the pneumothorax is located in the patient's left lung or the right lung. Therefore, the medical staff can immediately know where the tube is to be inserted even when viewing an image in which the patient's posture is not symmetrically photographed as shown in FIG. 1.

In the following, similar to the pneumothorax determination mechanism of a medical staff, a pneumothorax detection method of predicting a pneumothorax from an image by considering a patient's photographic posture and physical characteristics, and determining an area where the pneumothorax is located among the left and right regions of the patient will be described. Furthermore, a method of detecting emergency pneumothorax by simultaneously predicting the treated tube will be described.

Each of FIGS. 2 and 3 is a structural diagram of a pneumothorax detection system according to an embodiment, and FIG. 4 is a diagram illustrating a method of generating a learning image tagged with left and right discrimination labels according to an embodiment, and FIG. 5 is an embodiment. It is a diagram exemplarily illustrating an emergency pneumothorax classification method according to an example.

Referring to FIG. 2, the pneumothorax detection system 1 includes a training data generator 100 for generating training data from an image 10, a learner 200 for training a pneumothorax prediction model 20, and a learned pneumothorax prediction model ( It includes an emergency classifier 300 for classifying emergency patients with pneumothorax using 20). For the sake of explanation, the training data generator 100, the learner 200, and the emergency classifier 300 are referred to as being called, but these are computing devices operated by at least one processor. Here, the training data generator 100, the learner 200, and the emergency classifier 300 may be implemented in one computing device or distributedly implemented in a separate computing device. When distributed in separate computing devices, the training data generator 100, the learner 200, and the emergency classifier 300 may communicate with each other through a communication interface. The computing device may be a device capable of executing a software program written to perform the present invention, and may be, for example, a server, a laptop computer, or the like.

Each of the training data generator 100, the learner 200, and the emergency classifier 300 may be an artificial intelligence model, or may be implemented as a plurality of artificial intelligence models. In addition, the pneumothorax prediction model 20 may also be a single artificial intelligence model, or may be implemented as a plurality of artificial intelligence models. The pneumothorax detection system 1 may be a single artificial intelligence model, or may be implemented as a plurality of artificial intelligence models. Accordingly, one or a plurality of artificial intelligence models corresponding to the above-described configurations may be implemented by one or a plurality of computing devices.

Referring to FIG. 3, a pneumothorax prediction model that has been trained in the learner 200 and software implemented in the emergency classifier 300 may operate on one computing device 400. The computing device 400 may be plural, and may operate individually at various sites (eg, hospitals, different places within one or more hospitals). Alternatively, the pneumothorax prediction model that has been trained in the learner 200 is located on the server side, and the computing devices 400 equipped with the emergency classifier 300 may access the server through a network.

In addition, functions described in the present invention may be variously separated or integrated according to service types. For example, client devices of various sites (eg, hospitals, different places in one or a plurality of hospitals) may request an image analysis to a server through a network. An image captured by the imaging device may be requested to a server through a network. Then, the server may transmit a report on the pneumothorax prediction result for the requested image, whether it is an emergency, and an emergency treatment area (eg, the left area of the patient) to the client device.

Referring back to FIG. 2, the training data generator 100 receives images 10 used for training a pneumothorax prediction model. The training data generator 100 predicts a spine region from each image and generates a spine reference line that is a criterion for determining the left and right regions of the patient. The training data generator 100 generates the left and right discrimination labels generated based on the spine baseline as training data of the pneumothorax prediction model. The image may be a chest X-ray image, but may be an image captured by a different type of device according to medical technology. Training data may be generated according to the type and structure of the pneumothorax prediction model, and a learning method. For example, at least some of the images may be data tagged with pneumothorax information and/or tube information with a label, or may be data without a label.

Since there are not many images annotated with information on the spine region, it is difficult for the pneumothorax prediction model to learn the task of determining left and right from the images. Accordingly, the learning data generator 100 may learn a task of predicting a spine region of an image in which the spine region is not annotated by using the learning images in which the spine region is annotated. The learned training data generator 100 predicts the spine region from the images 10 used for the training of the pneumothorax prediction model, and generates a spine baseline, which is a criterion for discriminating the left and right regions of the patient from each image. I can.

On the other hand, if information on the spine region is annotated in some of the images 10 used for training of the pneumothorax prediction model, the training data generator 100 separates the annotated spine region from the image and Can be created.

Referring to FIG. 4, the training data generator 100 receives an image 10. When the spine region is not annotated in the image 10, the training data generator 100 trained to predict the spine region predicts the spine region from the image 10 and separates the predicted spine region 12. )do. The training data generator 100 post-processes the pixels separated into the spine region 12 to generate a spine reference line 14 representing the spine region. The training data generator 100 may generate the spine reference line 14 by post-processing the pixels separated into the spine region 12 using an algorithm such as polynomial regression.

The training data generator 100 may generate left and right discrimination labels based on the spine reference line 14. The training data generator 100 may divide left and right regions based on the spine reference line 14 and generate labels for the divided left and right regions. Since the left and right discrimination labels do not need to be accurate, a pseudo label can be tagged (annotated) on the left and right areas. The left and right regions can be divided based on, for example, the position of the heart. That is, of the two regions divided by the spine reference line 14, a label indicating the left region of the patient may be tagged on the region where the heart exists, and the label indicating the region where the heart does not exist may be tagged as the right region of the patient. Alternatively, the left and right discrimination label may tag a label of a first area and a label of a second area divided by a spine reference line without dividing the left and right.

In this way, the left and right discrimination label, which is an image label obtained by dividing the left area and the right area based on the spine reference line 14, is input together with the image 10 as a pneumothorax prediction model. The left and right discrimination labels are used as data to train the pneumothorax prediction model to predict the left and right regions simultaneously with the prediction of the pneumothorax.

The training data generator 100 generates an image tagged with left and right discrimination labels as training data. The training data is stored in the designated storage. The above-described left and right discrimination labels may or may not coincide with the actual left and right positions of the patient who captured the image. As described later, in order to determine whether a pneumothorax is in an emergency, the present disclosure is a configuration for determining whether the area where the pneumothorax has occurred and the emergency measure for the pneumothorax, for example, the area where the tube is inserted, is the same area. The left and right and left and right discrimination labels tagged to determine whether there is an emergency pneumothorax may not be the same.

A method of separating the spine region 12 from the image 10 may be various.

According to an embodiment, when information about the spine region is annotated in the image 10, the training data generator 100 may separate the annotated spine region 12 from the image 10.

According to another embodiment, the training data generator 100 may predict and separate the spine region 12 from the image 10 through machine learning-based segmentation.

The training data generator 100 may train a machine learning model for a task of detecting a spine region in an image. The machine learning model may be, for example, a convolutional neural network model. The learning data generator 100 trains a machine learning model using data annotated with a medical image and information about a spine region included in the medical image, and the spine region 12 of the input image 10 through the learned machine learning model. ) Can be predicted and separated. Further, the training data generator 100 may find a center line representing the spine region from pixels of the spine region using a multiple regression algorithm, and determine the center line as the spine reference line 14.

According to another embodiment, the training data generator 100 may train a machine learning model for a task of finding a center line that best follows an annotated spine region. The training data generator 100 may find a center line that best follows the spine region 12 of the input image 10 through the learned machine learning model and determine the center line as the spine reference line 14.

The spine reference line 14 may be generated by applying a multiple regression algorithm to pixels of the separated spine region. In this case, the training data generator 100 may divide the spine region into a plurality of regions, and then generate a spine reference line by connecting the center line extracted for each region. Alternatively, the training data generator 100 may generate a spine reference line by dividing the spine region into a plurality of regions, searching for a center point of the region for each region, and connecting the center points for each region.

Referring back to FIG. 2, the learner 200 receives images tagged (annotated) with left and right discrimination labels as training data, and trains the pneumothorax prediction model 20 using the training data. The pneumothorax prediction model 20 learns a task of predicting a pneumothorax from an input image and a task of predicting a spine baseline for discriminating left and right regions of a patient from the input image. In this case, the pneumothorax prediction model 20 may simultaneously learn a task of predicting a tube from an input image. The task of predicting a pneumothorax in an input image may be a task of outputting a value predicted that pixels of the image are pneumothorax. The task of predicting a tube from the input image may be a task of outputting a value predicted by which pixels of the image are predicted as tubes. The task of determining the left and right regions of the patient in the input image is the task of outputting a value predicted by the pixels of the image as a spine baseline, or predicting the left and right regions (or the first region and the second region) based on the predicted spine baseline. It can be a task. In the description, it is described as predicting the spine baseline.

The pneumothorax prediction model may be trained based on images annotated for each of the pneumothorax, spine, and tube. In this case, without the above-described configuration of the learner 200, a pneumothorax prediction model can be trained, and tasks for each of the pneumothorax, the spine, and the tube can be learned at once.

The pneumothorax prediction model 20 may output pneumothorax information, tube information, and a predicted spine reference line predicted from an input image. The pneumothorax information and tube information may include a predicted value calculated for each pixel. The predicted pneumothorax value and the predicted tube value may be visually displayed on the input image. For example, a heat map according to the predicted value may be displayed on the predicted image in which the predicted value is displayed. The emergency classifier 300 classifies a pneumothorax patient based on the pneumothorax information, tube information, and spine baseline output from the learned pneumothorax prediction model 20. The emergency classifier 300 may be classified into, for example, an emergency pneumothorax requiring an emergency procedure and a general pneumothorax that does not require an emergency procedure. Emergency procedures for emergency pneumothorax may include tube procedures. The emergency classifier 300 may classify cases that are not predicted as pneumothorax. For example, the emergency classifier 300 may classify normal or other lesions without pneumothorax.

When classifying a pneumothorax patient, a pneumothorax patient after a tube procedure is not classified as an emergency, so the emergency classifier 300 does not classify it as an emergency if it is predicted that pneumothorax and a tube exist in the same area among the areas separated based on the spinal baseline. If there is no tube in the area predicted to have a pneumothorax, it can be classified as an emergency. The emergency classifier 300 may first find a region predicted that a pneumothorax exists, and classify the input image as an emergency if there is no tube in the pneumothorax prediction region. The emergency classifier 300 classifies the input image as emergency if there is no tube in the area predicted that a pneumothorax exists, but uses the patient area information (for example, the patient's left lung area) requiring tube treatment to be information on the emergency pneumothorax. Can be printed with

Referring to FIG. 5, the emergency classifier 300 may determine left and right regions based on a spine baseline predicted value output from the pneumothorax prediction model 20 for the input image 40. The predicted value for the spine baseline may mean a predicted value for the above-described spine baseline. In addition, the emergency classifier 300 may determine a region including the pneumothorax prediction heat map from the pneumothorax prediction image 42. For example, the emergency classifier 300 may determine a region including the predicted heat map among left and right regions (or the first region and the second region) of the pneumothorax prediction image 42, and the pneumothorax is detected based on the predicted pneumothorax value. The existing area can be determined. In more detail, the emergency classifier 300 may determine a region in which a pneumothorax exists in the left region and the right region divided based on the spine baseline, based on the predicted pneumothorax value. Likewise, the emergency classifier 300 may determine an area in which the tube prediction heat map is located based on the spine baseline prediction value. For example, the emergency classifier 300 may determine which of the left and right regions (or the first region and the second region) the region where the tube prediction heat map is located based on the spine reference line, and calculate the tube prediction value. It is possible to determine the area in which the tube exists on the basis. The emergency classifier 300 may classify a pneumothorax as a general pneumothorax if a pneumothorax and a tube exist in the same region among regions separated from the left and right by the spine reference line.

For example, if the emergency classifier 300 predicts a pneumothorax in the right area on a patient basis (R_ptx=1, L_ptx=0), and predicts a tube in the right area on a patient basis (R_tube=1, L_ptx=0), Both the right area and the left area based on the patient can be classified as general, not emergency (R_emergency=0, L_emergency=0).

If the emergency classifier 300 predicts a pneumothorax in the right area on a patient basis (R_ptx=1, L_ptx=0), and predicts a tube in the left area on a patient basis (R_tube=0, L_ptx=1) or a tube is predicted If not (R_tube=0, L_ptx=0), the right area based on the patient may be classified as an emergency (R_emergency=1, L_emergency=0).

The emergency classifier 300 may determine whether a pneumothorax and a tube exist in the same region based on the predicted pneumothorax value, the predicted tube value, and the predicted spine baseline.

According to an embodiment, the emergency classifier 300 extracts a maximum value of a predicted pneumothorax value and a maximum value of a predicted tube value from each of the predicted heat maps in which the predicted pneumothorax value and the predicted tube value for the input image are displayed. The emergency classifier 300 checks the area including the location of the maximum value of the predicted pneumothorax value and the maximum value of the tube predicted value among the two areas (the first area and the second area) divided based on the predicted spine baseline, and based on the location of the maximum value. It can be determined whether pneumothorax and tube are present in the same area. If the maximum value of the pneumothorax predicted value and the maximum value of the tube predicted value exceed the POSITIVE determination threshold, it can be determined as POSITIVE.

According to another embodiment, the emergency classifier 300 includes a contour connecting points having a pneumothorax predicted value equal to or greater than a threshold value and a point in which the predicted tube value is greater than or equal to a threshold value in a predicted image including a predicted heat map for each of the pneumothorax and the tube. After drawing an area connecting them, an area including the location of the maximum value of each area can be checked, and based on the location of the maximum value, it is possible to determine whether a pneumothorax and a tube exist in the same area. If the maximum value of each area exceeds the POSITIVE determination threshold, it can be determined as POSITIVE.

According to another embodiment, the emergency classifier 300 connects the areas where the pneumothorax predicted value is greater than or equal to the threshold value and the area where the tube predicted value is greater than the threshold value in the predicted image including the predicted heat map for each of the pneumothorax and the tube. After drawing the area, the area including the location of the median value of each area can be checked, and based on the location of the median value, it is possible to determine whether a pneumothorax and a tube exist in the same area. If the maximum value of each area exceeds the POSITIVE determination threshold, it can be determined as POSITIVE.

According to another embodiment, the emergency classifier 300 connects the areas where the pneumothorax predicted value is greater than or equal to the threshold value and the area where the tube predicted value is greater than the threshold value in the predicted image including the predicted heat map for each of the pneumothorax and the tube. After drawing the desired area, an area in which the number of pixels included in each area is distributed is checked, and when the maximum value of the area in which the number of pixels is distributed exceeds the POSITIVE determination threshold, it can be determined as POSITIVE.

6 is a flowchart of a method of generating training data for a pneumothorax detection model according to an embodiment.

Referring to FIG. 6, the training data generator 100 receives images used for training a pneumothorax prediction model (S110).

The training data generator 100 separates the spine region from each image to which information about the spine region is annotated, or separates the spine region from each image through a machine learning model learned to detect the spine region (S120).

The training data generator 100 generates a spine reference line, which is a criterion for determining the left and right regions of the patient, from the spine regions of each image (S130). A baseline may be generated by post-processing the pixels separated into the spine region with a multiple regression algorithm.

The training data generator 100 tags the left and right discrimination labels generated based on the spine reference line on the corresponding image (S140). The left and right discrimination label is a label that separates the left area (or the first area) and the right area (the second area), and may be a pseudo label.

The training data generator 100 stores images tagged with left and right discrimination labels as training data (S150).

Images tagged with left and right discrimination labels are used for learning of the pneumothorax prediction model 20. The learner 200 receives images tagged with left and right discrimination labels, predicts a pneumothorax from the input image, predicts a tube from the input image, and predicts a spine baseline that determines the left and right regions of the patient from the input image. The pneumothorax prediction model 20 is trained as a task. The types, structures, and learning methods of the pneumothorax prediction model 20 may be designed in various ways.

7 is a flowchart of a pneumothorax detection method using a pneumothorax detection model according to an embodiment.

Referring to FIG. 7, the emergency classifier 300 inputs a request image to the learned pneumothorax prediction model 20 (S210).

The emergency classifier 300 acquires pneumothorax information (pneumothorax predicted value), tube information (tube predicted value), and predicted spine baseline from the pneumothorax prediction model 20 for the input image (S220). Each of the pneumothorax information and the tube information may be output as a predicted heat map in which the predicted pneumothorax value and the predicted tube value are displayed in the input image. An image including the predicted heat map may be referred to as a predicted image.

The emergency classifier 300 determines whether the representative position of the pneumothorax information (pneumothorax predicted value) and the representative position of the tube information (tube predicted value) exist in the same region in the image in which the region is divided based on the spine reference line (S230). The emergency classifier 300 may determine a pixel location in which the predicted pneumothorax value and the predicted tube value are the maximum values as the representative location. The emergency classifier 300 may determine regions based on the predicted values of the pneumothorax and the tube, and determine a maximum or median pixel position within the region as a representative position of the predicted pneumothorax value or the predicted tube value.

When the representative position of the predicted pneumothorax value and the representative position of the predicted tube value exist in the same area, the emergency classifier 300 classifies it as a general pneumothorax (S240). That is, the emergency classifier 300 classifies the treated tube as a general pneumothorax because the treated tube exists in the area where the pneumothorax is found.

The emergency classifier 300 classifies as an emergency pneumothorax when the representative location of the tube prediction value does not exist in the region where the representative location of the predicted pneumothorax value exists (S250). That is, since the emergency classifier 300 does not find a tube in the area where the pneumothorax is found, it classifies the area (eg, the left lung area of the patient) as an emergency pneumothorax requiring an emergency procedure, for example, a tube procedure. .

The emergency classifier 300 outputs a result including whether the input image has an emergency pneumothorax (S260). In the case of an emergency pneumothorax, the emergency classifier 300 outputs an alarm indicating an emergency pneumothorax. In this case, the emergency classifier 300 may output information related to pneumothorax as well as whether the input image has an emergency pneumothorax. For example, the emergency classifier 300 is information related to emergency pneumothorax included in the input image, and provides various results such as a pneumothorax prediction result, a tube prediction result, and an emergency tube treatment site (for example, a patient's left lung area). Can be printed.

In this way, the pneumothorax detection system 1 not only predicts the pneumothorax from the image through the pneumothorax prediction model, but also predicts the presence or absence of the treated tube, and the baseline of the patient's spine, which is the criterion for discriminating the left and right regions, through which emergency surgery is performed. It can quickly and accurately determine if it is a required emergency pneumothorax. In particular, since the pneumothorax detection system 1 can determine the left and right regions for each patient without error using the patient's spine baseline predicted from the image, the pneumothorax can be predicted without being affected by the patient's photographic posture and physical characteristics. .

8 is a hardware configuration diagram of a computing device according to an embodiment.

Referring to FIG. 8, the training data generator 100, the learner 200, and the emergency classifier 300 are instructions described to execute the operation of the present invention in the computing device 500 operated by at least one processor. Run the program containing (instructions).

The hardware of the computing device 500 may include at least one processor 510, a memory 530, a storage 550, and a communication interface 570, and may be connected through a bus. In addition, hardware such as an input device and an output device may be included. The computing device 500 may be equipped with various software including an operating system capable of driving a program.

The processor 510 is a device that controls the operation of the computing device 500 and may be various types of processors that process instructions included in a program. For example, a CPU (Central Processing Unit) or a MPU (Micro Processor Unit) may be used. ), microcontroller unit (MCU), graphic processing unit (GPU), and the like. The memory 530 loads a corresponding program so that instructions described to perform the operation of the present invention are processed by the processor 510. The memory 530 may be, for example, read only memory (ROM), random access memory (RAM), or the like. The storage 550 stores various types of data, programs, etc. required to perform the operation of the present invention. The communication interface 570 may be a wired/wireless communication module.

The embodiments of the present invention described above are not implemented only through an apparatus and a method, but may be implemented through a program that realizes a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (16)

A method for detecting a pneumothorax by a computing device operated by at least one processor,
From the learned pneumothorax prediction model, obtaining predicted pneumothorax information, predicted tube information, and predicted spine baseline for the input image,
In the prediction image displaying the predicted pneumothorax information and the predicted tube information, determining at least one representative pneumothorax location for the predicted pneumothorax information and at least one representative tube location for the predicted tube information,
Dividing the predicted image into a first region and a second region by the predicted spine reference line, and
Determining an area in which the at least one representative pneumothorax location and the at least one tube representative location exist among the first area and the second area
Pneumothorax detection method comprising a. In claim 1,
Classifying the input image as an emergency pneumothorax when the tube representative location does not exist in the first region where the at least one representative pneumothorax location exists.
Further comprising a pneumothorax detection method. In paragraph 2,
Outputting information on the first area requiring tube treatment with an alarm indicating the emergency pneumothorax
Further comprising a pneumothorax detection method. In claim 1,
Classifying the input image as a general pneumothorax when the tube representative location is present in the first region where the at least one representative pneumothorax location exists
Further comprising a pneumothorax detection method. In claim 1,
Each of the representative pneumothorax positions and the representative tube positions
In a prediction image in which the predicted pneumothorax information or the predicted tube information is displayed as a predicted value, at least one of a position where the predicted value is the maximum value, the position of the maximum value in the region where the predicted value is greater than or equal to the threshold value, or the position of the median value in the region where the predicted value is greater than or equal to the threshold value. Determined by one, pneumothorax detection method. In claim 1,
Further comprising the step of learning the pneumothorax prediction model for at least one task,
The at least one task is
A task of predicting a pneumothorax from the training images, a task of predicting a tube from the training images, and the learning based on the learning images tagged with the left and right discrimination labels in the first region and the second region divided by the spine baseline. Comprising the task of predicting the spine baseline from images, pneumothorax detection method. A method for a computing device to detect a pneumothorax, comprising:
Obtaining the predicted pneumothorax information and the predicted spine baseline for the input image from the pneumothorax prediction model learned to predict the pneumothorax using the learning images tagged with the left and right discrimination labels, and
Determining a pneumothorax presence region among left and right regions of the patient using the predicted pneumothorax information and the predicted spine baseline
Including,
The left and right discrimination labels include area information divided based on a spine reference line of each training image. In clause 7,
When the pneumothorax prediction model is trained to further predict a tube from the training images, obtaining predicted tube information for the input image from the pneumothorax prediction model,
Determining a tube presence area among left and right areas of the patient using the predicted tube information and the predicted spine baseline, and
Classifying the input image as an emergency pneumothorax when the area where the pneumothorax exists is the first area, the area where the tube exists is the second area, or the area where the tube does not exist among the left and right areas of the patient
Further comprising a pneumothorax detection method. A method for detecting a pneumothorax by a computing device operated by at least one processor,
Receiving the learning images,
Separating a spine region from each learning image and generating a spine reference line representing the spine region,
Tagging left and right discrimination labels based on the spine reference line in each of the training images,
Generating the training images tagged with the left and right discrimination labels as training data, and
Including the step of training a pneumothorax prediction model using the training data,
The step of training the pneumothorax prediction model
For at least one training image corresponding to the training data, comprising the step of learning the pneumothorax prediction model for each of a task for predicting a pneumothorax, a task for predicting a tube, and a task for predicting the spine baseline Detection method. In claim 9,
Generating the spine baseline comprises:
A pneumothorax detection method in which the spine region is separated from each learning image to which the spine region information is annotated, or the spine region is separated from each learning image through a machine learning model learned with a spine region detection task. In claim 9,
Generating the spine baseline comprises:
A method of detecting a pneumothorax by post-processing the pixels separated into the spine region with a multiple regression algorithm to generate a baseline. In claim 9,
Inputting the request image into the learned pneumothorax prediction model,
Obtaining, from the pneumothorax prediction model, predicted pneumothorax information, predicted tube information, and predicted spine baseline for the request image,
Dividing the predicted pneumothorax information and the predicted image on which the predicted tube information is displayed into two regions using the predicted spine baseline, and
Comparing an area in which the predicted pneumothorax information and the predicted tube information exist among the two areas, and determining whether the request image is emergency
Further comprising a pneumothorax detection method. In claim 12,
The step of determining whether the request image is emergency
If the predicted tube information does not exist in the region where the predicted pneumothorax information exists, the request image is classified as an emergency pneumothorax,
When the predicted tube information is present in the region where the predicted pneumothorax information exists, the request image is classified as a general pneumothorax. As a computing device,
Memory, and
Including at least one processor to execute instructions (instructions) of the program loaded in the memory,
The above program is
Inputting the request image as a pneumothorax prediction model,
Obtaining, from the pneumothorax prediction model, predicted pneumothorax information, predicted tube information, and predicted spine baseline for the request image,
Dividing the predicted pneumothorax information and the predicted image on which the predicted tube information is displayed into two regions using the predicted spine baseline, and
Computing device comprising instructions to execute the step of determining whether or not the request image is emergency by comparing an area in which the predicted pneumothorax information and the predicted tube information exist among the two areas. In clause 14,
The above program is
The computing device comprising the pneumothorax prediction model learned about a task of predicting a pneumothorax from an input image, a task of predicting a tube from the input image, and a task of predicting the spine baseline from the input image. In clause 14,
The step of determining whether the request image is emergency
In the predicted image, determining at least one representative pneumothorax location for the predicted pneumothorax information and at least one representative tube location for the predicted tube information,
Determining an area in which the at least one representative pneumothorax location and the at least one tube representative location exist among two areas of the prediction image separated by using the predicted spine baseline, and
Classifying the request image as an emergency pneumothorax if the tube representative location does not exist in the region where the at least one representative pneumothorax location exists.
A computing device comprising instructions described to execute a.

Images