KR102275760B1 - Method for predicting cancer suspicious area and device for predicting cancer suspicious area using the same - Google Patents

Abstract

The present invention predicts whether a subject suspected of gastric cancer will develop early gastric cancer by using an early gastric cancer prediction model configured to predict whether or not onset of early gastric cancer will occur based on the steps of receiving an endoscopy image for a subject suspected of gastric cancer, and based on the endoscopy image To provide a method for predicting early gastric cancer, and a device using the same, comprising the steps of: and providing a prediction result.

Description

Prediction method for early gastric cancer and device using the same {METHOD FOR PREDICTING CANCER SUSPICIOUS AREA AND DEVICE FOR PREDICTING CANCER SUSPICIOUS AREA USING THE SAME}

The present invention relates to a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same, and more specifically, to predict the onset of early gastric cancer based on a gastroscopic image of a suspected gastric cancer individual, and predict the degree of tumor invasion. It relates to a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same.

There are several types of malignant tumors in the stomach, such as gastric adenocarcinoma, malignant lymphoma, and gastrointestinal stromal tumor, but of these, gastric adenocarcinoma is the most common.

On the other hand, gastric cancer can be classified into early gastric cancer (EGC) and advanced gastric cancer (AGC) according to the degree of tumor progression. More specifically, early gastric cancer may refer to a condition in which tumor cells have invaded the mucosal layer or the submucosal layer of the stomach, regardless of whether or not metastasis to the lymph nodes is present. Advanced gastric cancer may refer to a condition in which tumor cells can invade surrounding organs through the submucosal layer and penetrate beyond the muscle layer. In this case, treatment applied for treatment of early gastric cancer and advanced gastric cancer may be different. For example, in the case of early gastric cancer, relatively simple procedures such as endoscopic mucosal resection (EMR), endoscopic submucosal dissection (ESD), and local curative surgery can be applied, and advanced gastric cancer In contrast to topical treatment for early gastric cancer, chemotherapy can be applied along with radical surgery such as lymph node dissection.

In this case, early gastric cancer and advanced gastric cancer may be different not only in the treatment method performed, but also in the treatment prognosis and further the recurrence rate. For example, when treatment is performed at an early stage of gastric cancer, the treatment prognosis may be relatively good and the recurrence rate may also be low. Accordingly, the importance of diagnosing gastric cancer at an early stage is continuously increasing.

Gastric endoscopy is the most frequently used method for diagnosing early gastric cancer. However, in the case of early gastric cancer, the lesion site may be very small or it may be difficult to distinguish it from normal tissue, so the accuracy may be lowered.

As the accuracy of diagnosis is further required for the prevention of medical accidents and improvement of medical services, the development of a new diagnostic method for early gastric cancer diagnosis, particularly for diagnosis using an endoscope, is continuously required.

The description underlying the invention has been prepared to facilitate understanding of the invention. It should not be construed as an admission that the matters described in the background technology of the invention exist as prior art.

Meanwhile, as a new diagnostic method for gastric cancer, a molecular biological diagnostic method based on the level of a biomarker associated with gastric cancer has been proposed. However, the molecular biological diagnosis method can diagnose whether or not gastric cancer occurs in an individual suspected of gastric cancer, but has a limitation in accurately classifying and diagnosing gastric cancer having different biological characteristics.

The inventors of the present invention have found that, in relation to the diagnosis of gastric cancer, especially in the high-accuracy diagnosis of early gastric cancer, even in early gastric cancer, the treatment method is different depending on the depth of invasion, and the recurrence pattern and prognosis are different for each patient even if they are at the same tumor stage. Note that each may be different. More specifically, the inventors of the present invention have found that accurately predicting the depth of tumor invasion for early gastric cancer of mucosal or submucosal invasion is an accurate and effective treatment tailored to the individual's condition. It was recognized that it can lead to a good prognosis. Even in the same early gastric cancer, in the case of the mucosal layer, it is often possible to cure only endoscopic treatment rather than surgical treatment such as gastrectomy. Therefore, accurately predicting the depth of invasion of tumor cells before resection is a very important issue in ensuring that no patients who do not need gastrectomy undergo gastrectomy or patients who undergo additional surgery due to incomplete treatment after endoscopic resection. .

Furthermore, the inventors of the present invention can recognize that a predictive model learned from data of an endoscopy image can be used as a method for solving the problems of the conventional diagnostic system using an endoscope and increasing the accuracy of early gastric cancer diagnosis. there was.

As a result, the inventors of the present invention probabilistically predict early gastric cancer and normal mucosal changes of mucosal invasion or submucosal invasion with respect to the endoscopic image of an individual suspected of gastric cancer using the prediction model, so the onset of early gastric cancer It has led to the development of a new early gastric cancer prediction system that can provide diagnostic information on the depth of tumor invasion along with the presence of cancer.

Furthermore, the inventor of the present invention trained a predictive model to predict a lesion suspicious region for early gastric cancer in a gastroendoscopic image and determine a location for a lesion suspicious region based on a predetermined landmark. As a result, it was possible to develop an early gastric cancer prediction system configured to predict and provide the presence of normal or early gastric cancer for the lesion site found in the gastroscopic image, and to provide location information in case of early gastric cancer.

At this time, the inventors of the present invention pay attention to the high probability that the lesion area is predicted as a background when, in the prediction using the predictive model, only the suspected lesion area occupies a very small area in the entire gastroscopic image is predicted. did.

Accordingly, the inventors of the present invention additionally used an upper endoscopic image for learning cropped to include only the lesion region in the upper endoscopic image for learning the predictive model, and as a result, problems caused by the above data imbalance were solved. .

Accordingly, the problem to be solved by the present invention is to predict the onset of early gastric cancer and predict the result of early gastric cancer by using an early gastric cancer prediction model configured to predict the onset of early gastric cancer and further the depth of invasion of early gastric cancer based on the received gastroscopic image. To provide a method for predicting early gastric cancer configured to provide

Another problem to be solved by the present invention is to predict the lesion suspected region of early gastric cancer in the upper endoscopy image using a lesion suspicious region prediction model configured to predict the lesion suspicious region of early gastric cancer in the upper endoscopy image, and predict the results of the prediction. It is to provide a method for predicting early gastric cancer, configured to provide.

Another problem to be solved by the present invention is to predict the location of a suspected lesion region and provide it by using a lesion suspicious region prediction model configured to predict a predetermined landmark within a gastroendoscopic image, It provides a predictive method.

Another problem to be solved by the present invention is to receive a plurality of endoscopic images and predict whether or not early gastric cancer will develop for each of the plurality of endoscopic images using an early gastric cancer prediction model. An object of the present invention is to provide a diagnostic method for early gastric cancer prediction.

Another problem to be solved by the present invention includes an endoscopic imaging unit configured to acquire an endoscopic image of a suspected gastric cancer object, and a processor operatively connected to the endoscopic imaging unit and configured to use various predictive models, It is to provide a device for early gastric cancer prediction.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above problems, a method for predicting early gastric cancer according to an embodiment of the present invention is provided. The method includes the steps of receiving an endoscopy image for a suspected gastric cancer individual, and predicting whether an early gastric cancer onset for a suspected gastric cancer individual using an early gastric cancer prediction model configured to predict the early onset of gastric cancer based on the gastric endoscopy image and providing a prediction result.

According to a feature of the present invention, the early gastric cancer prediction model may be further configured to predict mucosal invasion early gastric cancer or submucosal invasion early gastric cancer based on an endoscopy image. Furthermore, the step of predicting the occurrence of early gastric cancer includes predicting gastric cancer differently from normal for suspected gastric cancer individuals using an early gastric cancer prediction model, and predicting whether the predicted gastric cancer is early gastric cancer with invasion into the mucosal layer or early gastric cancer with invasion into the submucosal layer. includes steps.

According to another feature of the present invention, the step of predicting whether or not early gastric cancer will develop is predicted using an early gastric cancer prediction model to predict gastric cancer differently from normal for an individual suspected of gastric cancer, and then the predicted gastric cancer probability of mucosal invasion early gastric cancer; The method may include calculating each probability of submucosal invasion early gastric cancer.

According to another feature of the present invention, the predicting of the occurrence of gastric cancer includes using a lesion suspicious area prediction model configured to predict an early gastric cancer lesion suspected area in the gastroscopic image, and the early gastric cancer lesion in the gastroscopic image. The method may further include predicting a suspicious region. Furthermore, the providing may further include providing the predicted lesion suspicious region when the lesion suspected region of early gastric cancer is predicted in the gastroendoscopic image by the lesion suspicious region prediction model.

According to another feature of the present invention, the step of predicting whether the disease is onset includes calculating the probability of a lesion area or a normal probability with respect to a specific area in the upper endoscopy image using a lesion suspicious area prediction model, The method may include determining a specific region as a lesion suspicious region when the probability of the lesion region is greater than or equal to a predetermined level.

According to another feature of the present invention, the lesion suspicious region prediction model may be further configured to predict the lesion region of the mucosal invasion early gastric cancer or the lesion region of the submucosal invasion early gastric cancer within the gastroscopic image. In addition, the predicting of the onset includes: when the suspected lesion region is determined by the lesion-suspicious region prediction model, the probability of mucosal invasion early gastric cancer lesion region and the submucosal invasion of the suspected lesion region using the lesion-suspicious region prediction model The method may further include calculating the probability of a lesion region of early gastric cancer, respectively, and determining the condition of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer with respect to the lesion suspect region based on the calculated probability of the lesion suspicious region. can Furthermore, the providing may further include providing a condition for the determined suspected lesion region.

According to another feature of the present invention, the lesion suspicious region prediction model may be further configured to predict a predetermined landmark in the gastroscopic image. In addition, the predicting of the occurrence of gastric cancer further includes predicting the landmark in the gastroendoscopic image using a lesion suspicious region prediction model, and predicting the location of the lesion suspicious region based on the landmark. can do. Furthermore, the providing may further include providing the predicted location of the suspected lesion region.

According to another feature of the present invention, the step of receiving the gastroscopic image may include receiving a plurality of gastroscopic images of the suspected gastric cancer object, and the predicting of early gastric cancer onset includes: a region suspected of a lesion. The method may further include predicting a landmark for each of the plurality of gastroendoscopic images by using the prediction model, and predicting the location of the suspected lesion region for each of the plurality of gastroscopic images based on the landmark. . Furthermore, the providing may further include providing the predicted location of the suspected lesion region for each of the plurality of gastroendoscopic images.

According to another feature of the present invention, an endoscopic image of mucosal invasion early gastric cancer including the lesion region of mucosal invasion early gastric cancer, a submucosal invasion endoscopic image including the lesion region of submucosal invasion early gastric cancer, and a normal gastroscope for a normal subject The method may further include learning an early gastric cancer prediction model to predict whether or not early gastric cancer will occur with respect to the learning gastroscopic image based on at least one learning gastroscopic image selected from the group consisting of images.

According to another feature of the present invention, the learning step includes normalizing the pixels of the gastroscopic image for learning to a predetermined level, and among the learning gastroscopic images, the mucosal invasion early gastric cancer endoscopy image and the submucosal invasion early gastric cancer endoscope. Masking the lesion area of mucosal invasion early gastric cancer and the lesion area of submucosal invasion early gastric cancer for each image. masking, and training the early gastric cancer prediction model to predict at least one condition from the group consisting of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, and normal based on the masked lesion area and background area. can

According to another feature of the present invention, the step of standardizing includes only the lesion region of the mucosal invasion early gastric cancer and the lesion region of the submucosal invasion early gastric cancer for each of the mucosal invasion early gastric cancer endoscopic image and the submucosal invasion early gastric cancer endoscopic image Normalizing pixels of the cropped mucosal invasion early gastric cancer endoscopic image, the cropped submucosal invasion early gastric cancer endoscopic image, and the normal gastroscopic image to a predetermined level. have.

According to another feature of the present invention, the step of cropping may include cropping while maintaining a ratio of the lesion area of the mucosal invasion early gastric cancer and the lesion area of the submucosal invasion early gastric cancer.

According to another feature of the present invention, the receiving of the gastroscopic image may include receiving a plurality of gastroscopic images of a suspected cancer object, and the predicting of whether or not gastric cancer will develop may include early gastric cancer prediction. The method may include predicting whether or not early gastric cancer will develop with respect to each of the plurality of gastroendoscopic images by using the model. Furthermore, the providing may include providing prediction results for each of the plurality of gastroscopic images in real time.

In order to solve the above problems, a device for predicting early gastric cancer according to another embodiment of the present invention is provided. The device includes an endoscopic imaging unit configured to acquire an endoscopic image of a suspected gastric cancer object, and a processor operatively connected to the endoscopic imaging unit. In this case, the processor is configured to predict the early onset of gastric cancer for a suspected gastric cancer individual by using an early gastric cancer prediction model configured to predict whether or not early gastric cancer will develop based on the gastroendoscopic image.

According to a feature of the present invention, the early gastric cancer prediction model may be further configured to predict mucosal invasion early gastric cancer or submucosal invasion early gastric cancer based on the gastroscopic image, and the processor uses the early gastric cancer prediction model to suspect gastric cancer and predicting the condition of at least one of the group consisting of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, and normal for the subject.

According to another feature of the present invention, the processor is further configured to predict a lesion suspicious region of early gastric cancer in the gastroendoscopic image, using a lesion suspicious region prediction model configured to predict a lesion suspicious region of early gastric cancer in the gastroscopic image can be

According to another feature of the present invention, the lesion suspicious region prediction model may be further configured to predict the lesion region of the mucosal invasion early gastric cancer or the lesion region of the submucosal invasion early gastric cancer within the gastroscopic image. Furthermore, when the suspected lesion region is determined by the lesion suspicious region prediction model, the processor uses the lesion suspicious region prediction model to determine the probability of the lesion region of the mucosal invasion early gastric cancer and the lesion region of the submucosal invasion early gastric cancer with respect to the lesion suspect region. It may be further configured to calculate the probabilities, respectively, and to determine the condition of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer with respect to the suspected lesion region based on the calculated probability for the lesion suspicious region.

According to another feature of the present invention, the lesion suspicious region prediction model may be further configured to predict a predetermined landmark in the gastroscopic image. In addition, the processor may be further configured to predict a landmark in the gastroscopic image using the lesion suspicious region prediction model, and predict the location of the lesion suspicious region based on the landmark.

According to another feature of the present invention, the device for early gastric cancer prediction may further include a laser irradiator configured to irradiate a therapeutic laser to the suspected lesion region.

The present invention provides a method for predicting early gastric cancer using an early gastric cancer prediction model configured to predict the onset of early gastric cancer and the condition of the gastric cancer with respect to an endoscopy image, and a device using the same, thereby providing accurate diagnostic information to an individual suspected of gastric cancer. can have an effect.

More specifically, the present invention can provide information predicting the depth of invasion for an early tumor of mucosal invasion or submucosal invasion, and can provide accurate and effective treatment tailored to an individual's condition according to the prediction result. Accordingly, the present invention has an effect that can contribute to providing a good prognosis for early gastric cancer treatment.

In addition, the present invention provides a high-accuracy diagnosis for early gastric cancer by processing the upper endoscopy image for learning to reduce the error of predicting a hole in the upper endoscopy image as a lesion region and using it to learn a predictive model. It has the ability to provide information.

Furthermore, the present invention has an effect of reducing the possibility that a suspected lesion region is predicted as a background, which is generated when only a lesion region occupying a very small region in the entire gastroscopic image is predicted.

In addition, the present invention can provide location information on the lesion by using a prediction model configured to determine a location for a predicted lesion region based on a predetermined landmark. Accordingly, the medical personnel can accurately identify the lesion site based on the location information, and thus can more easily perform early gastric cancer treatment.

Furthermore, according to the use of a predictive model configured to predict the location of a lesion based on a landmark, the present invention has the effect of providing a diagnostic result in which the gastroscopic apparatus, residual food, and bleeding site appearing in the gastroscopic image are distinguished. have.

The present invention receives a plurality of gastroscopy images, and predicts the onset of early gastric cancer and, furthermore, the depth of tumor invasion for each gastroscopic image, a diagnostic system capable of monitoring the stomach in real time and predicting early gastric cancer at the same time has the effect of providing

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.

1 illustrates the configuration of a device for predicting early gastric cancer according to an embodiment of the present invention.
2A illustrates a procedure of a method for predicting early gastric cancer according to an embodiment of the present invention.
2B exemplarily illustrates a procedure for predicting the condition of early gastric cancer with respect to a gastroendoscopic image by the method for predicting early gastric cancer according to an embodiment of the present invention.
3A illustrates a procedure of a method for predicting early gastric cancer according to another embodiment of the present invention.
3B exemplarily illustrates a procedure of predicting a lesion suspicious region and predicting a condition of an early gastric cancer with respect to a gastroendoscopic image by the method for predicting early gastric cancer according to an embodiment of the present invention.
4A to 4C are gastroendoscopic image data for learning about an early gastric cancer prediction model and a lesion suspicious region prediction model, which are used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention; will be.
4D exemplarily illustrates an early gastric cancer prediction model and a lesion suspicious region prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.
5A to 5I illustrate evaluation results according to a learning method and output value setting for an early gastric cancer prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.
6 is a diagram illustrating an evaluation result of a lesion suspicious region prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.

Advantages of the invention, and methods of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in a variety of different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the technical field to which the present invention belongs It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims.

The shapes, sizes, proportions, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are exemplary, and thus the present invention is not limited to the illustrated matters. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When 'including', 'having', 'consisting', etc. mentioned in this specification are used, other parts may be added unless 'only' is used. When a component is expressed in a singular, the case in which the plural is included is included unless otherwise explicitly stated.

In interpreting the components, it is interpreted as including an error range even if there is no separate explicit description.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and technically various interlocking and driving are possible, as will be fully understood by those skilled in the art, and each embodiment may be independently implemented with respect to each other, It may be possible to implement together in a related relationship.

For clarity of interpretation of the present specification, terms used herein will be defined below.

As used herein, the term “early gastric cancer” may refer to a condition in which tumor cells have invaded the mucosal layer or submucosal layer of the stomach. Meanwhile, even with the same early gastric cancer tumor stage, the prognosis or recurrence pattern may be different depending on the depth of invasion of the mucosal or submucosal invasion to the early tumor. Accordingly, predicting the depth of an early tumor may provide an accurate and effective treatment tailored to an individual's condition.

As used herein, the term “gastric endoscopy image” may refer to any image including the gastric mucosa received from an endoscopic device such as a gastroendoscopic diagnosis apparatus. Preferably, the gastroscopic image may be an RGB color image, but is not limited thereto. Meanwhile, the upper endoscopy image may be a two-dimensional image, a three-dimensional image, a still image of one cut, or a moving image composed of a plurality of cuts. For example, when the gastroscopy image is a moving picture composed of a plurality of cuts, according to the method for predicting early gastric cancer according to an embodiment of the present invention, the occurrence of early gastric cancer may be predicted for each of the plurality of gastroscopic images. . As a result, according to the present invention, the diagnosis information on the predicted lesion site can be provided in real time by performing the prediction of early gastric cancer simultaneously with the reception of the gastroscopic image from the endoscope apparatus.

On the other hand, in the early gastric cancer stage, the lesion area including cyst, inflammation, or mass tissue may exist in a very small area compared to normal tissue in the upper endoscopy image, so the diagnosis rate for the lesion site according to the user's skill level This may vary.

In order to solve this problem, a predictive model trained to predict whether or not early gastric cancer occurs, furthermore, a region suspected of lesion and a condition in the gastroscopic image may be used.

As used herein, the term “early gastric cancer prediction model” may be a model configured to predict early gastric cancer or normal with respect to an endoscopy image. More specifically, the early gastric cancer prediction model is a predictive model trained to predict the depth of tumor invasion for early gastric cancer of mucosal invasion or submucosal invasion based on the RGB color of the received gastroscopic image, or to classify as normal. can be For example, an early gastric cancer prediction model uses a data set of a learning gastroscopy image including an area confirmed as early gastric cancer by a gastroscopy or biopsy, and the mucosal invasion early gastric cancer or submucosal layer for the gastroscopic image. It may be a model trained to probabilistically predict and classify invasive early gastric cancer or a normal condition. Furthermore, the early gastric cancer prediction model may be a model trained to predict the onset of gastric cancer based on the learning gastroscopic image cropped to include only the lesion region in the gastroscopic image. By such a learning method, the early gastric cancer prediction model can reduce the error of predicting the lesion region as a background, which occurs when the early gastric cancer prediction model is configured to predict only the suspicious lesion region occupying a very small portion of the entire gastroscopic image.

Meanwhile, the early gastric cancer prediction model may be configured to classify two classes of early gastric cancer (early gastric cancer with mucous membrane invasion and early gastric cancer with submucosal invasion) or normal with respect to the input gastroendoscopic image. Furthermore, it may be configured to classify three classes of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, or normal. The early gastric cancer prediction model may be further configured to classify two classes of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer.

Meanwhile, in the early gastric cancer prediction method according to various embodiments of the present disclosure, a lesion suspicious region prediction model configured to predict a lesion suspicious region suspected of early gastric cancer in the received gastroscopic image may be further used.

As used herein, the term "suspicious lesion area prediction model" predicts the presence of a suspected lesion area within a gastroscopic image of a suspected gastric cancer individual, and a tumor of mucosal invasion or submucosal invasion of the predicted suspected lesion area. It can be a model trained to predict the depth of invasion. More specifically, the lesion suspicious area prediction model calculates the probability of a lesion area or a normal probability for a specific area in the upper endoscopy image, and selects the specific area when the probability of the lesion area for the specific area is greater than or equal to a predetermined level. It can be determined as a suspected lesion area. For example, when the probability that the specific area is the lesion suspicious area is 60% or more (normal probability is 40%), the specific area may be determined as the lesion suspicious area. Furthermore, the suspected lesion region prediction model may be configured to predict the depth of tumor invasion of mucosal invasion or submucosal invasion based on the RGB color for the determined suspicious lesion region. Furthermore, the suspected lesion region prediction model may be a model configured to predict a predetermined landmark in the gastroscopic image and further predict the location of the suspicious lesion region based thereon.

Meanwhile, the suspected lesion region prediction model may be a single model capable of simultaneously performing prediction of the suspected lesion region and classification of the condition for the predicted lesion suspicious region, and respectively predicting and classifying the lesion suspicious region. There may be two predictive models that can do this.

As used herein, the term “suspicious lesion region” may refer to a region suspected of being a tumor in the upper endoscopy image. For example, the region suspected of a lesion may be a region for a mucosal invasion tumor tissue or a submucosal layer invasion tumor tissue, or a region for a normal tissue having a shape similar to a lesion. Meanwhile, the suspicious lesion region may have different pixel values and textures with respect to other regions. Accordingly, the suspected lesion region prediction model may predict the suspicious lesion region based on a pixel value or texture.

As used herein, the term "predetermined landmark" may refer to a specific region or region that can be a reference in identifying the location of the suspected lesion region. The landmark predetermined in the present specification may be a 'hole' that appears in the gastroscopic image as the endoscopic device moves through the inner mucosa of the stomach. However, the landmark is not limited thereto.

As described above, the prediction models used in various embodiments of the present invention may be used singly or in combination in the prediction of early gastric cancer.

Hereinafter, a device for predicting early gastric cancer according to an embodiment of the present invention will be described with reference to FIG. 1 .

First, a device for predicting early gastric cancer according to an embodiment of the present invention will be described in detail with reference to FIG. 1 . 1 illustrates the configuration of a device for predicting early gastric cancer according to an embodiment of the present invention.

Referring to FIG. 1 , the device 100 for predicting early gastric cancer includes an endoscopic image capturing unit 110 , an input unit 120 , an output unit 130 , a processor 140 , and a storage unit 150 .

Specifically, the endoscopic imaging unit 110 may be configured to take an endoscopic image of the gastric mucosa of an object suspected of gastric cancer. In this case, the upper endoscopy image acquired by the endoscopic image capturing unit 110 may be an RGB color image, but is not limited thereto. Meanwhile, the endoscopic image capturing unit 110 may be further configured to transmit the captured gastroscopic image to the processor 140 to be described later. Furthermore, the endoscopic image capturing unit 110 may perform a function of a receiving unit capable of receiving an upper endoscopy image from the outside. On the other hand, the upper endoscopy image acquired through the endoscopic image capturing unit 110 may include a 'suspicious lesion region'.

The input unit 120 may set the early gastric cancer prediction device 100 and instruct the operation of the above-described endoscopic image capturing unit 110 . For example, the user may instruct the operation of the endoscopic image capturing unit 110 to obtain a desired still cut through the input unit 120 . On the other hand, the input unit 120 may have a stick shape provided with a lever to be directly connected to the endoscopic image capturing unit 110 to operate, but is not limited thereto, and may be a keyboard, a mouse, or a touch screen panel.

Meanwhile, the output unit 130 may visually display the upper endoscopy image obtained from the endoscope image capturing unit 110 . Furthermore, the output unit 130 may be configured to display diagnostic information including a condition determined by the processor 140 in the gastroscopic image, or a probability predicted for a suspected lesion region.

The processor 140 may be a component for providing an accurate prediction result to the device 110 for predicting early gastric cancer. In this case, for the prediction of early gastric cancer, the processor 140 may be configured to classify the condition as mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, or normal with respect to the gastroendoscopic image, and predict a suspected lesion region. For example, the processor 140 classifies an early gastric cancer or normal condition with respect to a gastroscopic image of an object suspected of gastric cancer, and further predicts a lesion area existing in the image and a prediction model configured to classify the condition for the lesion area. is available. Accordingly, the processor 140 may provide early gastric cancer diagnosis information for the suspected gastric cancer object based on the RGB color of the gastroscopic image received from the endoscopic image capturing unit 110 , for example, the gastroscopic image. In this case, the prediction models configured to predict the onset of early gastric cancer may be based on various learning models learned based on the image. For example, prediction models used in various embodiments of the present invention are a Deep Neural Network (DNN), a Convolutional Neural Network (CNN), a Deep Convolution Neural Network (DCNN), a Recurrent Neural Network (RNN), and a Restricted Boltzmann Machine (RBM). ), DBN (Deep Belief Network), SSD (Single Shot Detector) model, or a prediction model based on U-net, but is not limited thereto.

The storage unit 150 stores the gastroscopic image of the suspected gastric cancer object obtained through the endoscopic image capturing unit 110 , and stores the instruction of the early gastric cancer prediction device 100 set through the input unit 120 . can be configured. Furthermore, the storage 150 is configured to store the results classified or predicted by the processor 140 . However, it is not limited to the above, and the storage unit 150 may store various pieces of information determined by the processor 140 for the prediction of early gastric cancer.

Hereinafter, a method for predicting early gastric cancer according to an embodiment of the present invention will be described in detail with reference to FIGS. 2A to 2B . 2A illustrates a procedure of a method for predicting early gastric cancer according to an embodiment of the present invention. 2B exemplarily illustrates a procedure for predicting the condition of early gastric cancer with respect to a gastroendoscopic image by the method for predicting early gastric cancer according to an embodiment of the present invention.

Referring to FIG. 2A , a procedure for predicting early gastric cancer according to an embodiment of the present invention is as follows. First, a gastroscopic image of a suspected gastric cancer object is received (S200). Next, using an early gastric cancer prediction model configured to predict the early onset of gastric cancer with respect to the gastroendoscopic image, it is predicted whether or not the early gastric cancer will develop in a suspected gastric cancer individual ( S210 ). Finally, a result predicted by the early gastric cancer prediction model is provided (S220).

For example, referring to FIG. 2B , in the step of receiving the gastroscopic image ( S200 ), the gastroscopic image 202 of the object suspected of gastric cancer may be received. In this case, the upper endoscopy image 202 may be an RGB color image.

According to an embodiment of the present invention, in the step of receiving the gastroscopic image (S200), the gastroscopic image 202 that has been pre-processed so as to have a certain pixel so that the gastroscopic image 202 can be quickly analyzed is further received. can do. Otherwise, after the step of receiving the gastroscopic image (S200), the size is adjusted to have a certain pixel unit with respect to the received gastroscopic image 202, or the contrast, resolution, contrast, or left-right symmetry is adjusted. A pre-processing step for the endoscopic image 202 may be further performed. As a result of the pre-processing step, the gastroscopic image 202 has a resolution or size required for a prediction model to be described later, and the resolution or size may be smaller than that of the original gastroscopic image, so that the processing speed in the prediction model will be improved. can

According to another embodiment of the present invention, in the step of receiving the upper endoscope image ( S200 ), a plurality of upper endoscopy images may be received. For example, as a gastroscopic image captured in real time may be acquired according to the movement of the endoscopic image capturing unit, a plurality of gastroscopic images may be acquired in the step of receiving the gastroscopic image ( S200 ).

Next, referring to FIG. 2B , in the step of predicting whether or not early gastric cancer occurs ( S210 ), the gastroscopic image 202 obtained in the step ( S200 ) of receiving the gastroscopic image to the early gastric cancer prediction model 212 . The RGB color 204 for ) is input. In this case, the early gastric cancer prediction model 212 can predict the mucosal invasion early gastric cancer, the submucosal invasion early gastric cancer, or a normal condition based on the RGB color 204 for the gastroscopic image 202, but is limited thereto. no. More specifically, in the step of predicting the occurrence of early gastric cancer (S210), the early gastric cancer prediction model 212 is based on the input RGB color 204 to the mucosal layer invasion early gastric cancer and submucosal layer for the gastroendoscopic image 202. A probability of invasive early gastric cancer or normal may be calculated, and a condition of the gastroscopic image 202 may be classified based on the calculated probability value.

Accordingly, again referring to FIG. 2B , in the step of providing the prediction result ( S220 ), the occurrence of early gastric cancer is predicted by the early gastric cancer prediction model 212 in the step ( S210 ) of predicting whether or not the occurrence of early gastric cancer will occur. An endoscopic image 222 may be provided. More specifically, in the step of providing the prediction result ( S220 ), the probability of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, or normal for the gastroscopic image 202 calculated by the early gastric cancer prediction model 212 is calculated. Including, early gastric cancer onset prediction information 224 may be provided together.

According to the above-described method for predicting early gastric cancer according to an embodiment of the present invention, an accurate prediction of early gastric cancer may be possible regardless of a user's skill level according to the classification of a condition on an endoscopy image obtained from a suspected gastric cancer individual. Accordingly, the present invention can be applied to a diagnostic system based on an endoscopy image.

Hereinafter, a method for predicting early gastric cancer according to another embodiment of the present invention will be described in detail with reference to FIGS. 3A to 3B . 3A illustrates a procedure of a method for predicting early gastric cancer according to another embodiment of the present invention. 3B exemplarily illustrates a procedure of predicting a lesion suspicious region and predicting a condition of an early gastric cancer with respect to a gastroendoscopic image by the method for predicting early gastric cancer according to an embodiment of the present invention.

Referring to FIG. 3A , a procedure for predicting early gastric cancer according to an embodiment of the present invention is as follows. First, a gastroscopic image of a suspected gastric cancer object is received (S300). Next, the suspicious lesion region is predicted in the gastroscopic image by using the lesion suspicious region prediction model configured to predict the lesion suspicious region with respect to the upper endoscopic image (S310). Finally, the suspected lesion region predicted by the lesion suspicious region prediction model is provided (S320).

For example, referring to FIG. 3B , in the step of receiving an endoscopy image ( S300 ), a gastroscopic image 302 of a suspected gastric cancer object may be received. In this case, the upper endoscopy image 302 may be an RGB color image.

Referring to FIG. 3B , in the step of predicting the suspicious lesion area ( S310 ), RGB for the gastroscopic image 302 obtained in the step ( S300 ) of receiving the gastroscopic image to the lesion suspicious area prediction model 312 . A color 304 is input. In this case, the suspected lesion region prediction model 312 may predict the suspected lesion region based on the RGB color 304 of the upper endoscopy image 302 . However, it is not limited thereto.

According to another embodiment of the present invention, in the step of predicting the suspected lesion region (S310), the lesion suspicious region prediction model 312 is based on at least one of a pixel value of the region, a texture, and a pixel difference degree from the surrounding region, The area suspected of lesion can be predicted.

According to another embodiment of the present invention, when the suspicious lesion area 324 is predicted by the lesion suspicious area prediction model 312 in the step of predicting the suspicious lesion area (S310), the The RGB color 326 may additionally be input to the lesion suspicious region prediction model 312 . As a result, the condition for the predicted lesion suspicious region 324 can be determined. More specifically, in the step of predicting the suspected lesion region ( S310 ), the lesion suspicious region prediction model 312 is based on the input RGB color 326 , the mucosal invasion early gastric cancer and submucosal layer for the lesion suspicious region 324 . A probability of invasive early gastric cancer or normal may be calculated, and a condition of the suspicious lesion region 324 in the gastroendoscopic image 302 may be classified based on the calculated probability value.

Meanwhile, according to another embodiment of the present invention, in the step of predicting the suspicious lesion area ( S310 ), a predetermined landmark is predicted in the gastroendoscopic image 302 using the lesion suspicious area prediction model 312 . and predicting the position of the suspicious lesion region 324 based on the landmark may be further performed. In this case, the landmark may be a 'hole' that appears in the gastroscopic image as the endoscope device moves through the internal mucosa of the stomach, but is not limited thereto. According to the prediction of the position of the suspicious lesion region 324 based on the landmark, the suspicious lesion region 324 and the gastroendoscopic apparatus, the remaining food, and the bleeding site appearing in the gastroscopic image 302 may be distinguished.

Next, referring to FIG. 3B , in the step of providing the predicted suspicious lesion area ( S320 ), in the step of predicting the suspected lesion area ( S310 ), the lesion suspicious area prediction model 312 uses the lesion suspicious area prediction model 312 , A gastroscopic image 332 of which a condition is predicted may be provided.

According to another embodiment of the present invention, in the step of providing the predicted suspected lesion region (S320), the suspicious lesion region 324 in the gastroendoscopic image 302 calculated by the lesion suspicious region prediction model 312 is Early gastric cancer onset prediction information 334 including the probability of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, or normal may be provided together.

According to another embodiment of the present invention, in the step of providing the predicted suspected lesion region ( S320 ), information on the position of the suspected lesion region 324 predicted by the lesion suspicious region prediction model 312 may be provided. have.

By the method for predicting early gastric cancer according to another embodiment of the present invention, the suspected lesion region and its location are predicted with respect to the gastroscopic image obtained from the suspected gastric cancer individual, and the condition of the predicted lesion suspicious region is classified and provided. Accordingly, accurate prediction of early gastric cancer may be possible regardless of the user's skill level. Accordingly, the present invention can be applied to a diagnostic system based on an endoscopy image.

Hereinafter, a learning method of the early gastric cancer prediction model and the lesion suspicious region prediction model used in various embodiments of the present invention will be described with reference to FIGS. 4A to 4C . 4A and 4B are gastroendoscopic image data for learning about an early gastric cancer prediction model and a lesion suspicious region prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention; will be. 4C schematically illustrates an early gastric cancer prediction model and a lesion suspicious region prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.

At this time, the image used for learning is a gastroendoscopic image including 200 mucosal invasion tumors, 200 gastroendoscopic images including submucosal invasion tumors, and 200 endoscopic gastroscopic images of normal individuals. Each of 150 gastroendoscopic images except for 50 images to be used for evaluation data is used.

Referring to (a) to (c) of Figure 4a, in the case of the early gastric cancer prediction model and the lesion suspicious area prediction model, for learning, a gastroendoscopic image including a mucosal invasion tumor (Fig. 4a (a)), A gastroendoscopic image (Fig. 4a (b)) including a submucosal invasion tumor and a gastroscopic image of a normal subject (Fig. 4a (c)) are used. At this time, with respect to each of the upper endoscopy images for learning of early gastric cancer, a region for a mucosal invasion tumor and a region for a submucosal invasion tumor are masked as correct answers, respectively. On the other hand, in the upper endoscopic image for learning of a normal object, in order to prevent a hole existing in the image from being predicted as the correct answer, the background area except for the hole is masked as the correct answer.

Based on the learning gastroscopic image, the early gastric cancer prediction model may be trained to classify the condition for the training gastroscopic image, and the lesion suspicious area prediction model predicts the lesion suspicious area within the training gastroscopic image and predicts the predicted area. can be learned to classify conditions against

On the other hand, for learning the predictive model, the upper endoscopic image for training may be standardized to have predetermined pixels. For example, the upper endoscopic image for learning may be standardized to have 101 x 101 pixels, but is not limited thereto. Furthermore, the gastroscopic image for learning includes a gastroscopic full image including a lesion having a predetermined pixel unit, a gastroscopic image obtained by enlarging and cropping the lesion to have a predetermined pixel unit, or a lesion having a predetermined pixel unit. It may be a gastroendoscopic image cropped in consideration of the ratio.

Referring to (a) to (c) of Figure 4b, the upper endoscopic image for learning may be inverted, rotated, or transformed and processed to amplify the learning data. However, the upper endoscopic image for learning used for learning the predictive model is not limited to the above, and may be processed in more various ways.

On the other hand, the early gastric cancer prediction model and the lesion suspicious area prediction model used in various embodiments of the present invention include a Deep Convolution Neural Network (DCNN) suitable for image recognition or a meaningful part in a gastroscopic image in a pixel unit. A predictable U-net may be used. However, the predictive models of the present invention are not limited thereto and may be based on more various algorithms.

Referring to FIG. 4C , a predictive model based on DCNN is exemplarily disclosed. More specifically, more specifically, a predictive model based on DCNN consists of a convolutional layer, a max pooling layer that extracts local maxima and uses them as features, and a fully connected layer. can be When the RGB color of the gastroscopic image is input to the prediction model, it can be configured to classify two classes of early gastric cancer (early gastric cancer with mucosal invasion and early gastric cancer with submucosal invasion) or normal by a multi-layered neural network structure. Furthermore, gastroscopic imaging can be classified into 3 classes of early gastric cancer with mucosal invasion, early gastric cancer with submucosal invasion, or normal, and the predictive model is configured to classify 2 classes of mucosal invasion early gastric cancer, or submucosal invasion early gastric cancer. can

The early gastric cancer prediction model and the lesion suspicious region prediction model used in the early gastric cancer prediction method and the early gastric cancer prediction device using the same according to an embodiment of the present invention are not limited to the above, and may be learned in more various ways. .

Example 1: Evaluation of an early gastric cancer prediction model

In Example 1 below, an evaluation method and results of an early gastric cancer prediction model used in various embodiments of the present invention will be described with reference to FIGS. 5A to 5I . 5A to 5I illustrate evaluation results according to a learning method and output value setting for an early gastric cancer prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.

More specifically, referring to FIG. 5A , three early gastric cancer prediction models are a model configured to classify three classes of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, or normal with respect to the input gastroendoscopic image (A. 3 class classification), early gastric cancer (mucous layer invasion early gastric cancer and submucosal invasion early gastric cancer), or, a model configured to classify 2 classes of normal (B. Class 2 classification), and mucosal invasion early gastric cancer, or submucosal invasion early gastric cancer It is a model configured to classify two classes (C. Class two classes).

Furthermore, the learning method applied to the above predictive model learning is a learning method based on the entire image of the gastroscopy including the lesion having a predetermined pixel unit (1), a gastroscope in which the lesion is enlarged and cropped to have a predetermined pixel unit A learning method (2) based on an image, and a learning method (3) based on an upper endoscopy image cropped in consideration of a ratio to a lesion to have a predetermined pixel unit.

Referring again to FIG. 5A , a model configured to classify 3 classes of mucosal invasion early gastric cancer, or submucosal invasion early gastric cancer, or normal with respect to the gastroscopic image (A. 3 class classification) has a predetermined pixel unit When the learning method (2) based on gastroendoscopic images obtained by magnifying and cropping the lesion is applied, the average detection rate is as high as 0.77.

Early gastric cancer (early gastric cancer with mucosal invasion and early gastric cancer with submucosal invasion) or a model configured to classify 2 classes of normal (B. 2 class classification) is a gastroendoscopic image that enlarges and crops the lesion to have a predetermined pixel unit When the learning method based on (2) is applied, the average detection rate is 0.98, which is higher than when other learning methods are applied. That is, when the early gastric cancer prediction model of the present invention is applied, the early gastric cancer or normal prediction result may have a high confidence level.

Finally, a model configured to classify 2 classes of mucosal invasion early gastric cancer, or submucosal invasion early gastric cancer (C. 2 class classification), is a learning based on gastroendoscopic images in which the lesion is enlarged and cropped to have a predetermined pixel unit. When method (2) is applied, the average detection rate is 0.73, which is higher than when other learning methods are applied. That is, the early gastric cancer prediction model of the present invention can predict the invasion depth of mucosal invasion or submucosal invasion with higher reliability with respect to the gastroendoscopic image including the lesion of early gastric cancer.

According to the above results, based on the learning method (2) based on the gastroscopy image in which the lesion is enlarged and cropped to have a predetermined pixel unit, a predictive model configured to classify the tumor invasion depth for the input gastroscope according to the present invention It may be used in various embodiments of, but is not limited thereto.

Referring to FIG. 5B , mucosal invasion early gastric cancer, or submucosal invasion early gastric cancer, or 3 classes of normal, learned by the learning method (1) based on the full image of the gastroscopy including the lesion having a predetermined pixel unit The evaluation results for the model configured to classify (A. 3 class classification) are shown.

Referring to FIG. 5B (a), for each of the three classes of mucosal invasion (mucosal), submucosal invasion (submucosal) gastric cancer, and normal (normal), a predetermined correct answer (true label) and predicted result (predicted label) The average detection rates (recall) of the early gastric cancer prediction model calculated based on , respectively, are 0.62, 0.72, and 0.70. Referring to (b) of FIG. 5B , for the three classes of gastric cancer with mucosal invasion, submucosal invasion, and normal, the prediction accuracy of the early gastric cancer prediction model calculated based on the predetermined correct answer and the prediction result is 0.633, respectively. , 0.590 and 0.875.

Referring to FIG. 5C , early gastric cancer (early gastric cancer with mucosal invasion and early gastric cancer with submucosal invasion), or normal, learned by the learning method (1) based on the full image of the gastroscopy including the lesion having a predetermined pixel unit The evaluation results for the model configured to classify two classes of ? (B. two-class classification) are shown.

Referring to (a) of FIG. 5C , for each of the two classes of early gastric cancer (invasion of the mucous membrane and invasion of the submucosal layer) and the normal, the average detection rate of the early gastric cancer prediction model calculated based on the predetermined correct answer and prediction result is 0.94, respectively. and very high levels of 0.92. Referring to (b) of FIG. 5C , for two classes of early gastric cancer and normal, the prediction accuracy of the early gastric cancer prediction model calculated based on the predetermined correct answer and the prediction result is 0.959 and 0.885, respectively, at very high levels.

Referring to FIG. 5D , it is configured to classify two classes of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer, learned by the learning method (1) based on the gastroscopic full image including the lesion with a predetermined pixel unit. The evaluation results for the model (C. 2 class classification) are shown.

Referring to (a) of FIG. 5D , for each of the two classes of invasion of the mucosal layer and invasion of the submucosal layer, the average detection rates of the early gastric cancer prediction model calculated based on predetermined correct answers and prediction results are 0.60 and 0.74, respectively. Referring to (b) of FIG. 5D , for two classes of invasion of the mucosal layer and invasion of the submucosal layer, the prediction accuracy of the early gastric cancer prediction model calculated based on the predetermined correct answer and the prediction result is 0.698 and 0.649, respectively.

5E to 5G show the image classification results for the three-class prediction model learned by the learning method (1) based on the entire upper endoscope image including the lesion having a predetermined pixel unit.

More specifically, referring to FIG. 5E , the probability of mucosal invasion early gastric cancer (red, M) calculated for each gastroscopic image including 50 intestinal mucosal invasion tumors by the early gastric cancer prediction model (red, M), the date of early gastric cancer invasion into the submucosal layer Probabilities (blue, S) and normality (green, N) are shown. In the early gastric cancer prediction model, when the probability of mucosal invasion early gastric cancer is the highest among the calculated probabilities, the input gastroscopic image is classified as an image of 'mucous layer invasion early gastric cancer' (red).

According to the results, it appears that the early gastric cancer prediction model accurately predicted 31 images of the gastroendoscopic images including 50 mucosal invasion tumors. On the other hand, it is extremely rare that the endoscopic image of the mucosal invasion tumor is predicted to be normal by the early gastric cancer prediction model.

Referring to FIG. 5F , the probability of early gastric cancer with mucosal invasion (red, M), the probability of submucosal invasion early gastric cancer calculated for each gastroscopic image including 50 submucosal invasion tumors by the early gastric cancer prediction model ( Blue, S) and probability of being normal (green, N) are shown. In the early gastric cancer prediction model, when the probability of submucosal invasion early gastric cancer is the highest among the calculated probabilities, the input gastroscopic image is classified as an image (blue) of 'submucosal invasion early gastric cancer'.

According to the results, it appears that the early gastric cancer prediction model accurately predicted 36 images of gastroendoscopic images including 50 submucosal invasion tumors. On the other hand, only two cases of gastric endoscopic images of submucosal invasion tumors were predicted to be normal by the early gastric cancer prediction model, which is extremely rare.

Referring to FIG. 5G , the probability of mucosal invasion early gastric cancer (red, M), the probability of submucosal invasion early gastric cancer (blue, S) calculated for each gastroscopic image of 50 normal individuals by the early gastric cancer prediction model ) and the probability of being normal (green, N) are shown. The early gastric cancer prediction model classifies the input endoscopic image as a 'normal' image (green) when the highest probability of being normal among the calculated probabilities.

According to the results, it appears that the early gastric cancer prediction model accurately predicted 35 images out of 50 normal gastroscopic images.

5H and 5I show the image classification results for the two-class prediction model of early gastric cancer learned by the learning method (1) based on the entire image of the gastroscopy including the lesion having a predetermined pixel unit.

Referring to FIG. 5H , the probability of mucosal invasion early gastric cancer (red, M) and submucosal invasion early gastric cancer probability (blue, S) calculated for each of the 50 mucosal invasion gastric endoscopy images by the early gastric cancer prediction model are is shown In the early gastric cancer prediction model, when the probability of mucosal invasion gastric cancer is the highest among the calculated probabilities, the input gastroscopic image is classified as an image (red) of 'mucous layer invasion early gastric cancer'.

According to the results, the early gastric cancer prediction model accurately predicted 30 images out of 50 mucosal invasion gastric endoscopy images.

Referring to FIG. 5I , the probability of early gastric cancer with mucosal invasion (red, M) and the probability of submucosal invasion early gastric cancer (blue, S) calculated for each of the 50 submucosal invasion gastric endoscopic images by the early gastric cancer prediction model (blue, S) This is shown. The early gastric cancer prediction model classifies the input gastroscopic image as an image (blue) of 'submucosal invasion early gastric cancer' when the highest probability of submucosal invasion is among the calculated probabilities.

According to the results, the early gastric cancer prediction model accurately predicted 37 images out of 50 mucosal invasion gastric endoscopy images.

As a result of Example 1 above, the method for predicting early gastric cancer and the device using the same according to various embodiments of the present invention based on the early gastric cancer prediction model may provide accurate diagnostic information to an individual suspected of gastric cancer. More specifically, the present invention can provide a result by predicting the depth of invasion for an early tumor of mucosal invasion or submucosal invasion with respect to the upper endoscopic image, and according to the prediction result, accurate and effective treatment tailored to the individual's condition can provide Accordingly, the present invention can contribute to providing a good prognosis for early gastric cancer treatment.

Example 2: Evaluation of the lesion suspicious area prediction model

In Example 2 below, an evaluation method and results of a lesion suspicious region prediction model used in various embodiments of the present invention will be described with reference to FIG. 6 . 6 is a diagram illustrating an evaluation result of a lesion suspicious region prediction model used in a method for predicting early gastric cancer and a device for predicting early gastric cancer using the same according to an embodiment of the present invention.

More specifically, referring to FIGS. 6 (a) to 6 (c), an early gastric cancer gastroendoscopic image including a mucosal invasion early gastric cancer lesion region and a submucosal invasion early gastric cancer lesion region masked with the correct answer in advance and The prediction result of the lesion suspicious area prediction model for the gastroscopic image of a normal subject is shown. At this time, correct answer masking was not performed on the normal gastroscopic image.

Referring to (a) of FIG. 6 , it appears that the lesion area for the mucosal invasion tumor is accurately predicted in the upper endoscopy image by the lesion suspicious area prediction model.

Referring to FIG. 6B , it appears that the lesion area for the submucosal invasion tumor is accurately predicted in the gastroendoscopic image by the lesion suspicious area prediction model.

Referring to (c) of FIG. 6 , it appears that no region is predicted with respect to a normal gastroscopic image by the lesion suspicious region prediction model. This may mean that the lesion suspicious area prediction model accurately predicted the normal gastroscopic image.

As a result of Example 2 above, the early gastric cancer prediction method and the device using the same according to various embodiments of the present invention based on the lesion suspicious area prediction model have the effect of predicting the suspicious lesion area within the gastroendoscopic image with high accuracy. have.

On the other hand, the effect of the present invention is not limited thereto. For example, the present invention may provide location information on a lesion by using a suspected lesion area prediction model configured to determine a location for a predicted lesion area based on a predetermined landmark. Accordingly, the medical personnel can accurately identify the lesion site based on the location information, and thus can more easily perform early gastric cancer treatment.

Furthermore, the present invention has the effect of providing a diagnosis result in which the gastroscopic apparatus, the residual food, and the bleeding site appear in the gastroscopic image are distinguished by predicting the location of the lesion based on the landmark.

In addition, the present invention receives a plurality of gastroscopic images, and predicts whether or not early gastric cancer occurs and, further, the depth of tumor invasion for each gastroscopic image, so that it is possible to monitor the stomach in real time and predict early gastric cancer at the same time. A diagnostic system may be provided.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: device for predicting early gastric cancer
110: endoscope recording unit
120: input unit
130: output unit
140: processor
150: storage
202, 302: gastroscopy image
204, 304, 326: RGB color
212: Early gastric cancer prediction model
222: Gastric endoscopy image predicted to develop early gastric cancer
224, 334: Early gastric cancer onset prediction information
312: lesion suspicious area prediction model
322: Gastric endoscopy image with predicted lesion suspicious area
324: lesion suspicious area
332: Gastric endoscopy image with predicted pathology for the suspected lesion area

Claims (19)

To a method for predicting early gastric cancer performed by a device for predicting early gastric cancer comprising a processor and a receiver,
receiving, through the receiver, an endoscopy image of a suspected gastric cancer object;
predicting, through the processor, whether or not the early gastric cancer onset for the suspected gastric cancer individual using an early gastric cancer prediction model configured to predict whether or not early gastric cancer will occur based on the gastroendoscopic image; and
providing a prediction result;
The early gastric cancer prediction model,
Further configured to predict mucosal invasion early gastric cancer or submucosal invasion early gastric cancer based on the gastroscopic image,
The step of predicting whether the early gastric cancer onset is,
Predicting, through the processor, at least one condition from the group consisting of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, and normal for the suspected gastric cancer individual using the early gastric cancer prediction model,
The step of predicting whether the early gastric cancer onset is,
Using the processor, using a lesion suspicious area prediction model configured to predict a lesion suspicious area of early gastric cancer in the gastroendoscopic image, further comprising predicting a lesion suspicious area of early gastric cancer in the upper endoscopy image, ,
The providing step is
When the lesion suspected region of the early gastric cancer is predicted in the gastroendoscopic image by the lesion suspicious region prediction model, the method further comprising the step of providing the predicted lesion suspicious region,
The lesion suspicious area prediction model is,
Further configured to predict a predetermined landmark in the gastroscopic image,
The step of predicting whether the early gastric cancer onset is,
predicting, through the processor, the landmark in the gastroendoscopic image using the lesion suspicious region prediction model; and
Further comprising the step of predicting the location of the lesion suspicious area based on the landmark,
The providing step is
The method of predicting early gastric cancer, further comprising the step of providing the predicted location of the suspected lesion region. delete According to claim 1,
The step of predicting whether the early gastric cancer onset is,
Calculating, through the processor, the probability of early gastric cancer with mucosal invasion, the probability of early gastric cancer with submucosal invasion, and the probability of being normal with respect to the suspected gastric cancer individual using the early gastric cancer prediction model, and
A method for predicting early gastric cancer, comprising the step of determining at least one condition of the group with respect to the suspected gastric cancer individual based on the calculated probability of the gastric cancer suspected individual. delete According to claim 1,
The step of predicting whether the early gastric cancer onset is,
calculating, through the processor, a probability of being a lesion area or a probability of being normal with respect to a specific area in the upper endoscopy image using the lesion suspicious area prediction model;
The method of predicting early gastric cancer, further comprising the step of determining the specific area as a lesion suspicious area when the probability that the specific area is the lesion area is greater than or equal to a predetermined level. 6. The method of claim 5,
The lesion suspicious area prediction model is,
It is further configured to predict a lesion area of mucosal invasion early gastric cancer or a lesion area of submucosal invasion early gastric cancer within the gastroscopic image,
The step of predicting whether the early gastric cancer onset is,
When the suspected lesion region is determined by the lesion suspicious region prediction model through the processor,
Calculating a probability of a lesion region of an early gastric cancer invasion of the mucosal layer and a probability of a lesion region of an early gastric cancer invading the submucosal layer with respect to the suspected lesion region using the lesion suspicious region prediction model, and
Further comprising the step of determining the condition of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer with respect to the lesion suspicious region based on the calculated probability of the lesion suspicious region,
The providing step is
The method of predicting early gastric cancer, further comprising the step of providing a condition for the determined suspected lesion region. delete According to claim 1,
Receiving the upper endoscopy image comprises:
Receiving a plurality of gastroscopic images of the suspected gastric cancer object through the receiving unit,
The step of predicting whether the early gastric cancer onset is,
Predicting the landmark for each of a plurality of gastroscopic images by using the lesion suspicious region prediction model through the processor, and
Further comprising the step of predicting the location of the suspected lesion region for each of the plurality of gastroendoscopic images based on the landmark,
The providing step is
The method further comprising the step of providing the predicted location of the suspected lesion region with respect to each of the plurality of gastroendoscopic images, early gastric cancer prediction method. According to claim 1,
Through the processor, an endoscopic image of early gastric cancer with mucosal invasion including the lesion region of mucosal invasion early gastric cancer, a submucosal invasion endoscopic image including the lesion region of submucosal invasion early gastric cancer, and a normal gastroscopic image of a normal individual. The method of predicting early gastric cancer, further comprising: learning the early gastric cancer prediction model to predict whether or not early gastric cancer will occur with respect to the learning gastroscopic image based on at least one learning gastroscopic image selected from among. 10. The method of claim 9,
The learning step is
normalizing, through the processor, the pixels of the upper endoscopic image for learning to a predetermined level;
masking the lesion region of the mucosal invasion early gastric cancer and the lesion region of the submucosal invasion early gastric cancer with respect to each of the mucosal invasion early gastric cancer endoscopic image and the submucosal invasion early gastric cancer endoscopic image among the upper endoscopy images for learning;
masking a background area excluding a predetermined landmark different from the lesion area with respect to the normal gastroscopic image of the upper endoscopy image for learning; and
Based on the masked lesion region and the background region, the early gastric cancer prediction model comprises learning to predict at least one condition from the group consisting of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, and normal. prediction method. 11. The method of claim 10,
The standardizing step is
Cropping through the processor to include only the lesion region of the mucosal invasion early gastric cancer and the lesion region of the submucosal invasion early gastric cancer for each of the mucosal invasion early gastric cancer endoscopic image and the submucosal invasion early gastric cancer endoscopic image step, and
A method of predicting early gastric cancer, comprising normalizing pixels of a cropped mucosal invasion early gastric cancer endoscopic image, a cropped submucosal invasion early gastric cancer endoscopic image, and the normal gastric endoscopic image to a predetermined level. 12. The method of claim 11,
The cropping step is
and cropping through the processor while maintaining a ratio of the lesion area of the mucosal invasion early gastric cancer and the lesion area of the submucosal invasion early gastric cancer. According to claim 1,
Receiving the upper endoscopy image comprises:
Receiving a plurality of gastroscopic images of the suspected gastric cancer object through the receiving unit,
The step of predicting whether the early gastric cancer onset is,
Predicting whether or not early gastric cancer will occur for each of the plurality of gastroendoscopic images by using the early gastric cancer prediction model through the processor,
The providing step is
A method of predicting early gastric cancer, comprising the step of providing prediction results for each of the plurality of gastroscopic images in real time. An endoscopic imaging unit configured to acquire a gastroscopic image of a suspected gastric cancer object, and
and a processor operatively connected to the endoscopic image capturing unit,
The processor is
Using an early gastric cancer prediction model configured to predict the onset of early gastric cancer based on the gastroendoscopic image, it is configured to predict whether or not the early gastric cancer onset for the suspected gastric cancer individual,
The early gastric cancer prediction model,
Further configured to predict mucosal invasion early gastric cancer or submucosal invasion early gastric cancer based on the gastroscopic image,
The processor is
Using the early gastric cancer prediction model, it is further configured to predict at least one condition from the group consisting of mucosal invasion early gastric cancer, submucosal invasion early gastric cancer, and normal for the suspected gastric cancer individual,
The processor is
Using a lesion suspicious area prediction model configured to predict a lesion suspicious area of early gastric cancer in the gastroendoscopic image, it is further configured to predict a lesion suspicious area of early gastric cancer in the gastroscopic image,
The lesion suspicious area prediction model is,
Further configured to predict a predetermined landmark in the gastroscopic image,
The processor is
The device for predicting early gastric cancer, further configured to predict the landmark in the gastroendoscopic image using the lesion suspicious region prediction model and predict the location of the lesion suspicious region based on the landmark. delete delete 15. The method of claim 14,
The lesion suspicious area prediction model is,
It is further configured to predict a lesion area of mucosal invasion early gastric cancer or a lesion area of submucosal invasion early gastric cancer within the gastroscopic image,
The processor is
When the suspected lesion region is determined by the lesion suspicious region prediction model,
Using the suspected lesion region prediction model, the probability of a lesion region of mucosal invasion early gastric cancer and a lesion region of submucosal invasion early gastric cancer are respectively calculated for the suspected lesion region, and the calculated probability of the suspected lesion region is calculated. The device for predicting early gastric cancer, further configured to determine the condition of mucosal invasion early gastric cancer or submucosal invasion early gastric cancer with respect to the suspected lesion region based on the lesion suspicious region. delete 15. The method of claim 14,
The device for early gastric cancer prediction, further comprising a laser irradiator configured to irradiate a laser for treatment to the suspected lesion region.

Images