KR102095730B1 - Method for detecting lesion of large intestine disease based on deep learning - Google Patents

Abstract

According to an embodiment of the present invention, a method of detecting a lesion of large intestine based on deep learning comprises the following steps of: (A) generating a diagnosis model by training a lesion of a large intestine tissue through deep learning by using learning data; (B) inputting a diagnosis target image; (C) detecting a lesion part from the diagnosis target image based on the diagnosis model; and (D) displaying the lesion part on the diagnosis target image, wherein the step (A) includes the following steps of: (A1) inputting, as first learning data, a plurality of tissue element images obtained by subdividing a large intestine tissue in each large intestine image for each tissue element; (A2) generating a tissue element classification model through deep learning using the first learning data as input; (A3) inputting, as second learning data, a normal image without a lesion and a lesion image with a lesion for each tissue element; and (A4) generating a lesion detection model for detecting a lesion part for each tissue element through deep learning using the second learning data as input. In addition, in the step (C), it is desirable to apply the tissue element classification model and the lesion detection model as a diagnosis model.

Description

METHOD FOR DETECTING LESION OF LARGE INTESTINE DISEASE BASED ON DEEP LEARNING}

The present invention relates to a method for detecting colon lesions based on deep learning, and more specifically, to learn whether or not lesions of colon tissues are learned through deep learning technology, and deep learning based colon lesions capable of displaying lesions on an image to be diagnosed. It relates to a detection method.

Colorectal cancer is a common carcinoma that ranks second in cancer rates in 2018 (as of 2018). With the development of westernized eating habits and diagnostic technology, the incidence of colorectal cancer is increasing more rapidly. 80-90% of colorectal cancer is a small lump on the large intestine that originated from the polyps (adenomas). Early detection and removal of this polyp through colonoscopy will reduce mortality from colon cancer by 66%.

Professional medical staff diagnoses the patient's illness through various tests and suggests treatment. In some cases, for more accurate diagnosis, a radiographic image or a pathological image obtained by scanning a tissue slide is read.

However, it is really difficult to find an abnormal area of 100 × 100 pixels in the image of 100,000 × 100,000 pixels with the human eye. It is not easy to distinguish whether or not the tumor is normal with the naked eye even by a skilled practitioner who has undergone a professional training course, and depending on the image, analysis may take several tens to several hours.

In addition, situations that often miss an unusual case often occur. The large intestine is long, curved, and has many wrinkled shapes. This is why it is difficult for an expert to observe the entire surface of the large intestine perfectly. If there are small lesions inside these bends, it is not easy to find, and it is difficult to easily discriminate minute changes in the colon mucosa or cancer cells of 1 mm in size.

Also, if it is difficult to read, the number of images that a doctor can analyze per day is only 2-3 people at most. Since it is a human eye check, it is possible to make different diagnoses for the same patient.

Given the large amount of information a specialist needs to review within a limited time, the possibility of misdiagnosis cannot be completely ruled out. Representatively, it is known that the cause of cancer misdiagnosis is the highest rate of negligence of additional tests or a false positive.

In addition, the smaller the size of the polyp, the higher the tendency to overlook, and for this reason, cases of colon cancer are often diagnosed within a few years after colonoscopy.

In addition, the possibility that the image may not be completely displayed depending on factors such as resolution, contrast ratio, and luminance supported by the monitor cannot be completely excluded. Reading monitors must support elements such as medical digital imaging and communications standards (DICOM), 5 million pixels, high illumination, and high resolution with at least 8-bit conditions, but in reality, purchase or use is limited due to expensive equipment. It is likely to be. To this end, deep learning has been introduced for the purpose of efficiently reading medical images.

Artificial intelligence (AI) technology based on machine learning, such as deep learning, has been the basis for bringing about rapid advances in diagnosing patients' diseases using medical images.

Deep learning refers to a machine learning method based on an artificial neural network that allows a machine to learn by simulating a human neuron. Recently, deep learning technology has been rapidly developed in the field of image recognition, and is widely used in the field of diagnosis using medical images.

In deep learning technology, a learning model is formed by repeatedly learning learning data to form a diagnostic model for diagnosing a disease. It is important to develop a diagnostic model specialized for each disease because various types of diseases are used as learning data.

If a deep learning algorithm that recognizes polyps by analyzing colon images is introduced, it is expected to greatly help to significantly reduce the probability of missing an abnormal case when reading colon diseases. In addition, even a colonoscopy tester with low technical skill is expected to be able to detect and ablate polyps with high accuracy if this system is used, and it is necessary to develop a technique for reading colon images using deep learning technology.

In the 'medical image-based disease diagnosis information calculation method and apparatus' disclosed in Korean Patent Publication No. 10-2017-0017614, the region of interest in which an object to be analyzed is photographed is detected, a variation coefficient is calculated, and a variation coefficient image is prepared. And comparing it with a reference sample, and refers to an effect of diagnosing a patient's disease degree by using medical images obtained through CT, MRI, and ultrasound imaging devices.

Accordingly, the present invention was devised to solve the above-described problems, and by learning whether or not lesions of the large intestine tissues are performed through deep learning technology, deep learning based large intestines capable of automatically and more accurately detecting lesions in the large intestine tissues. The aim is to provide a method for detecting lesions.

A deep learning based colon lesion detection method according to an embodiment of the present invention includes: (A) deep learning of a lesion of a colon tissue using learning data, thereby generating a diagnostic model; (B) inputting an image to be diagnosed; (C) based on the diagnostic model, the lesion site is detected in the image to be diagnosed; And (D) comprising the step of displaying the lesion site in the image to be diagnosed, (A) step, (A1) a plurality of tissue elements, and a plurality of tissue elements that are divided into the tissues of the large intestine tissue in each large intestine image Inputting the image as first learning data; (A2) a step of generating an organizational element classification model through deep learning using the first learning data as an input; (A3) for each tissue element, a normal image without a lesion and a lesion image with a lesion are input as second learning data; And (A4) through deep learning using the second learning data as input, a lesion detection model for detecting a lesion site for each tissue element is generated, and in step (C), the tissue element classification model and the lesion detection model are included. It is preferably applied as a diagnostic model.

In one embodiment of the present invention, in step (A2), the plurality of tissue element images are divided into images of mucosal, mucosal myocardial and submucosal tissues, which are tissue elements of the large intestine tissue, and the first learning data is pre-registered classification. It is desirable to deep-learn through an algorithm.

In one embodiment of the present invention, in step (A4), the second learning data is divided into tissue elements, mucosal learning data, mucosal myocardial learning data, submucosal tissue learning data, and step (A4), the mucosal normal image And a mucosal lesion image is input as mucosal learning data, and a mucosal lesion detection model is generated through deep learning using mucosal learning data as input; A mucosal myocardial lesions detection model is inputted as mucosal myocardial lesions learning data, and a mucosal myocardial lesion detection model is generated through deep learning using mucosal myocardial learning data as an input; And it is preferable that the submucosal tissue normal image and the mucosal myocardial lesion image are input as submucosal tissue learning data, and a submucosal lesion detection model is generated through deep learning using submucosal tissue learning data as input.

In one embodiment of the present invention, step (C) comprises: (C1) generating a mucosal image, a mucosal myocardial image and a submucosal tissue image from a diagnosis target image through a tissue element classification model from the diagnosis target image; (C2) detecting a mucosal lesion site from a mucosal image through a mucosal lesion detection model; (C3) detecting a lesion site of the mucosal myocardium from the mucosal myocardial image through a mucosal myocardial lesion detection model; And (C4) through the submucosal tissue lesion detection model, it is preferable to include the step of detecting the lesion site of the submucosal tissue from the submucosal tissue image.

In one embodiment of the present invention, step (D) is, on the image to be diagnosed, through the pre-registered region extraction-merge algorithm, the lesion site of the mucous membrane, the lesion site of the mucosal myocardium detected through the stage (C). Alternatively, it is preferable to mark the lesion site of the submucosal tissue.

In one embodiment of the present invention, step (C), (C0) it is preferable to further include the step of image processing the same size as the size of the colon image.

The (C0) step may include, through a morphology algorithm, removing noise existing on the image to be diagnosed; Generating a colon tissue image by extracting a colon tissue from an image to be diagnosed through a histogram algorithm; Image processing with a constant color intensity of the large intestine tissue image through a color normalization algorithm; And resizing the colon tissue image to the same image size as the colon image.

In one embodiment of the present invention, the step of generating an image of the large intestine comprises: obtaining a horizontal coordinate value for a first start point and a first end point of the horizontal axis from a horizontal axis histogram indicating the color intensity of a large horizontal axis in an image to be diagnosed. ; Obtaining vertical coordinate values for a second start point and a second end point of the vertical axis from a vertical axis histogram indicating the color intensity of the vertical axis for the image to be diagnosed; And generating a colon tissue image for each colon tissue present on the diagnosis target image by calculating the ordinate and abscissa values.

According to the configuration as described above, according to the present invention, two processes of the diagnosis process, that is, the process of extracting the images of the mucosal, mucosal myocardial and submucosal tissues, which are the elements of the large intestine, from the image to be diagnosed by the tissue element classification model, Through the process of detecting the lesion site in the mucous membrane, mucosal myocardium, and submucosal tissue, it is possible to indicate which part of the colon tissue on the image to be diagnosed exists.

In the present invention, a lesion site is displayed on an image to be diagnosed, which is an original image of colon tissue, and a lesion site that is difficult for a doctor to read with the naked eye can be detected, thereby improving diagnosis accuracy.

1 is a view for explaining a deep learning-based colon lesion detection method according to the present invention,
2 is a diagram for explaining a process of generating a tissue element classification model in a deep learning method according to an embodiment of the present invention.
3 is a diagram for explaining a process of generating a lesion detection model in a deep learning method according to an embodiment of the present invention.
4 and 5 are diagrams for explaining a process of detecting a lesion site in colon tissues according to a diagnostic model of a deep learning-based colon lesion detection method according to the present invention.
6 to 8 are views for explaining a process of processing an image to be diagnosed in an embodiment of the present invention.
9 is a diagram for explaining a U-NET algorithm, which is an example of a region extraction-merge algorithm in an embodiment of the present invention.
10 is an example of a diagnosis result according to a deep learning-based method for detecting colon lesions of the present invention.
11 to 17 are examples of colon images by disease types of the colon.

The present invention can be applied to various changes and may have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described herein, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

Before explaining the present invention, a brief description of colon tissue and colon disease is as follows.

Colon disease is classified according to whether the lesion occurs in the mucosal, mucosal myocardium, or submucosal tissues, which are elements of the colon tissue.

The large intestine tissue (see FIG. 11 (a)) is largely divided into mucosa, mucosal myocardium (muscularis mucosa), and submucosal tissue (submucosa). As shown in Fig. 11 (b), there are a plurality of glands inside the mucous membrane. The gland of the “normal” large intestine has a round shape.

Mucosal myocardium exists between the mucosa and submucosal tissue. Figure 8 (b) is a mucosal image of the colon in a "normal" state without colon disease.

Colon diseases are classified into normal (see FIGS. 11 to 13), benign, or malignant, depending on the mucosal, mucosal myocardial, and submucosal tissue deformities. When the colon disease is positive (see FIGS. 14 and 15), shape and cell deformation of the gland in the mucosa occurs.

In the case of adenocarcinoma (malignant, see FIG. 16), the disease of the large intestine disease is a lesion in which the damaged tissue penetrates the mucosal myocardium as the mucous membrane is damaged. In addition, when the pathology of colon disease is carcinoid (malignant, see 17), the submucosal tissue has a linearly deformed lesion.

As described above in the background art, even if the type of disease is read through a medical image, there is a possibility that minute changes in the colon mucosa or cancer cells having a size of 1 mm are difficult to be easily discerned with the naked eye, and thus cannot accurately grasp the lesion site.

The present invention has been devised to solve the above problems, and based on deep learning technology, when a lesion is present in the large intestine tissue on the image to be diagnosed, the lesion site is displayed on the image to be diagnosed to improve the accuracy of the medical diagnosis. To improve.

As a deep learning technology, a convolutional neural network (CNN) method is applicable. The CNN method is a technique applied to an image by utilizing the characteristics of the neural network created and studied by the human brain neural network, and is an algorithm of recognizing objects in an image by calculating features by extracting pixels and regions of the image.

Referring to FIG. 1, a method of detecting deep bowel-based colon lesions according to the present invention will be described as follows.

When learning data is input (S10), colon disease is deep-learned using the learning data (S20). As a result of deep learning, a diagnostic model is generated (S30). Subsequently, when an image to be diagnosed in which the lesion site is not read is input (S40), the lesion site is detected based on the diagnostic model (S50).

Referring to S50 with reference to FIG. 4, the image to be diagnosed is image-processed to the same size as the large intestine image (S51). Thereafter, based on the tissue element classification model, mucosal images, mucosal myocardial images, and submucosal tissue images are extracted from the diagnosis target image (S52). Based on the mucosal lesion detection model, a lesion site existing on the mucous membrane is detected from the mucosal image (S53). Then, on the basis of the mucosal myocardial lesion detection model, a lesion site existing in the mucosal myocardium is detected from the mucosal myocardial image (S53). Based on the submucosal lesion detection model, a lesion site present in the submucosal tissue is detected from the submucosal tissue image (S53). The detected lesion site is marked with a lesion site on the original diagnosis target image (S54). The diagnosis target image in which the lesion site is displayed is output as a diagnosis result (S60).

Referring to FIGS. 2 and 3, a process (S10 to S30) of generating a diagnostic model in a method for detecting colon lesions based on deep learning according to the present invention will be described. The diagnostic model is divided into a tissue element classification model and a lesion detection model.

First, referring to FIG. 2, the tissue element classification model will be described as follows.

The first learning data is input to the organization element classification unit 110. The first learning data is a plurality of large intestine images and a plurality of tissue element images for each large intestine image. The plurality of tissue element images are divided into mucosal images, mucosal myocardial images, and submucosal tissue images.

In the organization element classification unit 110, the first learning data is deep-learned through a pre-registered classification algorithm. The organization element classification unit 110 generates an organization element classification model using the first learning data as input.

The tissue element classification model learns mucosal, mucosal myofascial and submucosal tissues from the colon tissue on the input colon image, extracts, creates and classifies images for each tissue element. The machine-learned tissue element classification model extracts a mucosal region, a mucosal myofascial region, and a submucosal tissue region from the large intestine tissue on the diagnosis target image when a diagnosis target image is input, thereby generating a mucosal image, a mucosal myocardial image, and a submucosal tissue image.

Referring to Figure 3, the description of the lesion detection model is as follows.

The second learning data is input to the lesion detection unit 120. The lesion detection unit includes a mucosal lesion detection unit 121, a mucosal myocardial lesion detection unit 123 and a submucosal tissue lesion detection unit 125. The second learning data is divided into organizational elements, such as mucosal learning data, mucosal myocardial learning data, and submucosal organizational learning data.

The mucosal lesion detection unit 121 generates a mucosal lesion detection model through deep learning using mucosal learning data as input. The mucosal lesion detection model is for detecting a lesion site due to deformation of the mucous membrane from an input mucosal image. The mucosal learning data are normal images of the mucous membrane (see FIGS. 11 (b), 12 and 13) and lesion images of the mucous membrane (see FIGS. 14 and 15).

Referring to Figure 11 (b) to 15 are as follows. Figure 11 (b) is a case where the mucous membrane is completely "normal", the gland inside the mucosa is entirely small, maintains a round shape, and the cytoplasm inside the gland also maintains a round shape.

12 is a case where the mucous membrane is inflamed, the gland within the mucous membrane maintains a round shape, but the nucleus, the black border of the gland is damaged.

FIG. 13 shows that in the case of hyperplasia / proliferative polyp, the nucleus of the gland (black border) in the mucous membrane is not damaged, but the lumen of the gland is elongated or deformed in a serrated shape.

FIG. 14 is an image of adenoma with low grade dysplasia, in which the nucleus of the gland in the mucous membrane grows toward the cytoplasm, and the overall shape of the gland is elongated.

15 is an image of adenoma with high grade dysplasia, in which the nucleus of the gland in the mucous membrane grows toward the cytoplasm, the entire shape of the gland is elongated, and fusion between the glands in the mucosa occurs. Yes.

The mucosal myopia lesion detection unit 123 generates a mucosal myopia lesion detection model through deep learning using mucosal myopia learning data as an input. The mucosal myocardial lesion detection model is for detecting a lesion site due to deformation of the mucosal myocardium from an input mucosal myocardial image. The mucosal myocardial learning data are the mucosal myocardial normal image and the mucosal myocardial lesion image (see FIG. 16).

FIG. 16 is an image of adenocarcinoma of the large intestine, and when adenocarcinoma occurs in the large intestine, the mucous membrane is damaged and the damaged tissue penetrates the mucosal myocardium and the mucosal myocardium collapses.

The submucosal lesion detection unit 125 generates a submucosal lesion detection model through deep learning using submucosal tissue learning data as input. The submucosal lesion detection model is for detecting a lesion site due to deformation of the submucosal tissue from the input submucosal tissue image. Submucosal tissue learning data are submucosal tissue normal images and mucosal myocardial lesion images (see Figure 17).

Referring to FIG. 17, when the colon tissue is carcinoid, the submucosal tissue is linearly deformed. In the case of carcinoids, mucosal and mucosal myocardial deformation does not occur.

Hereinafter, a process of detecting and displaying a lesion site from an image to be diagnosed will be described with reference to FIG. 5.

First, before the image to be diagnosed is input to the tissue element classification unit 110, image processing for the image to be diagnosed is preceded. The input diagnostic image (see FIG. 5 (a)) is subjected to a first resizing process in which the file format is changed and the image size is reduced. Then, through the morphology algorithm and the histogram algorithm, the colon tissue is extracted from the first resized image (see FIG. 5 (b)) to generate the colon tissue image.

Referring to FIGS. 6 to 7, a description will be given of a process of extracting only a portion corresponding to the large intestine tissue from the diagnosis target image 10 in which a plurality of large intestine tissues exist.

Multiple colon tissues suspected of having lesions are imaged. Subsequently, as shown in FIG. 6 (a), multiple colon tissue images are merged into one diagnostic target image 10. FIG. 6 is an exemplary photograph of a case where two large intestine tissues are present in one diagnosis target image 10.

Through the morphology opening operation, noise existing on the image is removed from the image to be diagnosed (see FIG. 6 (a)) (see FIG. 6 (b)). Thereafter, the size of the region from which noise is removed is expanded through a morphology dilation operation (see FIG. 6 (c)).

For reference, referring to FIG. 8 (a), the morphology opening is described, and the morphology opening considers the figure 42, which is a reference in a certain area 40, as a brush, and displays an area that does not reach when painting. It is an erasing technique. Referring to FIG. 8 (b), morphology dilation is described, the morphology dilation increases the area 45 when painting by aligning the reference figure 42 with the area border portion 41 It is a technique.

When noise is removed from the image 10 to be diagnosed through a morphology algorithm, a tissue is extracted so that a plurality of large intestine tissues are present only through a histogram algorithm.

The process of extracting the target tissue images 20 and 30 from the diagnosis target image 10 through the histogram algorithm is as follows.

First, as illustrated in FIG. 6 (d), horizontal coordinate values for the first start point and the first end point of the horizontal axis are obtained from a horizontal axis histogram indicating the color intensity of the horizontal axis in the image to be diagnosed.

Referring to FIG. 6 (e), two colon tissue images are extracted with a region between a start point and an end point based on the horizontal axis.

Each large intestine tissue image is once again eliminated through a morphology algorithm. FIG. 7 (b) is the same as FIG. 6 (b), and FIG. 7 (c) is the same as FIG. 6 (c).

Then, through the histogram algorithm, vertical coordinate values for the second start point and the second end point of the vertical axis are obtained from the vertical axis histogram indicating the color intensity of the vertical axis for the image to be diagnosed.

Subsequently, the histogram algorithm computes the ordinate and abscissa values to generate a colon tissue image for each colon tissue present on the diagnosis target image (see FIG. 7 (e)).

Meanwhile, the large intestine tissue image (see FIG. 5 (c)) image-processed by the above process is image-processed with a constant color intensity through a color normalization algorithm. Because medical pathology images have different colors depending on the concentration of chromosomes, conversion processing is performed with a constant color intensity.

The color normalized image (see FIG. 5 (d)) is subjected to a color bit reduction process that converts 24-bit to 8-bit color to reduce the file size for the deep learning model to learn, and then has the same size as the image size of the training data. The second is resized. The second resized image is input to the tissue element classification unit 110.

The tissue element classification unit 110 generates a mucosal image, a mucosal myocardial image, and a submucosal tissue image from a diagnosis object image through a tissue element classification model through a tissue element classification model.

As shown in Figure 5 (g), the mucosal image is input to the mucosal lesion detection unit 121. The mucosal image is an image generated by extracting only the mucosal region of the large intestine tissue on the image to be diagnosed. The mucosal lesion detection unit 121 detects a lesion site of the mucosa from the mucosal image through the mucosal lesion detection model.

The mucosal myocardial image is input to the mucosal myocardial lesion detection unit 123. The mucosal myocardial image is an image created by extracting only the mucosal myocardial region of the large intestine tissue on the diagnosis target image. The mucosal myocardial lesion detection unit 123 detects a lesion site of the mucosal myocardium from the mucosal myocardial image through a mucosal myocardial lesion detection model.

The submucosal tissue image is input to the submucosal lesion detection unit 125. The submucosal tissue image is an image generated by extracting only the submucosal region of the large intestine tissue on the diagnosis target image. The submucosal lesion detection unit 125 detects a lesion site for a submucosal tissue from an image of the submucosal tissue through a submucosal lesion detection model.

The mucosal image, the mucosal myocardial image, and the submucosal tissue image showing the lesion site are merged with the image to be diagnosed (original image) through a region extraction-merge algorithm registered in the lesion detection unit. Through this, the lesion site of the mucous membrane, the lesion site of the mucosal myocardium, or the lesion site of the submucosal tissue is displayed on the diagnosis target image (the original image) (see FIGS. 5 (h) and 10).

 The region extraction-merge algorithm is an algorithm that merges and processes mucosal images, mucosal myofascial images, and submucosal tissue images into a single image. U-NET algorithm is applicable as the region extraction-merge algorithm. However, the region extraction-merge algorithm is not limited to the U-NET algorithm, and a region extraction-merge algorithm can be used if it is an algorithm that merges and processes multiple images within the scope apparent to those skilled in the art.

Referring to FIG. 9, the U-NET algorithm is a “U” shaped algorithm that analyzes an original image, extracts a desired region, and displays the region on the original image. The U-NET algorithm is a specialized CNN technology that can display a trained area on an image.

According to the configuration as described above, according to the present invention, two processes of the diagnosis process, that is, the process of extracting the images of the mucosal, mucosal myocardial and submucosal tissues, which are the elements of the large intestine, from the image to be diagnosed by the tissue element classification model, Through the process of detecting the lesion site in the mucous membrane, mucosal myocardium, and submucosal tissue, it is possible to indicate which part of the colon tissue on the image to be diagnosed exists.

Referring to FIG. 10, the lesion site is indicated by a purple block, and it can be seen that there is a lesion in the mucosal portion. The doctor may diagnose the colon disease by more accurately recognizing the lesion name and the lesion region of the colon disease by referring to the image of the diagnosis target where the lesion region is displayed as shown in FIG. 10.

Although some embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that the present invention may be modified without departing from the principles or spirit of the present invention. . The scope of the invention will be defined by the appended claims and their equivalents.

110: organization element classification unit
120: lesion detection unit
121: mucosal lesion detection unit
123: mucosal myocardial lesion detection unit
125: submucosal tissue lesion detection unit

Claims (8)

In a method performed by the information processing device,
(A) deep learning of the lesions of the large intestine tissue using the learning data, generating a diagnostic model;
(B) inputting an image to be diagnosed;
(C) based on the diagnostic model, a lesion site is detected in the image to be diagnosed; And
(D) the step of displaying the lesion site in the image to be diagnosed,
Step (A) is,
(A1) a step of inputting a plurality of large intestine images and a plurality of tissue element images in which the large intestine tissues in each of the large intestine images are divided into tissue elements as first learning data;
(A2) through deep learning using the first learning data as an input, generating an organizational element classification model;
(A3) for each tissue element, a normal image without a lesion and a lesion image in which the lesion is present are input as second learning data; And
(A4) through deep learning using the second learning data as input, a step of generating a lesion detection model for detecting a lesion site for each tissue element,
Step (C) is
The image to be diagnosed is input to the tissue element classification model and classified according to the tissue element image.
A method of detecting colon lesions based on deep learning, characterized in that each of the tissue element images classified by the tissue element classification model is input to the lesion detection model and a lesion site is detected for each tissue element image. According to claim 1,
In the step (A2), the plurality of tissue element images are divided into images of mucosal, mucosal myofascial and submucosal tissues that are tissue elements of the large intestine tissue,
The first learning data is deep learning based on the pre-registered classification algorithm, deep learning based colon lesion detection method. According to claim 2,
In the step (A4), the second learning data is divided into mucosal learning data, mucosal myocardial learning data, and submucosal tissue learning data for each organizational element,
Step (A4) is,
A mucosal lesion image and a mucosal lesion image are input as the mucosal learning data, and a mucosal lesion detection model is generated through deep learning using the mucosal learning data as input;
A mucosal myocardial lesion image is inputted into the mucosal myocardial lesion learning image, and a mucosal myocardial lesion detection model is generated through deep learning using the mucosal myocardial learning data as input; And
Deep learning, characterized in that the submucosal tissue normal image and the mucosal myocardial lesion image are input as the submucosal tissue learning data, and a submucosal lesion detection model is generated through deep learning using the submucosal tissue learning data as input. -Based method for detecting colon lesions. The method of claim 3, wherein (C) step,
(C1) generating a mucosal image, a mucosal myocardial image and a submucosal tissue image from the diagnosis target image through the tissue element classification model;
(C2) detecting a lesion site of the mucous membrane from the mucosal image through the mucosal lesion detection model;
(C3) detecting a lesion site of the mucosal myocardium from the mucosal myocardial image through the mucosal myocardial lesion detection model; And
(C4) A deep learning based colon lesion detection method comprising the step of detecting a submucosal lesion site from the submucosal tissue image through the submucosal lesion detection model. The method of claim 4,
The (D) step, through a pre-registered region extraction-merge algorithm, on the diagnosis target image, the lesion site of the mucous membrane detected through the step (C), the lesion site of the mucosal myocardium, or the submucosal tissue Deep learning based colon lesion detection method, characterized in that to display the lesion site. The method of claim 1, wherein (C) step,
(C0) Deep learning-based colon lesion detection method, characterized in that it further comprises the step of processing the image to be the same size as the size of the colon image. The method of claim 6, wherein (C0) step,
Removing noise existing on the diagnosis target image through a morphology algorithm;
Generating a colon tissue image by extracting a colon tissue from the diagnosis target image through a histogram algorithm;
Image processing with a constant color intensity of the large intestine tissue image through a color normalization algorithm; And
A method of detecting colon lesions based on deep learning, further comprising resizing the colon tissue image to the same image size as the colon image. The method of claim 7, wherein the step of generating the colon tissue image,
Obtaining horizontal coordinate values for a first start point and a first end point of the horizontal axis from a horizontal axis histogram indicating the color intensity of the horizontal axis on the diagnosis target image;
Obtaining vertical coordinate values for the second start point and the second end point of the vertical axis from a vertical axis histogram representing the color intensity of the vertical axis for the image to be diagnosed; And
And generating the colon tissue image for each colon tissue present on the diagnosis target image by calculating the ordinate value and the abscissa coordinate value.

Images