KR102513394B1 - Device and Method for Generating Synthetic Endoscope Image Using General Adversarial Network - Google Patents

Abstract

The present invention provides a first normal endoscopic image 211a, a second normal endoscopic image 211b, a first edge image 212a, a second edge image 212b, a polyp mask 213, a similar endoscopic image 214, A learning model composed of a dataset management unit 200 that manages a dataset 210 including a plurality of endoscope images 215 and synthesized endoscope images 216, a generator 310, and a classifier 320 ( 300) and a learning unit 400 for training the generator 310 and the classifier 320 based on GAN (General Adversarial Network), and the dataset management unit 200 performs edge detection Edge extraction to generate a first edge image 212a and a second edge image 212b, respectively, in which a plurality of characteristic lines are extracted from the first normal endoscopic image 211a and the second normal endoscopic image 211b using the technique It is characterized in that it includes a polyp detection unit 230 that detects the polyp mask 213 including the polyp from the unit 220 and the first edge image 212a.

Description

Apparatus and method for generating synthetic endoscope image using GAN {Device and Method for Generating Synthetic Endoscope Image Using General Adversarial Network}

The present invention relates to an apparatus and method for generating synthetic endoscopic images, and more particularly, to synthesize a plurality of synthetic endoscopic images including malignant polyps by type using a GAN (General Adversarial Network) from normal endoscopic images taken with an endoscope of the inside of a human body. It relates to an apparatus and method for generating synthetic endoscopic images using GANs capable of generating.

GAN (Generative Adversarial Network) is a machine learning (Machine Learning) model that can automatically generate images that are almost similar to real images while generators and classifiers compete with each other.

A GAN consists of a constructor trained to create fake images to fool the discriminator as much as possible, and a discriminator trained to classify fake and real images as accurately as possible. GANs can generate fake images that are very similar to real images through an adversarial process in which generators and discriminators are sequentially and repeatedly learned.

As image generation technology using GAN is widely used in the general industry, a GAN-based endoscopic image generation device capable of generating sophisticated images that are almost similar to actual endoscopic images is being introduced in the medical field as well. .

However, existing endoscopic image generating apparatuses have a problem in that the quality of endoscopic images generated by the endoscopic image generating apparatus is degraded because noise is used as an input value.

In addition, existing image generating apparatuses do not have a means for verifying the reliability of the previously generated endoscopic image, so that the reliability of the endoscopic image is low.

KR 10-2021-0012464 B1

The present invention has been devised to solve the above problems, and an object of the present invention is to generate a synthesized endoscopic image including a malignant polyp using a GAN (General Adversarial Network) from a normal endoscopic image taken endoscopically of the inside of the human body. An object of the present invention is to provide an apparatus and method for generating synthetic endoscopic images using GAN capable of generating multiple types for each type.

In order to solve the above technical problems, the synthesized endoscopic image generating apparatus 100 using GAN according to the present invention includes a first normal endoscopic image 211a, a second normal endoscopic image 211b, and a first edge image ( 212a), second edge image 212b, polyp mask 213, similar endoscopic image 214, ideal endoscopic image 215, and synthetic endoscopic image 216. Learning the generator 310 and the classifier 320 based on the learning model 300 including the set management unit 200, the generator 310 and the classifier 320 and a general adversarial network (GAN) and a learning unit 400 for the first normal endoscopic image 211a and the second normal endoscopic image 211b using an edge detection technique. An edge extractor 220 for generating a first edge image 212a and a second edge image 212b from which lines are extracted, respectively, and a polyp mask 213 including polyps from the first edge image 212a It is characterized by including a polyp detection unit 230 for detecting.

In addition, the first normal endoscope image 211a is an image including a polyp when the inside of the human body is photographed through an endoscope, and the second normal endoscope image 211b includes a polyp when the inside of the human body is photographed through an endoscope. The abnormal endoscopic image 215 is an image that includes a malignant polyp when the inside of the human body is photographed with an endoscope.

In addition, the generator 310 generates a similar endoscopic image 214 similar to the first normal endoscopic image 211a from the first edge image 212a and the polyp mask 213, and the classifier 320 determines whether the first normal endoscopic image 211a and the similar endoscopic image 214 generated by the creator 310 are authentic, and the learning unit 400 determines whether the first normal endoscopic image 211a and classifier learning for learning the classifier 320 to receive the similar endoscopic image 214, classify the first normal endoscopic image 211a as real, and classify the similar endoscopic image 214 as fake. The generator 310 to generate a similar endoscopic image 214 similar to the first normal endoscopic image 211a so that the unit 410 and the classifier 320 can classify the similar endoscopic image 214 as genuine. ) It is characterized in that it comprises a generator learning unit 420 for learning.

In addition, the generator 310 randomly selects one of the plurality of polyp masks 213 for the second edge image 212b after the sequential iterative learning of the classifier 320 and the generator 310 is completed. It is characterized in that a synthesized endoscopic image 216 including a malignant polyp is generated by synthesizing the image.

In addition, the dataset manager 200 further includes a synthesized image verification unit 240 for verifying the reliability of the synthesized endoscopic image 216, and the synthesized image verifying unit 240 determines the ideal endoscopic image 215 and a classifier model 241 that receives a synthetic endoscopic image 216 including malignant polyps of the same type as the ideal endoscopic image 215 and outputs a predetermined number of vectors, respectively, and the synthesized image verification unit ( 240) is characterized in that a predetermined number of vectors for the ideal endoscopic image 215 and the synthesized endoscopic image 216 are input and expressed in a two-dimensional graph, respectively, using a dimensionality reduction algorithm.

In addition, the synthesized image verifying unit 240, when the distribution of the 2D graph of the synthesized endoscopic image 216 overlaps with the distribution of the 2D graph of the ideal endoscopic image 215 by more than a preset range, performs the synthesized image. It is characterized by classifying the endoscopic image 216 as a reliable endoscopic image.

In the synthetic endoscopic image generation method using GAN according to the present invention, the edge extractor 220 extracts a plurality of characteristic lines from a plurality of first normal endoscopic images 211a using an edge detection technique, respectively, a plurality of first edges In the first step of generating the image 212a, the polyp detection unit 230 uses a plurality of polyp masks 213 each including a polyp from the plurality of first edge images 212a generated by the edge extraction unit 220. ), a third step in which the learning unit 400 learns the generator 310 and the classifier 320 based on GAN, the edge extractor 220 uses an edge detection technique to A fourth step of generating a plurality of second edge images 212b in which a plurality of characteristic lines are respectively extracted from the second normal endoscopic image 211b of ) by selecting and synthesizing one of the plurality of polyp masks 213 to generate a plurality of synthesized endoscopic images 216 including malignant polyps, respectively, and the synthesized image verification unit 240 It is characterized in that it includes a sixth step of verifying the reliability of the synthesized endoscopic image 216 of.

In addition, in the third step, the generator 310 obtains a first normal image from the first edge image 212a generated by the edge extraction unit 220 and the polyp mask 213 generated by the polyp detection unit 230. In the 3-1st step of generating a similar endoscopic image 214 similar to the endoscopic image 211a, the classifier learning unit 410 generates the first normal endoscopic image 211a and the above generated by the generator 310. Step 3-2 of learning the classifier 320 to receive the similar endoscopic image 214, classify the first normal endoscopic image 211a as real, and classify the similar endoscopic image 214 as fake , The classifier 320 receives the first normal endoscopic image 211a and the similar endoscopic image 214, and determines whether the first normal endoscopic image 211a and the similar endoscopic image 214 are authentic. Step 3-3, the generator learning unit 420 converts the first normal endoscope image 211a so that the classifier 320 can classify the similar endoscopic image 214 generated by the generator 310 as real. The 3-4th step of training the generator 310 to generate the similar endoscopic image 214 similar to ) and the 3-1st step for the plurality of first edge images 212a and the polyp mask 213. In the step, it is characterized in that it includes a 3-5 step in which the 3-4 steps are repeated a predetermined number of times.

Also, in the sixth step, the synthesized image verification unit 240 converts the abnormal endoscope image 215 and the synthesized endoscope image 216 including the same type of malignant polyp as the classifier model 241 ), and in the 6-1st step of outputting a certain number of vectors, the synthesized image verification unit 240 uses a dimensionality reduction algorithm to generate the ideal endoscope image 215 and the synthesized endoscope image 216. In the 6-2 step of receiving a certain number of vectors for each and expressing them in a 2D graph, and the distribution of the 2D graph of the synthesized endoscope image 216 by the synthesized image verification unit 240 is the ideal endoscope image ( 215), the 6-3 step of classifying the synthesized endoscopic image 216 as a reliable endoscopic image when it overlaps with the distribution of the 2D graph by more than a preset range.

An apparatus and method for generating synthetic endoscopic images using GAN according to the present invention generate multiple synthetic endoscopic images for each type using a general adversarial network (GAN) from a normal endoscopic image taken by an endoscope of the inside of a human body. , it can solve the problem of lack of endoscopic images including malignant polyps in the medical field.

In addition, since the synthesized endoscopic image generation apparatus and method using GAN according to the present invention uses an edge image extracted from a normal endoscopic image and an image synthesized from a polyp mask as an input value, conventional image generation used in the medical field It can produce synthetic endoscopic images that are superior to the device.

In addition, in the synthetic endoscope image generating apparatus and method using GAN according to the present invention, only when the distribution of the two-dimensional graph of the synthetic endoscope image overlaps the distribution of the two-dimensional graph of the ideal endoscope image by more than a preset range, the synthetic endoscope Since the image is used, there is an advantage in that a highly reliable synthetic endoscopic image can be used.

1 is a block diagram of an apparatus for generating synthetic endoscopic images using GAN according to the present invention.
Figure 2 is a configuration diagram of the dataset shown in Figure 1.
FIG. 3 is a diagram for explaining a process of generating a similar endoscopic image in the synthesized endoscopic image generating apparatus shown in FIG. 1 .
FIG. 4 is a diagram for explaining a process of generating a synthetic endoscopic image in the synthetic endoscopic image generating apparatus shown in FIG. 1 .
FIG. 5 is a diagram for explaining a process of verifying a synthesized endoscopic image by the synthesized image verifying unit shown in FIG. 1 .
6 is a flowchart illustrating a method for generating a synthesized endoscopic image using GAN according to the present invention.
FIG. 7 is a flowchart for explaining step S130 shown in FIG. 6 .
FIG. 8 is a flowchart for explaining step S160 shown in FIG. 6 .

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings in order to describe in detail enough for those skilled in the art to easily implement the technical idea of the present invention.

However, the following examples are merely examples to aid understanding of the present invention, and the scope of the present invention is not reduced or limited thereby. In addition, the present invention may be embodied in many different forms and is not limited to the embodiments described herein.

First, the synthetic endoscopic image generating apparatus 100 using GAN according to the present invention will be described.

1 is a block diagram of an apparatus for generating synthetic endoscopic images using GAN according to the present invention. Referring to FIG. 1 , an apparatus 100 for generating a synthesized endoscopic image using GAN according to the present invention includes a dataset management unit 200, a learning model 300, and a learning unit 400.

Figure 2 is a configuration diagram of the dataset shown in Figure 1. Referring to FIG. 2 , the dataset management unit 200 includes a first normal endoscopic image 211a, a second normal endoscopic image 211b, a first edge image 212a, a second edge image 212b, and a polyp mask ( 213), a similar endoscopic image 214, a synthetic endoscopic image 216, and an ideal endoscopic image 215 are managed.

Here, the first normal endoscopic image 211a is an image including a polyp when the inside of the human body, such as the stomach, large intestine, and esophagus, is taken with an endoscope, and here, the second normal endoscopic image 211b is an image of the inside of the human body endoscopically. It means an image that does not contain polyps at the time of shooting. Further, the abnormal endoscopic image 215 refers to an image including a malignant polyp when the inside of the human body is photographed through an endoscope.

FIG. 3 is a diagram for explaining a process of generating a similar endoscopic image in the synthesized endoscopic image generating apparatus shown in FIG. 1 . Referring to FIG. 3 , the dataset management unit includes an edge extraction unit 220 and a polyp detection unit 230.

The edge extractor 220 generates a first edge image 212a by extracting a plurality of characteristic lines from the first normal endoscopic image 211a using an edge detection technique.

At this time, the edge detection is an algorithm for generating line segments that are highlighted in a normal endoscopic image in the form of an image, and as the threshold increases, only a large number of clear lines are created in the form of an image. For example, when the threshold value is 17, edge detection can express even detailed lines in the form of an image. On the other hand, when the threshold value is 50, edge detection can express only clear lines in the form of an image.

The polyp detection unit 230 detects the polyp mask 213 including polyps on a white background from the first edge image 212a generated by the edge extraction unit 220 .

Referring to FIG. 3 , the learning model 300 includes a generator 310 and a discriminator 320 .

First, the generator 310 generates a first edge image 212a generated by the edge extraction unit 220 and a polyp mask 213 generated by the polyp detection unit 230, similar to the first normal endoscopic image 211a. An endoscopic image 214 is created.

In addition, the classifier 320 determines whether the first normal endoscopic image 211a and the similar endoscopic image 214 generated by the creator 310 are authentic.

Further, the learning unit 400 trains the generator 310 and the classifier 320 of the learning model 300 based on a general adversarial network (GAN). The learning unit 400 includes a classifier learning unit 410 and a generator learning unit 420 .

First, the classifier learning unit 410 receives the first normal endoscopy image 211a and the similar endoscopy image 214 generated by the generator 310, and classifies the first normal endoscopy image 211a as real, The classifier 320 is trained to classify the similar endoscopic images 214 as fake.

Meanwhile, the classifier 320 learned by the classifier learning unit 410 outputs 1 when receiving the first normal endoscope image 211a. Further, when the classifier 320 receives the similar endoscopy image 214, the corresponding image is compared with the first normal endoscopy image 211a, and if the corresponding image is close to the first normal endoscopy image 211a, 1 outputs , otherwise outputs 0.

Thereafter, the generator learning unit 420 generates a similar endoscopic image similar to the first normal endoscopic image 211a so that the classifier 320 can truly classify the similar endoscopic image 214 generated by the generator 310 ( 214) to train the constructor 310.

In addition, the generator 310 learned by the generator learning unit 420 is generated from the first edge image 212a generated by the edge extraction unit 220 and the polyp mask 213 generated by the polyp detection unit 230, A similar endoscopic image 214 similar to the first normal endoscopic image 211a is recreated.

After that, the learning of the classifier 320 by the classifier learning unit 410 and the learning of the generator 310 by the generator learning unit 420 are sequentially repeated a predetermined number of times. As the learning of the classifier 320 for classifying the first normal endoscopic image 211a and the similar endoscopic image 214 and the constructor 310 for generating the similar endoscopic image 214 is sequentially repeated, the classifier ( The classification accuracy of 320 and the similarity of the similar endoscopic image 214 to the ideal endoscopic image 215 of the creator 310 are further improved.

FIG. 4 is a diagram for explaining a process of generating a synthetic endoscopic image in the synthetic endoscopic image generating apparatus shown in FIG. 1 .

Referring to FIG. 4 , after the sequential iterative learning of the classifier 320 and the generator 310 is completed, the edge extractor 220 uses an edge detection technique to obtain a second normal endoscopic image 211b. ), a second edge image 212b is generated by extracting a plurality of characteristic lines.

Thereafter, the generator 310 randomly selects and synthesizes one of the plurality of polyp masks 213 on the second edge image 212b, thereby generating a synthesized endoscopic image 216 including a malignant polyp.

FIG. 5 is a diagram for explaining a process of verifying a synthesized endoscopic image by the synthesized image verifying unit shown in FIG. 1 . Referring to FIGS. 1 and 5 , the dataset manager 200 further includes a synthesized image verification unit 240 that verifies reliability of the synthesized endoscopic image 216 .

The synthesized image verification unit 240 is a classifier that receives the abnormal endoscopic image 215 and the synthesized endoscopic image 216 including the same type of malignant polyp as the abnormal endoscopic image 215 and outputs a certain number of vectors, respectively. It is configured to include a model 241.

For example, the synthesized image verifier 240 may input the synthesized endoscopic image 216 to the classifier model 241 and output a certain number of vectors from a final layer.

Here, the same type means that the type of malignant polyp, that is, the cancer is the same. In other words, if the ideal endoscopic image 215 is a colon cancer image, the corresponding synthesized endoscopic image 216 is also a colon cancer image.

Thereafter, the synthesized image verification unit 240 receives a predetermined number of vectors for the ideal endoscopic image 215 and the synthesized endoscopic image 216 by using a dimensionality reduction algorithm, and expresses them in a two-dimensional graph, respectively. Here, the dimension reduction algorithm is an algorithm that receives a certain number of vectors as input and reduces the dimension to a specific dimension.

Thereafter, the synthesized image verifying unit 240, when the distribution of the 2D graph of the synthesized endoscopic image 216 overlaps with the distribution of the 2D graph of the abnormal endoscopic image 215 by more than a preset range, the synthesized endoscopic image (216) can be classified as a reliable endoscopic image. For example, if the distribution of the 2D graph of the synthesized endoscope image 216 overlaps the distribution of the two-dimensional graph of the abnormal endoscope image 215 by 90% or more, the corresponding synthesized endoscope image 216 is regarded as a reliable endoscope image. can be classified.

After that, the user randomly ranks the plurality of synthesized endoscopic images 216 and the ideal endoscopic image 215, and then presents them to a medical imaging expert to further verify the reliability of the plurality of synthesized endoscopic images 216. can do.

Next, a synthetic endoscopic image generation method using GAN according to the present invention will be described.

6 is a flowchart illustrating a method for generating a synthesized endoscopic image using GAN according to the present invention. Referring to FIG. 6 , the edge extraction unit 220 generates a plurality of first edge images 212a by extracting a plurality of characteristic lines from a plurality of first normal endoscopic images 211a by using an edge detection technique. .(S110)

Thereafter, the polyp detection unit 230 detects a plurality of polyp masks 213 each including a polyp from the plurality of first edge images 212a generated by the edge extraction unit 220 (S120).

After that, the learning unit 400 trains the generator 310 and the classifier 320 based on the GAN (S130).

FIG. 7 is a flowchart for explaining step S130 shown in FIG. 6 . Referring to FIG. 7 , the generator 310 outputs a first normal endoscopic image 211a from the first edge image 212a generated by the edge extraction unit 220 and the polyp mask 213 generated by the polyp detection unit 230. A similar endoscopic image 214 similar to ) is generated (S131).

Thereafter, the classifier learning unit 410 receives the first normal endoscopy image 211a and the similar endoscopy image 214 generated by the creator 310, classifies the first normal endoscopy image 211a as real, and , The classifier 320 is trained to classify the similar endoscopic image 214 as fake. (S132)

After that, the classifier 320 receives the first normal endoscope image 211a and the similar endoscopy image 214 and determines whether the first normal endoscope image 211a and the similar endoscope image 214 are genuine. (S133)

Thereafter, the generator learning unit 420 generates a similar endoscopic image similar to the first normal endoscopic image 211a so that the classifier 320 can truly classify the similar endoscopic image 214 generated by the generator 310 ( 214) to train the constructor 310. (S134)

Thereafter, steps S131 to S134 are repeated a predetermined number of times for the plurality of first edge images 212a and the polyp mask 213 (S135).

After that, the edge extractor 220 generates a plurality of second edge images 212b by extracting a plurality of characteristic lines from the plurality of second normal endoscopic images 211b by using an edge detection technique. ( S140)

Thereafter, the generator 310 selects and synthesizes one of the plurality of polyp masks 213 with the plurality of second edge images 212 to generate a plurality of synthesized endoscopic images 216 including malignant polyps, respectively. (S150)

After that, the synthesized image verifying unit 240 verifies the reliability of the plurality of synthesized endoscopic images 216 (S160).

FIG. 8 is a flowchart for explaining step S160 shown in FIG. 6 . Referring to FIG. 8 , the synthesized image verification unit 240 inputs an abnormal endoscopic image 215 and a synthesized endoscopic image 216 including the same type of malignant polyp as the abnormal endoscopic image 215 into a classifier model 241. Then, a certain number of vectors are output. (S161)

After that, the synthesized image verification unit 240 receives a certain number of vectors for the ideal endoscopic image 215 and the synthesized endoscopic image 216 by using a dimensionality reduction algorithm, and expresses them in a 2D graph, respectively. ( S162)

Thereafter, the synthesized image verification unit 240, when the distribution of the 2D graph of the synthesized endoscope image 216 overlaps with the distribution of the 2D graph of the abnormal endoscope image 215 by more than a preset range, synthesizes the endoscope image ( 216) is classified as a reliable endoscopic image. (S163)

The synthetic endoscopic image generating apparatus 100 and method using GAN according to the present invention generate a general adversarial network (GAN) from a first normal endoscopic image 211a and a second normal endoscopic image 211b taken by an endoscope of the inside of a human body. By generating a plurality of synthetic endoscopic images 216 including malignant polyps for each type using , it is possible to solve the problem of lack of endoscopic images including malignant polyps in the medical field.

In addition, the synthetic endoscopic image generating apparatus 100 and method using GAN according to the present invention use edge images 212a and 212b and polyp mask 213 extracted from non-noise normal endoscopic images 211a and 211b as input values. Since the synthesized image is used, it is possible to generate a synthesized endoscopic image 216 that is superior to conventional image generating devices used in the medical field.

In addition, in the synthetic endoscopic image generating apparatus 100 and method using the GAN according to the present invention, the distribution of the 2D graph of the synthetic endoscopic image 216 is the distribution of the 2D graph of the ideal endoscopic image 215 and a preset range In the case of overlapping more than one, there is an advantage in that a highly reliable synthesized endoscope image 216 can be used because the synthesized endoscope image is used.

As described above, the main technical idea of the present invention is to provide a synthetic endoscopic image generating apparatus 100 and method using GAN, and the embodiment described above with reference to the drawings is only one embodiment, and the present invention The scope of the true rights of the invention will be based on the scope of the patent claims, but will also extend to equivalent embodiments that may exist in various ways.

100: Synthetic endoscopic image generating device using GAN
200: dataset management unit
210: dataset
211a: first normal endoscopic image
21ba: first normal endoscopic image
212a: first edge image
212b: second edge image
213: polyp mask
214 Similar endoscopic image
215 Abnormal endoscopic image
216: synthetic endoscopic image
220: edge extraction unit
230: polyp detection unit
240: composite image verification unit
241: classifier model
300: learning model
310: Constructor
320: classifier
400: learning unit
410: classifier learning unit
420: constructor learning unit

Claims (9)

A first normal endoscopic image 211a, a second normal endoscopic image 211b, a first edge image 212a, a second edge image 212b, a polyp mask 213, a similar endoscopic image 214, and an abnormal endoscopic image a dataset management unit 200 that manages a dataset 210 including a plurality of endoscopic images 215 and synthesized endoscopic images 216;
a learning model 300 comprising a generator 310 and a classifier 320; and
A learning unit 400 for learning the generator 310 and the classifier 320 based on a General Adversarial Network (GAN);
The dataset management unit 200
A first edge image 212a and a second edge image 212b obtained by extracting a plurality of characteristic lines from the first normal endoscopic image 211a and the second normal endoscopic image 211b using an edge detection technique Edge extraction unit 220 for generating each; and
A synthetic endoscopic image generating apparatus using GAN, comprising: a polyp detection unit 230 for detecting a polyp mask 213 containing a polyp from the first edge image 212a. According to claim 1,
The first normal endoscopic image 211a is
When the inside of the human body is taken with an endoscope, it is an image containing polyps,
The second normal endoscopic image 211b is
When the inside of the human body is taken with an endoscope, it is an image that does not include polyps,
The abnormal endoscopic image 215 is
An apparatus for generating a synthetic endoscopic image using a GAN, characterized in that an image containing a malignant polyp when photographing the inside of a human body with an endoscope. According to claim 1,
The constructor 310 is
generating a similar endoscopic image 214 similar to the first normal endoscopic image 211a from the first edge image 212a and the polyp mask 213;
The classifier 320 is
determining whether the first normal endoscopic image 211a and the similar endoscopic image 214 generated by the creator 310 are authentic;
The learning unit 400
The classifier to receive the first normal endoscopic image 211a and the similar endoscopic image 214, classify the first normal endoscopic image 211a as real, and classify the similar endoscopic image 214 as fake a classifier learning unit 410 for learning 320; and
A generator that trains the generator 310 to generate a similar endoscopic image 214 similar to the first normal endoscopic image 211a so that the classifier 320 can truly classify the similar endoscopic image 214 An apparatus for generating synthetic endoscopic images using GAN, characterized in that it includes a learning unit 420. According to claim 3,
The constructor 310 is
After the sequential iterative learning of the classifier 320 and generator 310 is completed, a malignant polyp is included by randomly selecting and synthesizing one of the plurality of polyp masks 213 in the second edge image 212b. An apparatus for generating a synthetic endoscopic image using GAN, characterized in that for generating a synthesized endoscopic image 216. According to claim 1,
The dataset management unit 200
Further comprising a synthesized image verification unit 240 for verifying the reliability of the synthesized endoscopic image 216,
The composite image verification unit 240
A classifier model 241 that receives an abnormal endoscopic image 215 and a synthetic endoscopic image 216 including malignant polyps of the same type as the abnormal endoscopic image 215 and outputs a predetermined number of vectors, respectively,
The composite image verification unit 240
Using a dimensionality reduction algorithm, a certain number of vectors for the ideal endoscope image 215 and the synthesized endoscope image 216 are input and expressed as a two-dimensional graph, respectively. . According to claim 5,
The composite image verification unit 240
When the distribution of the 2D graph of the synthesized endoscope image 216 overlaps with the distribution of the 2D graph of the ideal endoscope image 215 beyond a preset range, the synthesized endoscope image 216 is a reliable endoscope image. Synthetic endoscopic image generation apparatus using GAN, characterized in that classified into. A first step of generating, by the edge extraction unit 220, a plurality of first edge images 212a obtained by extracting a plurality of characteristic lines from the plurality of first normal endoscopic images 211a, respectively, using an edge detection technique;
a second step of detecting, by the polyp detection unit 230, a plurality of polyp masks 213 each including a polyp from the plurality of first edge images 212a generated by the edge extraction unit 220;
A third step in which the learning unit 400 learns the generator 310 and the classifier 320 based on the GAN;
a fourth step of generating, by the edge extraction unit 220, a plurality of second edge images 212b obtained by extracting a plurality of characteristic lines from the plurality of second normal endoscopic images 211b, respectively, using an edge detection technique;
The creator 310 selects and combines one of the plurality of polyp masks 213 with the plurality of second edge images 212 to generate a plurality of synthesized endoscopic images 216 including malignant polyps, respectively. The fifth step of doing; and
A synthetic endoscopic image generation method using GAN, characterized in that it includes a sixth step of verifying the reliability of the plurality of synthetic endoscopic images 216 by the synthetic image verification unit 240. According to claim 7,
The third step is
From the first edge image 212a generated by the generator 310 from the edge extraction unit 220 and the polyp mask 213 generated from the polyp detection unit 230, a first normal endoscopic image 211a similar to a 3-1st step of generating a similar endoscopic image 214;
The classifier learning unit 410 receives the first normal endoscopy image 211a and the similar endoscopy image 214 generated by the generator 310, and classifies the first normal endoscopy image 211a as real. 3-2 step of training the classifier 320 to classify the similar endoscopic image 214 as fake;
The classifier 320 receives the first normal endoscopic image 211a and the similar endoscopic image 214 and determines whether the first normal endoscopic image 211a and the similar endoscopic image 214 are authentic Step 3-3;
The similar endoscope similar to the first normal endoscope image 211a so that the generator learning unit 420 can classify the classifier 320 as genuine the similar endoscope image 214 generated by the generator 310 Steps 3-4 of training the generator 310 to generate an image 214; and
and a 3-5 step in which the 3-1 to 3-4 steps are repeated a predetermined number of times for the plurality of first edge images 212a and the polyp mask 213. A synthetic endoscopic image generation method using GAN. According to claim 7,
The sixth step is
The synthesized image verification unit 240 inputs the abnormal endoscopic image 215 and the synthesized endoscopic image 216 including the same type of malignant polyp as the abnormal endoscopic image 215 into the classifier model 241 so that a certain number of A 6-1st step of outputting vectors of , respectively;
The synthesized image verification unit 240 receives a certain number of vectors for the ideal endoscope image 215 and the synthesized endoscope image 216 using a dimensionality reduction algorithm, and expresses them in a two-dimensional graph, respectively. Step 2; and
When the synthesized image verifying unit 240 overlaps the distribution of the two-dimensional graph of the synthesized endoscope image 216 with the distribution of the two-dimensional graph of the abnormal endoscope image 215 by more than a preset range, the synthesized endoscope image A synthetic endoscopic image generation method using GAN, characterized in that it includes a 6-3 step of classifying (216) as a reliable endoscopic image.

Images